Harmonizing Code and Melody: making electronical music in a matter of speaking! by Jan Van Wassenhov

Music Agent is a project that combines AI and coding to create electronic music using Sonic Pi, LLMs, and autonomous agents

The system uses multiple specialized agents (artist, composer, songwriter, reviewer) that work together through conversation chains to create music

Core technology stack includes:

• Sonic Pi for music generation
• OpenAI APIs for agent communication
• Python for core functionality
• Ruby code for Sonic Pi integration

The workflow follows phases:

• Conceptualization/theme development
• Composition and arrangement
• Code generation and review
• Recording/mixing
• Cover art and booklet creation

Output includes:

• Sonic Pi Ruby code file
• WAV recording
• Album cover art
• Song booklet with lyrics
• Complete documentation

Key limitations:

• Primarily electronic/digital sound
• Simple song structures
• No singing/vocals
• Quality not matching professional musicians
• Token limitations in LLM conversations

Users maintain creative control by:

• Providing initial direction
• Reviewing and modifying code
• Adjusting arrangements
• Multiple iteration cycles
• Configurable agent roles and chains

Project emphasizes human creativity and guidance rather than full automation

Available as open source on GitHub with flexible configuration options

Built for sustainability with lower energy footprint compared to text-to-speech models