TDD & generative AI - a perfect pairing? by Bouke Nijhuis

TDD and generative AI make an effective pairing - tests serve as clear specifications for AI to generate implementations

Two key purposes of tests in AI-assisted development:

• Input/specifications for the AI to generate code
• Validation that AI-generated code works correctly

Local LLMs vs Cloud LLMs:

• Local models are faster but less capable
• Cloud models produce better results but cost money
• Cloud models don’t have context of local codebase/dependencies

Main challenges when using AI code generation:

• Sometimes produces partial/non-compiling code
• Struggles with less common dependencies and frameworks
• May get stuck or need multiple attempts
• Requires automation to be practical

The test-driven generation workflow:

• Human writes tests first
• AI generates implementation
• Run tests to validate
• If tests fail, feed errors back to AI and retry
• Maximum of 5 retries per implementation attempt

Benefits of this approach:

• Tests prevent biased implementations
• Forces thorough thinking about requirements upfront
• Provides clear validation of AI output
• Works well with pair programming practices
• Can save significant development time when successful

Recommendations for implementation:

• Add clear test messages/documentation
• Start with simple cases before complex ones
• Provide adequate context to the AI
• Be prepared to retry with different prompts/models
• Automate the feedback loop between tests and AI

Future potential improvements:

• IDE plugin integration
• Support for more testing frameworks
• Better handling of dependencies
• Support for multiple test files
• Improved context sharing with AI models