AI Assistance Beyond Code: What Do We Need to Make it Work? • Birgitta Böckeler • GOTO 2024

AI assistance for software delivery goes beyond just code generation - it can help with requirements, architecture, threat modeling, and other knowledge work

Key challenges in making AI assistance work:

• Need proper context orchestration and knowledge curation
• Must carefully consider what knowledge to inject into prompts
• Balance between structure and flexibility in interactions
• Different interaction types needed for different tasks/users

Knowledge orchestration is crucial:

• Pull in context from wikis, issue trackers, code bases
• Use retrieval augmented generation (RAG)
• Curate and structure knowledge intentionally
• Consider scope and relevance of injected knowledge

User experience considerations:

• Allow different interaction patterns (chat, structured input, etc)
• Give users ways to override and adapt suggestions
• Provide step-by-step guidance rather than just large outputs
• Consider experience levels and domain knowledge of users

Shared prompts and practices:

• Codify organizational practices in prompts
• Share prompts between teams to spread knowledge
• Balance between guiding users and maintaining flexibility
• Include domain terminology and context

Practical implementation tips:

• Start small and focused rather than connecting everything
• Invest in knowledge curation and prompt engineering
• Monitor and iterate based on actual usage
• Keep humans in the loop for validation

Areas for potential impact:

• Breaking down epics into stories
• Threat modeling and security analysis
• Architecture decision documentation
• Domain knowledge sharing and learning
• Cross-functional requirements discovery

Success requires:

• Mindset shift from traditional software tools
• Investment in knowledge orchestration
• Thoughtful user experience design
• Balance between structure and flexibility
• Understanding of organizational context

AI assistance should:

• Help think through problems, not just generate artifacts
• Promote good practices while remaining flexible
• Support learning and knowledge sharing
• Complement rather than replace human expertise