How to add genuinely useful AI to your webapp (not just chatbots) - Steve Sanderson

AI can provide genuine value beyond chatbots through semantic search, summarization, intelligent form-filling, and workflow automation

Key patterns for integrating AI include:

• Semantic search for finding relevant content without exact matches
• Zero-shot classification for categorizing text
• Retrieval Augmented Generation (RAG) for grounding AI responses in real data
• Smart UI components that enhance existing forms and inputs

Testing and evaluation are critical:

• Use end-to-end testing with deterministic results
• Cache language model responses during tests
• Measure accuracy, relevance, and groundedness of AI outputs
• Compare against known ground truth data

Development best practices:

• Start with generated test data before using production data
• Use smaller, local models during development
• Implement proper evaluation metrics from the beginning
• Focus on genuine user value rather than novelty

Technical implementation tips:

• Consider using Semantic Kernel for .NET integration
• Aspire can help orchestrate AI services
• Python services can be integrated when needed
• Cache embeddings and vector searches for performance
• Split large text into appropriate chunks for processing

The goal should be making users more productive by:

• Automating mundane tasks
• Surfacing relevant information quickly
• Enhancing existing workflows rather than replacing them
• Letting humans focus on creative and strategic work