RAG for a medical company: the technical and product challenges [PyCon DE & PyData Berlin 2024]

RAG implementation for a French medical company (Vidal) helped reduce drug information search time from 2-3 minutes to 50 seconds for doctors

Key technical improvements included:

• Making sources selectable by users
• Adding INN (International Non-proprietary Name) to question rephrasing
• Proper document chunking based on section boundaries
• Using GPT-4 for question rephrasing and answer generation
• Implementing yellow highlighting for relevant answer passages

Performance metrics improved to:

• 88% acceptable answers
• 12% “don’t know” responses
• 0% incorrect answers after 4 weeks of iteration

Major challenges addressed:

• AI reliability and hallucination mitigation
• Long document handling (40+ page monographs)
• Proper source attribution
• Mapping between brand names and generic drug names
• Table handling in documents

Product design considerations:

• Building doctor trust through source transparency
• Maintaining familiar search patterns for medical professionals
• Quick response times (under 1 minute)
• Integration with existing medical document workflows
• Clear display of source documents and relevant passages

The system architecture uses:

• Vector database for document storage
• Embedding model for document retrieval
• OpenAI functions for structured output
• Chainlit for frontend interface
• Custom prompt engineering for medical context

Evaluation approach included:

• Dataset of 100 real healthcare professional questions
• Manual validation by medical experts
• Automated metrics for retrieval performance
• Iterative improvement based on user feedback