Pattaniyil, Ravi, & Zengin - Using LLMs to improve your Search Engine | PyData Global 2023

LLMs can help improve e-commerce search by understanding query intent, item attributes, and relevancy beyond simple token matching

Core e-commerce search challenges include:

• Ambiguous or unclear user queries
• Keyword stuffing in product titles
• Matching query intent with item attributes
• Determining query-item relevance
• Managing vocabulary gaps between queries and items

Different prompting approaches can be used with LLMs:

• Zero-shot for basic tasks
• Few-shot with examples for better accuracy
• Structured output formatting for consistent results
• Role-based prompting for domain expertise

Model considerations important for implementation:

• Proprietary models (GPT-4, Palm) vs open source (Llama)
• Model size tradeoffs (7B vs 70B parameters)
• Latency and cost implications
• Fine-tuning vs prompt engineering

Key implementation strategies:

• Cache results to improve latency
• Use structured prompts for consistent output
• Consider hybrid approaches combining search engines with LLMs
• Extract attributes on both query and item sides
• Validate relevancy with human-annotated data

Production deployment considerations:

• Resource requirements for model hosting
• Latency requirements for real-time search
• Cost of API calls for hosted models
• Data quality for fine-tuning
• Evaluation metrics for relevance