Pushing LLMs over the Edge: Exploring the Limits of the Possible by Mario Fusco, Lize Raes

Using multiple specialized AI agents for different tasks (e.g., product selection, cart management) reduces hallucination risks compared to using a single agent

State machines help control LLM behavior and make interactions more reliable by defining clear transitions between states and limiting agent capabilities

GPT-4 shows significantly better security and reliability compared to GPT-3.5, especially regarding prompt injection resistance and following instructions

Local LLM models (like Ollama) can work but are less reliable with tool calls and complex interactions compared to cloud-based models

Adding guardrails and input/output validation is crucial for customer-facing AI applications to prevent malicious behavior and unexpected responses

Temperature settings affect LLM behavior - lower temperatures make responses more consistent but less creative, while higher temperatures increase creativity but may lead to unexpected results

Virtual threads are beneficial for handling multiple LLM calls in parallel, especially when dealing with operations that take 1-2 seconds each

The EU AI Act categorizes AI applications into risk levels, with chatbots requiring disclaimers and high-risk applications needing conformity assessments

Using human-in-the-loop approaches and confirmation steps helps maintain control over critical operations like order placement

Retrieval Augmented Generation (RAG) can be used to enhance LLM capabilities by providing access to external data sources and tools