Mirror, mirror: LLMs and the illusion of humanity - Jodie Burchell - NDC Oslo 2024

LLMs learn language through statistical pattern recognition from massive text datasets, but lack true understanding or grounding in the real world like humans have

Claims of LLM sentience and human-like intelligence have been greatly overstated - they can produce convincing outputs but lack coherent world models and integrated self-awareness

LLMs use distributional semantics (learning word meaning from context) rather than denotational semantics (grounding meaning in real-world experiences) which limits their true understanding

Current benchmarks focusing on task-specific performance are misleading - they don’t capture the system’s actual intelligence or ability to generalize broadly

Prompt injection attacks and jailbreaking demonstrate how LLMs lack robust understanding and can be manipulated by exploiting their pattern-matching nature

The “Kaggle Effect” shows how models can appear more capable than they are by memorizing training data rather than developing true generalization abilities

LLMs can encode syntactic information and some higher-order relationships but still struggle with coherence and consistency across different contexts

We should focus on what LLMs can actually do well within their limitations rather than projecting human-like qualities onto them

Current LLMs are still far from artificial general intelligence (AGI) - they operate through sophisticated pattern matching rather than human-like reasoning

Better assessment methods are needed that focus on broad generalization abilities rather than narrow task performance to avoid overestimating LLM capabilities