Talks - Sydney Runkle: Pydantic Power-up: Performance Tips for Lightning-Fast Python Applications

Sydney Runkle

Learn powerful techniques to optimize Pydantic performance in Python apps. From JSON validation tricks to tagged unions, discover how to speed up your data handling.

Key takeaways
  • Use model_validate_json() instead of combining model_validate() with json.loads() for better performance when validating JSON data

  • Utilize tagged/discriminated unions instead of standard unions to avoid validating against all possible types in a union

  • Leverage callable discriminators for complex discrimination behavior when simple string discriminators aren’t sufficient

  • Use specific type hints (like list) instead of more general ones (like Sequence) for better performance

  • Take advantage of lazy evaluation - attributes aren’t materialized until requested, reducing unnecessary object creation

  • Pydantic V2’s performance improvements come from using Rust for core validation logic instead of Python/Cython

  • Avoid initializing type adapters repeatedly in loops - initialize once and reuse

  • For nested models with unions, tagged unions are particularly beneficial as they skip unnecessary validation steps

  • Future performance improvements will focus on:

    • SIMD and Jitter for JSON parsing
    • Keeping data in Rust longer
    • Avoiding Python object materialization where possible
  • Schema building speedups are being implemented to improve handling of recursive and nested model structures