Talks - Sydney Runkle: Pydantic Power-up: Performance Tips for Lightning-Fast Python Applications

Learn powerful techniques to optimize Pydantic performance in Python apps. From JSON validation tricks to tagged unions, discover how to speed up your data handling.

Key takeaways
  • Use model_validate_json() instead of combining model_validate() with json.loads() for better performance when validating JSON data

  • Utilize tagged/discriminated unions instead of standard unions to avoid validating against all possible types in a union

  • Leverage callable discriminators for complex discrimination behavior when simple string discriminators aren’t sufficient

  • Use specific type hints (like list) instead of more general ones (like Sequence) for better performance

  • Take advantage of lazy evaluation - attributes aren’t materialized until requested, reducing unnecessary object creation

  • Pydantic V2’s performance improvements come from using Rust for core validation logic instead of Python/Cython

  • Avoid initializing type adapters repeatedly in loops - initialize once and reuse

  • For nested models with unions, tagged unions are particularly beneficial as they skip unnecessary validation steps

  • Future performance improvements will focus on:

    • SIMD and Jitter for JSON parsing
    • Keeping data in Rust longer
    • Avoiding Python object materialization where possible
  • Schema building speedups are being implemented to improve handling of recursive and nested model structures