Marco Gorelli - Polars and time zones: everything you need to know | PyData Global 2023

Learn how to handle time zones in Polars, from basic concepts to best practices. Compare calendar vs fixed durations, explore UTC storage, and master time zone operations.

Key takeaways
  • Polars handles time zones through the Rust library cronitz, providing better support than pandas (which uses pytz) for dates beyond 2038

  • Datetimes in Polars have two key components:

    • Time unit (smallest amount of time the datatype can represent)
    • Time zone (can be none or from the time zone database)
  • Calendar duration (1d) vs fixed duration (24h):

    • Calendar days can vary in length due to daylight savings
    • Fixed duration is always exactly 24 hours
    • Use calendar durations for date-based logic
  • Best practices for time zone handling:

    • Store datetimes in UTC
    • Convert to local time zones only when needed for logic/display
    • Never handle time zones manually
    • Use proper time zone names (e.g. ‘Asia/Kathmandu’) instead of abbreviations
  • Two main time zone operations in Polars:

    • convert_timezone(): Shows what time it is in another location
    • replace_timezone(): Changes the time zone while keeping the local time
  • When reading CSVs with datetime data:

    • Use try_parse_dates parameter for better parsing
    • Polars won’t automatically infer time zones - you must specify them
    • PyArrow strings provide better performance than regular strings
  • Date boundaries and calculations can be affected by:

    • Daylight savings transitions
    • Countries changing time zones
    • Different daylight savings rules between regions
    • Local days potentially having 23, 24, or 25 hours