Polars and Time Series: what it can do, and how to overcome any limitation

Learn how Polars excels at time series data processing, offering superior performance and syntax. Explore its key features and discover how to extend functionality using Rust plugins.

Key takeaways
  • Polars is gaining significant traction in both open source and production environments, with major companies adopting it for data processing

  • Key advantages include superior performance (up to 150x speedups reported) and improved syntax compared to pandas, especially for group-by operations and expressions

  • Polars expressions are a fundamental concept - they don’t have a value until given input and called within a dataframe context

  • The library offers strong time series support including:

    • Business days handling with custom weekdays/holidays
    • Upsampling/downsampling
    • Rolling windows and exponentially weighted moving averages
  • When Polars functionality isn’t sufficient, it can be extended through Rust plugins:

    • Basic Rust syntax knowledge is enough to get started
    • Plugin development typically takes less than 5 minutes
    • Cookie cutter templates are available
  • Time-aware operations are supported throughout, including reading CSV files with automatic date parsing

  • For new projects, Polars is recommended over pandas due to:

    • Better ergonomics
    • More intuitive syntax
    • Superior performance
    • Strong type system
  • Libraries like scikit-learn and statsforecast now support Polars integration

  • The list data type is considered one of Polars’ strongest features by many users

  • Companies are seeing significant cloud cost savings by switching to Polars for data processing