Polars and Time Series: what it can do, and how to overcome any limitation

-

Learn how Polars excels at time series data processing, offering superior performance and syntax. Explore its key features and discover how to extend functionality using Rust plugins.

Key takeaways
  • Polars is gaining significant traction in both open source and production environments, with major companies adopting it for data processing

  • Key advantages include superior performance (up to 150x speedups reported) and improved syntax compared to pandas, especially for group-by operations and expressions

  • Polars expressions are a fundamental concept - they don’t have a value until given input and called within a dataframe context

  • The library offers strong time series support including:

    • Business days handling with custom weekdays/holidays
    • Upsampling/downsampling
    • Rolling windows and exponentially weighted moving averages
  • When Polars functionality isn’t sufficient, it can be extended through Rust plugins:

    • Basic Rust syntax knowledge is enough to get started
    • Plugin development typically takes less than 5 minutes
    • Cookie cutter templates are available
  • Time-aware operations are supported throughout, including reading CSV files with automatic date parsing

  • For new projects, Polars is recommended over pandas due to:

    • Better ergonomics
    • More intuitive syntax
    • Superior performance
    • Strong type system
  • Libraries like scikit-learn and statsforecast now support Polars integration

  • The list data type is considered one of Polars’ strongest features by many users

  • Companies are seeing significant cloud cost savings by switching to Polars for data processing