Luca Baggi - How I used Polars to build built functime, a next gen ML forecasting library

Discover how Luca Baggi built functime, a scalable ML forecasting library using Polars. Learn about global forecasting, automated preprocessing & key design principles.

Key takeaways
  • Functime is a forecasting library built on Polars that enables efficient processing of multiple time series without requiring distributed systems

  • Key advantages of using Polars for forecasting:

    • Multi-threaded query engine
    • Lazy optimizations
    • Apache Arrow format for fast I/O
    • Query optimization leveraging relational database concepts
  • Global forecasting approach allows handling thousands of time series efficiently:

    • Fits one model on the entire dataset instead of individual models
    • Works well for 90% of common forecasting workflows
    • Particularly effective for dozens to hundreds of time series
  • Core features of Functime:

    • Scikit-learn compatible API
    • Built-in diagnostics and visualization tools
    • Automatic timestamp alignment
    • Native support for panel datasets
    • Conformal prediction intervals
  • Design principles:

    • Minimal dependencies for easy deployment
    • Focus on practical scale (hundreds/thousands of series) rather than extreme scale
    • Emphasis on the entire forecasting workflow, not just model accuracy
    • Automated feature engineering and preprocessing
  • Future developments planned:

    • Probabilistic forecasting capabilities
    • Rank-based interval forecasts
    • Enhanced plotting features
    • Integration with FLAML for hyperparameter tuning
    • Improved feature extraction methods