Cordier & Lacombe - Boosting AI Reliability: Uncertainty Quantification with MAPIE

MAPIE is a framework for uncertainty quantification in AI that works with any ML model and requires minimal assumptions

Key features include:

• Distribution-free coverage guarantees
• Model-agnostic design - works with any ML algorithm
• Supports classification, regression and time series
• Built-in conformal prediction capabilities
• Handles imbalanced datasets through Mondrian strategy

Provides meaningful uncertainty intervals/prediction sets with guaranteed coverage levels (e.g. 95% confidence)

Helps detect:

• Out of distribution samples
• Model degradation
• Data drift
• When model predictions cannot be trusted

Main use cases:

• Risk control in sensitive applications
• Regulatory compliance
• Safety-critical systems (autonomous vehicles, medical)
• Time series forecasting with changing patterns

Implementation requires:

• Training data
• Calibration data
• Test data
• Desired confidence level

Limitations:

• Requires calibration dataset
• More conservative predictions with smaller calibration sets
• May return infinite intervals in some cases
• Coverage guarantees are in expectation

Open source library with growing community contributions and documentation