PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization

Mouhcine Mendil (1) , Luca Mossina (1) , David Vigouroux (1)

1 IRT Saint Exupéry - Institut de Recherche Technologique

Predictive UNcertainty Calibration and Conformalization (PUNCC) is an open-source Python library integrating a collection of state-of-the-art Conformal Prediction (CP) algorithms and related techniques for regression and classification problems. This package aims to make conformal procedures accessible to non-experts using a simple and intuitive implementation. It is compatible with scikit-learn, PyTorch and TensorFlow and easily extensible to other prediction toolkits. PUNCC also comes with a low-level API that provides a unified workflow in a pythonic environment to build, combine and run inductive CP algorithms. It offers generic structures and consistent interfaces to design customized non-conformity scores, data partition schemes, and methods for constructing prediction sets. In this paper, we present the design of our library and demonstrate its use with various CP procedures, Machine Learning (ML) problems and models from different ML libraries. Source code, documentation and demos are available at https://github.com/deel-ai/puncc.

Mots clés: Uncertainty Quantification; Conformal prediction; Python; Library; Regression; Classification; Deep leaning

Domaines: Machine Learning [stat.ML]; Logiciel mathématique [cs.MS]

Mouhcine Mendil, Luca Mossina, David Vigouroux. PUNCC: a Python Library for Predictive Uncertainty Calibration and Conformalization. Conformal and Probabilistic Prediction with Applications, Sep 2023, Limasol, Cyprus. ⟨hal-04560192⟩