Confidence Intervals Python Packages

mapie

A scikit-learn-compatible library for estimating prediction intervals and controlling risks, based on conformal predictions.

crepes

Python package for conformal prediction

resample

Randomization-based inference in Python

duvida

Calculating exact and approximate confidence and information metrics for differentiable functions.

hypothesize

Robust statistics in Python

agentrial

Statistical evaluation framework for AI agents - pytest for agent trajectories

confinterval

A lightweight Python toolkit for computing confidence intervals for means, medians, percentiles, and proportions.

doubt

Bringing back uncertainty to machine learning.

binoculars

Confidence intervals for binomial proportions.

ci-methods-analyser

Analyse efficacy of your own confidence interval (CI) methods

kernel-quantile-regression

Kernel quantile regression

foostrap

Fast bootstrap resampling in Python

evalkit-research

Rigorous LLM evaluation: bootstrap CIs, significance testing, automated auditing

strapping

A fast implementation of bootstrapping supporting multi-columns data.

confidence-planner

A Python library for estimating confidence intervals around accuracy and sample sizes for classification experiments.

bsci

Constructing frequentist confidence intervals using the statistical bootstrap

pyellipse

Tools for creating and analyzing confidence ellipses, including Hotelling's T-squared ellipses for multivariate statistical analysis and data visualization.

feerci

FEERCI: A Package for Fast non-parametric confidence intervals for Equal Error Rates

robust-statistics-simulator

Interactive simulations that demonstrate properties of robust statistics