Uncertainty Calibration Python Packages

pymatviz

A toolkit for visualizations in materials informatics.

uncertainty-calibration

Calibration library and code for the paper: Verified Uncertainty Calibration. Ananya Kumar, Percy Liang, Tengyu Ma. NeurIPS 2019 (Spotlight).

uncertainty-toolbox

Uncertainty Toolbox: a Python toolbox for predictive uncertainty quantification, calibration, metrics, and visualization

thetiscore

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

superior-scoring-rules

Scoring rules like the Brier Score (Mean Squared Error, Quadratic Score) and Log Loss (Cross-Entropy, Negative Log-Likelihood, Logarithmic Score) can favor incorrect predictions. To address this limitation, the Probabilistic Brier Score (PBS) and Probabilistic Logarithmic Loss (PLL) have been introduced for probabilistic classifiers.

thetis

Service to examine data processing pipelines (e.g., machine learning or deep learning pipelines) for uncertainty consistency (calibration), fairness, and other safety-relevant aspects.

robust-deep-learning

A project to train your model from scratch or fine-tune a pretrained model using the losses provided in this library to improve out-of-distribution detection and uncertainty estimation performances. Calibrate your model to produce enhanced uncertainty estimations. Detect out-of-distribution data using the defined score type and threshold.

ml-matrics

A toolkit for visualizations in materials informatics.