Distribution Shift Python Packages

frouros

Frouros: an open-source Python library for drift detection in machine learning systems.

benchgap

Measure how much of a machine-learning benchmark score survives deployment. Grouped bootstrap, frozen thresholds, gap attribution.

population-resemblance

Python library for monitoring categorical population drift with the Population Resemblance Statistic (PRS), sample-size-aware thresholds, PSI and discrete KS benchmarks, and Monte Carlo simulation.

sustaingym

Reinforcement Learning Environments for Sustainable Energy Systems

whyshift

A python package providing a benchmark with various specified distribution shift patterns.

ocpdet

📦 A Python package for online changepoint detection, implementing state-of-the-art algorithms and a novel approach based on neural networks.

iscan-dag

A Python 3 package for identifying distribution shifts (a.k.a feature-shifts) between datasets. Official implementation of the paper: "iSCAN: Identifying Causal Mechanism Shifts among Nonlinear Additive Noise Models".

frd-score

Official implementation of the Fréchet Radiomic Distance

insurance-covariate-shift

Density ratio correction for insurance pricing book shifts — CatBoost/RuLSIF/KLIEP, LR-QR conformal, FCA SUP 15.3 diagnostics

biquality-learn

A Python Library for Biquality Learning

incerto

A comprehensive library for uncertainty quantification in machine learning.

greatx

A graph reliability toolbox based on PyTorch and PyTorch Geometric (PyG).

graphwar

A graph reliability toolbox based on PyTorch and PyTorch Geometric (PyG).

graph-ood

GOOD: A Graph Out-of-Distribution Benchmark [NeurIPS 2022 Datasets and Benchmarks]

winference

Calibrating pairwise rankings with accomodations for non-transitivity