Significance Testing Python Packages

hera-matlab

HERA: Hierarchical-Compensatory, Effect-Size Driven Ranking Algorithm.

deepsig

Enabling easy statistical significance testing for deep neural networks.

hypothesize

Robust statistics in Python

dnn-inference

Significance tests of feature relevance for a black-box learner

abeval

A/B-test statistics for LLM evals: error bars, paired significance tests, sample-size planning

siggate

The vendor-neutral significance gate for AI evals: tells you in CI whether an eval improvement is statistically REAL or within noise. Open-core, powered by deltagate.

honest-eval

Walk-forward split with embargo (no leakage) + paired significance gate with Kish's effective n and a confidence lower bound. Pure Python, no dependencies.

robust-statistics-simulator

Robust statistics in Python

scatterbin

Adjustable binning of scatter plot data

deep-signifcance

Enabling easy statistical significance testing for deep neural networks.