Model Validation Python Packages

deepchecks

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.

scores

scores: Metrics for the verification, evaluation and optimisation of forecasts, predictions or models.

decishift

Open-source ML behavioral regression testing for comparing decision-system versions, identifying changed actions, attributing shifts, and gating releases with Decision Contracts.

dcc-mcp-openscad

OpenSCAD MCP adapter for AI agents to inspect and validate parametric models, export geometry and render PNG previews through bounded native CLI operations in the DCC-MCP ecosystem.

omega-lock

Catch overfitting before you ship it: a walk-forward / held-out gate for any optimizer (Optuna, sweeps, prompt search). The highest score isn't deployable until it survives data it never saw. CLI + Python.

mlx-model-doctor

Validate an MLX / Hugging Face model repository before you load it.

stable-cart

Bootstrap audits of prediction instability for fitted models and model-building pipelines

targetleak

Find the data leakage that made your test score a lie - which column, why, and how to fix it.

valiron

Generate regulatory-grade AI validation evidence in minutes, not months.

modeltest

Unit tests for machine learning models — quality, robustness, fairness and data-invariant contracts that run in CI/CD, like pytest for your model.

gp3mlpy

Governance-first, leakage-resistant predictive modelling for Gazepoint research workflows in Python.

pyforecasttools

Forecast Verification/Validation Tools in Python

dataleaks

Detect, explain, and prevent data leakage across machine learning datasets and workflows.

tanml

Automated validation toolkit for tabular ML models in finance and regulated domains.

acteval-insurance

Model-agnostic evaluation, uncertainty and reporting for actuarial predictive models.

insurance-governance

Model governance for insurance pricing — PRA SS1/23 validation reports, model risk management, risk tier scoring

structural-fuzzing

Structural fuzzing framework for parameterized model validation

ionis-validate

IONIS V20 — HF Propagation Model Validation Suite

holdout-first

Validation harness that fits on a small slice and demands survival on a large held-out remainder, across instruments and periods, under a hard parameter budget.

insurance-validation

PRA SS1/23 compliant model validation reports for insurance pricing — Gini CI, A/E CI, double-lift, fairness, HTML/JSON

psi-inference

Critical values and p-values for population stability testing: Yurdakul-Naranjo PSI, the Population Resemblance Statistic, and the effect-size and overlapping tests. Pure numpy and scipy.

wandb-mv

Model versioning using Weight&Biases with Python.

ruvpy

Reference implementation of the Relative Utility Value metric for forecast value assessment

deepchecks-nlp

Deepchecks: Tests for Continuous Validation of ML Models & Data. Deepchecks is a holistic open-source solution for all of your AI & ML validation needs, enabling to thoroughly test your data and models from research to production.