Regression Detection Python Packages

calibrax

JAX benchmarking, profiling and evaluation metrics for Flax NNX: a registry of pure-function metrics (regression, classification, calibration, uncertainty, forecasting, generative, image, text, audio, graph, fairness), XLA FLOP counting, roofline analysis, GPU and energy monitoring, regression detection, publication exports, W&B and MLflow.

kalibra

Statistical regression detection and CI quality gates for AI agents

llmtestr

A new package that helps developers integration-test AI and LLM applications by validating structured outputs. It takes a user's test scenario or prompt as input, sends it to an LLM, and uses pattern

nf-runinsights-dashboard

Nextflow plugin for cross-run benchmarking. Records per-process metrics for every run into a local or shared history store, compares runs against their own history, and flags regressions with likely causes, over-provisioned resources, and retries. Includes a zero-dependency dashboard, an MCP server for AI assistants, and opt-in AI narration.

pytest-agent-health

Catch silent agent failures in CI. Pytest plugin with regression detection for LLM agents.