Regression Testing Python Packages

pytest-accept

A pytest plugin for automatically updating doctest outputs

testopshub

基于 pytest 的测试可信度治理实践：批量执行、终端输出结构化解析、Allure 桥接、CI 六步门禁（测试数下限/增量类型检查/依赖分级）拦截删测试换绿；跨平台进程树清理与 Redis 队列可靠性语义。支持 HTTP 与串口/Telnet（模拟验证）协议用例。

reframe-hpc

A powerful Python framework for writing and running portable regression tests and benchmarks for HPC systems.

pytest-ranking

A Pytest plugin for faster test failure detection

presence-audit

Given a declaration and a capture, say which declared points are reading, present but not reading, or absent. Domain-free: the vocabulary comes from a vertical you register, and it ships none.

jevnav

Page truth for browser agents — and decisions that replay, test and audit. Jev picks the element, risky actions are gated, every run replays offline in CI.

wickra-strategy-ci

Jest for trading strategies: golden-pin the BacktestReport, catch regressions in CI, property-test against fuzzed data — usable in all 10 languages plus a composite GitHub Action.

adk-tracegauge

See what your Google ADK agent costs (per-invocation USD from token usage) and fail CI when it gets more expensive.

evalarc

Review model and agent regressions behind better scores. Compare every changed check, inspect original model outputs and tool traces, and reproduce the decision offline.

gauntlet-robotics

Robustness evaluation harness for robot manipulation policies (VLA): perturbation sweeps, failure clustering, reproducible reports.

rhesis

The collaboration layer for AI teams: domain experts annotate and review agent behavior, engineers improve the agent from what they find.

evalview

Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.

orbit-eval

You have a robot. orbit tells you what to do next, and how you will know it worked. Zero-dependency CLI for LeRobot datasets and evaluations: what the recording got wrong, what it never covered, whether a change survived the noise.

decimalai

Python SDK for agent evals and skill routing — measure a skill's real lift before you trust it

rhesis-sdk

The collaboration layer for AI teams: domain experts annotate and review agent behavior, engineers improve the agent from what they find.

trace2eval-cli

属于 AI 的错题本 · Turn production LLM traces into a regression eval set

gauntlet-guard

Your AI “improved.” Did it get smarter—or memorize the test? Find out before you trust the score.

mcp-rig

Deterministic, CI-ready testing for MCP servers: YAML suites, state checks, snapshots, and JUnit — for local stdio and remote HTTP servers.

agentverity

Qualify repeated categorical AI-agent evidence before it becomes a regression reference

assay-server

Behavioral regression testing for AI apps and agents: pytest plugin, behavior diff, CI gate, and document extraction metrics

modelpin

Dependabot for AI models — replay your app's scenarios across a model migration and catch real behavioral regressions before they ship. Low false-positive by design. CLI + GitHub Action. Open-core, Apache-2.0.

wup

WUP (What's Up) - Intelligent file watcher for regression testing in large projects

assay-evals

Behavioral regression testing for AI apps and agents: pytest plugin, behavior diff, CI gate, and document extraction metrics

maida-ai

Pre-merge behavioral regression gate for AI agents. Check agent changes, inspect regressions, and block broken behavior before merge. Local-first.