Behavioral Testing Python Packages

decishift

Open-source ML behavioral regression testing for comparing decision-system versions, identifying changed actions, attributing shifts, and gating releases with Decision Contracts.

agentassert-abc

Runtime behavioral contracts for AI agents: define YAML contracts, check adapter outputs, and raise explicit contract violations. Agent Behavioral Contracts (ABC).

gauntlet-cli

Community-driven behavioral reliability benchmark for LLMs. 231 probes across 19 modules, deterministic scoring, perplexity correlation, layer sensitivity mapping, quant method capture, hardware-stratified community rankings. Every test contributes to the community dataset.

agentdelta

Diff and regression-detect LLM agent execution traces

agentgate-eval

E2E behavioral testing for AI agents — grounded in 24 research papers. pip install agentgate-eval

vlm-prior-probe

Measure when a vision-language model follows visual evidence versus learned priors.

modeldiffx

Model behavioral diffing - compare LLM outputs across versions, detect regressions.

specterqa

AI persona-based behavioral testing for web apps. No test scripts. YAML-configured. Vision-powered.

trajex

AI agent behavioral testing — learns what correct looks like, catches deviations automatically. Zero API keys needed.