Agent Testing Python Packages

langwatch-scenario

Agentic testing for agentic codebases

swarm-test

Chaos engineering & reliability testing for multi-agent AI systems

proofagent-harness

Open-source test harness for AI agents. Stress-test production agents with adversarial multi-turn scenarios in CI

evalcraft

Generate deterministic pytest tests for your AI agents from one real run, then replay them in CI for $0. Fast, flake-free agent testing.

knowlytix-core

Geometric Memory Systems — core geometry and LLM-layer foundations

agentbreak

Chaos proxy for testing LLM agent resilience — inject faults like latency, errors, and bad responses. Supports OpenAI, Anthropic, and MCP.

knowlytix-benchmark

Benchmark for structured information retrieval from financial documents using graph-verifiable questions

knowlytix-knowledge

GMS-powered document ingestion, query, and learning — DocGMS

knowlytix-harness

GMS-Harness — provider-agnostic DOE-driven black-box testing platform for LLM agents

agentos-platform

"The Operating System for AI Agents. Build, Test, Deploy, Monitor, Govern."

agentverify

Deterministic testing for AI agents. Assert agent actions, not vibes.

tracecore

Deterministic runtime for agent evaluation

agentassay

Formal regression testing for non-deterministic AI agent workflows

mcp-wallfacer

Runtime testing harness for MCP servers: fuzz tools, validate schemas, run YAML invariants and multi-step sequences, ship SARIF to CI. 17 embedded packs · stdio + HTTP · 5 install paths.

ghostlab

Local end-to-end testing lab for any MCP server: coding agents role-play real users, drive your tools over multiple turns, score runs, and render/interact with MCP Apps ui:// widgets.

replayd

Turn failed AI agent runs into replayable regression tests

volnix

Programmable worlds for artificial intelligence

toolcallcheck

Deterministic Python testing for tool-using agents. Mock MCP tools, assert exact tool calls and trajectories, verify headers, and run offline in CI.

soapbucket-mcptest

Placeholder. Full release coming soon. See https://mcptest.sh

tddf

Behaviour regression tests for AI agents — deterministic, local-first, framework-agnostic