Deterministic Testing Python Packages

hermes-jailbench

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

agent-observability-trace-cli

Deterministic record/replay for LLM agent HTTP traffic. Replay a failed run offline in under 1ms, zero API cost.

agentrunproof

Reproduce OpenAI Agents SDK runtime bugs without API keys — real Runner, scripted models, verifiable certificates.

tracecore

Deterministic runtime for agent evaluation

hermes-jailbreak-bench

Zero-LLM deterministic jailbreak regression benchmark: runs a repeatable battery of known-pattern attacks against an LLM endpoint and scores refusal/partial/compliance across runs. Catch when a model or prompt update silently got weaker. Single-turn, responsible-use. pip install hermes-jailbench

qiro-analyzer

Python CLI for environmental marketing-claim risk review with structured outputs, image support, and offline mock testing.

agent-observability-trace

Deterministic record/replay for LLM agent HTTP traffic. Replay a failed run offline in under 1ms, zero API cost.