Ai Evals Python Packages

calibrate-sdk

Python SDK for Calibrate's API

pixie-qa

Agent skill for AI agent development

agent-trajectory-diff

Compare AI agent execution paths (trajectories) side-by-side. Detect trajectory drift, redundant tool loops, and resource regressions in CI/CD.

agent-trust-plane

Authorization and security regression testing for AI agents and MCP tools.

ghostrun

pytest for AI Apps — Fast, free, offline testing and self-improving prompts for Python developers.

openjury

An sdk and framework for evaluating and comparing multiple model outputs using configurable LLM-based jurors

falsify-inspect

PRML pre-registration adapter for Inspect AI eval logs. Listed on the UK AI Safety Institute's Inspect extensions page. MIT.