Evaluations Python Packages

soma-governance

Deterministic governance and guardrails for autonomous AI coding agents. Features AST verification, evidence-based rule execution, and native MCP support for Antigravity and Claude Code.

railtracks

An agentic framework that helps developers build resilient agentic systems

clastogen

Do your LLM evals catch a broken prompt? Mutation testing for system prompts, in pytest.

log10-io

Python client library for improving your LLM app accuracy

apolien

AI Safety Evaluation Library

railtracks-cli

An agentic framework that helps developers build resilient agentic systems

ecp-runtime

ECP is a standardized interface for orchestrating, auditing, and enforcing authority limits in AI Agent evaluations. It moves evaluation from "brittle Python scripts" to a deterministic infrastructure protocol

ecp-sdk

ECP is a standardized interface for orchestrating, auditing, and enforcing authority limits in AI Agent evaluations. It moves evaluation from "brittle Python scripts" to a deterministic infrastructure protocol

evallint

Audits LLM eval datasets for the flaws that make evaluations silently lie — discrimination failure, near-duplicates, class imbalance.

mandoline

Official Python client for the Mandoline API

evret

Evals framework for Information Retrieval Systems

evaluations

This library implements various metrics (including Kaggle Competition, Medicine) for evaluating ML, DL, AI models, and algorithms. 📐📊📈📉📏

aniket-agentlens-sdk

Open-source observability, security, and evaluation platform for AI agents

helios-observatory-sdk

AI systems observability platform for tracing, evaluating, and debugging LLM applications, agents, and RAG pipelines.