Agent Evaluation Python Packages

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

agentenv-framework-protocol

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

agentenv-framework

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

giskard

🐢 Open-Source Evaluation & Testing library for LLM Agents

trulens-connectors-snowflake

Evaluation and Tracking for LLM Experiments and AI Agents

coder-eval

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.

trulens-providers-openai

Evaluation and Tracking for LLM Experiments and AI Agents

tradefloor

A simulated stock market for testing trading strategies and AI agents. Orders move prices, any market can be forked, and every run reproduces exactly.

isnad

Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines

trulens-apps-langchain

Evaluation and Tracking for LLM Experiments and AI Agents

calibrate-sdk

Python SDK for Calibrate's API

trulens-providers-litellm

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-apps-llamaindex

Evaluation and Tracking for LLM Experiments and AI Agents

evalseal

Git-style receipts for LLM evals: measure variance, capture provenance, detect flaky verdicts, and verify results in CI

eio-agents

Evaluation Intelligence Ontology (EIO) and Portable Evaluation Records for AI-agent evaluation: convert, validate, verify, explain.

abslang

Agent Behavior Specification — describe, run, and evaluate AI agent interactions from the command line

any-agent

A single interface to use and evaluate different agent frameworks

evalarc

Review model and agent regressions behind better scores. Compare every changed check, inspect original model outputs and tool traces, and reproduce the decision offline.

trulens-providers-cortex

Evaluation and Tracking for LLM Experiments and AI Agents