Evals Python Packages

logfire

AI observability platform for production LLM and agent systems.

harbor-rewardkit

Framework for evaluating and improving agents

arize-phoenix

AI Observability & Evaluation

arize-phoenix-otel

AI Observability & Evaluation

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

hud-python

RL environments + evals for AI agents. Define once, train anything.

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

verel

Verified agents — nothing is "done" until a grader returns a verdict. Eyes (AgentVision) + verdict bus + compounding memory + a fleet + agent-built tooling + agent-run CI/CD.

evalica

Evalica, your favourite evaluation toolkit

trulens-connectors-snowflake

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-providers-openai

Evaluation and Tracking for LLM Experiments and AI Agents

selectools

Production-ready Python framework for AI agents with built-in guardrails, audit logging, cost tracking, and hybrid RAG. Supports OpenAI, Anthropic, Gemini, Ollama. By NichevLabs.

brein-mcp

🧠 it remembers what your company forgets - an brain that measures itself and patches its own gaps.

trulens-apps-langchain

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-providers-litellm

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-providers-cortex

Evaluation and Tracking for LLM Experiments and AI Agents

multivon-eval

Practical LLM evaluation for teams that ship to production. Deterministic + LLM-as-judge evaluators, dataset support, CI/CD integration.