Evals Python Packages

logfire

AI observability platform for production LLM and agent systems.

harbor-rewardkit

Framework for evaluating and improving agents

arize-phoenix-otel

AI Observability & Evaluation

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

arize-phoenix

AI Observability & Evaluation

arize-phoenix-sqlean

AI Observability & Evaluation

agentops

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI

hud-python

RL environments + evals for AI agents. Define once, train anything.

hud

RL environments + evals for AI agents. Define once, train anything.

skill-lens

An Agent Skill is a prompt. Prompts regress. skill-lens turns "I think this SKILL.md got better" into a score, a report, and an exit code your pipeline can gate on.

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

benchflow

Research infra for creating RL environments, post-training, and evals.

harbor-langsmith

Framework for evaluating and improving agents

trulens-connectors-snowflake

Evaluation and Tracking for LLM Experiments and AI Agents

evalica

Evalica, your favourite evaluation toolkit

trulens-providers-openai

Evaluation and Tracking for LLM Experiments and AI Agents

overmind

The platform for continuously improving AI agents.

jevals

Agent evals and guardrails as Jev decisions: one request per trace, a fraction of a cent, fast enough for the agent loop. Runs locally with Kev or Laya.