Llm Evaluation Python Packages

mlflow-skinny

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

mlflow-tracing

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

deepeval

The LLM Evaluation Framework

opik

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

arize-phoenix-otel

AI Observability & Evaluation

arize-phoenix-client

AI Observability & Evaluation

arize-phoenix-evals

AI Observability & Evaluation

arize-phoenix

AI Observability & Evaluation

prompty

Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.

arize-phoenix-sqlean

AI Observability & Evaluation

ouroboros-ai

Agent OS: the agent gets smarter on its own. We just hold the line: Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 14 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.

judgeval

The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.

aig-skill-scan

A full-stack AI Red Teaming platform securing AI ecosystems via Agent Scan, Skills Scan, MCP scan, AI Infra scan and LLM jailbreak evaluation.

garak

the LLM vulnerability scanner

trulens-core

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-feedback

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-otel-semconv

Evaluation and Tracking for LLM Experiments and AI Agents

agentenv-framework-protocol

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

trulens-eval

Evaluation and Tracking for LLM Experiments and AI Agents

trulens-dashboard

Evaluation and Tracking for LLM Experiments and AI Agents

trulens

Evaluation and Tracking for LLM Experiments and AI Agents

agentenv-framework

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

opik-optimizer

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.