Llm Monitoring Python Packages

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

claude-telemetry

OpenTelemetry wrapper for Claude Code CLI that logs tool calls, token usage, costs, and execution traces to Logfire, Sentry, Honeycomb, or Datadog. Drop-in replacement that swaps 'claude' command for 'claudia'.

peekr

Zero-config observability for AI agents. Auto-instruments OpenAI & Anthropic SDKs.

spanlens

Open source LLM observability and monitoring. Drop-in proxy for OpenAI, Anthropic, and Gemini with request logging, cost tracking, and agent tracing. Self-host with one Docker command. MIT.

drift-detector-agent

Behavioral drift detection for LLM agents — catch vocabulary loss, tool changes, loops, and stagnation before they break production.

llm-spend-profiler

Track, visualize, and optimize LLM API spending. Two lines of code, zero config.

arguslm

The hundred-eyed watcher for your LLM providers. Monitor uptime, TTFT, TPS, and latency across OpenAI, Anthropic, Azure, Bedrock, Ollama, LM Studio, and 100+ providers through a single dashboard. Benchmark, compare, and get alerts — all self-hosted.

mesedi

Mesedi SDK — guardians for autonomous AI agents. Detect failures, halt runaways, escalate first occurrence.

opensentinel

Open Source Reliability Harness: Make your agents follow rules. One line of code to‎ ‎enforce, trace, and improve. ‎ ‎

openbias

Open Source Reliability Harness: Make your agents follow rules. One line of code to‎ ‎enforce, trace, and improve. ‎ ‎