Llm Monitoring Python Packages

claude-telemetry

OpenTelemetry wrapper for Claude Code CLI that logs tool calls, token usage, costs, and execution traces to Logfire, Sentry, Honeycomb, or Datadog. Drop-in replacement that swaps 'claude' command for 'claudia'.

spanlens

Open source LLM observability and monitoring. Drop-in proxy for OpenAI, Anthropic, and Gemini with request logging, cost tracking, and agent tracing. Self-host with one Docker command. MIT.

peekr

Zero-config observability for AI agents. Auto-instruments OpenAI & Anthropic SDKs.

llm-spend-profiler

Track, visualize, and optimize LLM API spending. Monitor OpenAI & Anthropic costs per feature, detect waste, suggest savings. Zero-config Python profiler.

drift-detector-agent

Behavioral drift detection for LLM agents — catch vocabulary loss, tool changes, loops, and stagnation before they break production.

openbias

Open Source policy enforcement proxy: Make your agents follow rules.

opensentinel

Open Source policy enforcement proxy: Make your agents follow rules.

arguslm

The hundred-eyed watcher for your LLM providers. Monitor uptime, TTFT, TPS, and latency across OpenAI, Anthropic, Azure, Bedrock, Ollama, LM Studio, and 100+ providers through a single dashboard. Benchmark, compare, and get alerts — all self-hosted.

te-drift-detector

Experimental Python tool for inspecting language and task-framing changes across long AI conversations.