Llm Monitoring Python Packages

agenta

The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.

claude-telemetry

OpenTelemetry wrapper for Claude Code CLI that logs tool calls, token usage, costs, and execution traces to Logfire, Sentry, Honeycomb, or Datadog. Drop-in replacement that swaps 'claude' command for 'claudia'.

drift-detector-agent

Behavioral drift detection for LLM agents

openbias

Open Source Reliability Harness: Make your agents follow rules. One line of code to‎ ‎enforce, trace, and improve. ‎ ‎

llm-spend-profiler

Track, visualize, and optimize LLM API spending. Monitor OpenAI & Anthropic costs per feature, detect waste, suggest savings. Zero-config Python profiler.

opensentinel

Reliability layer for AI agents - monitors workflow adherence and intervenes when agents deviate

arguslm

ArgusLM — Open-source LLM monitoring & benchmarking SDK