Prompt Caching Python Packages

distil-llm

Compression with a quality contract — cache-aware, causally-pruned LLM context compression for agentic runtimes, certified non-inferior across 7 domains. Works with any SDK.

modelmeld

OpenAI- and Anthropic-compatible AI gateway. Capability-based routing. Streaming. BYOK passthrough. No key custody.

justllm

Production LLM calls. Just the three lines. Cross-provider fallback, native caching, and reversible context compression on by default.

burnless

Multi-turn agent loops cost O(N²). Burnless makes them O(N). 88% cheaper at turn 10. MIT, provider-agnostic.

cognisn-spark

Secure Personal AI Research Kit - Multi-provider LLM web interface with MCP tool integration

ctxbudgeter

ContextOps toolkit for production AI agents. Compile, govern, scan, visualize & optimize LLM context before every model call — token budgets, policy governance, PII/secret scanning, Context Bill of Materials, diffing, Context MRI, MCP budgeting. Framework-agnostic. Local-first. Deterministic.

agentlite-py

A small, focused library for building Claude-powered agents in Python — minimal abstractions, prompt caching, permissions as first-class.

cachellm-py

Auto-optimize LLM prompt caching. Save 60-90% on Claude, GPT & Gemini API costs with one line of code.

metis-llm

Local-first AI dev agent for your terminal — provider-agnostic, cost-aware, self-improving. Apache-2.0.

cotarag

Cognitive Thought and Retrieval Augmented Generation - An advanced AI agent framework combining CoTAEngine and AcceleRAG