Semantic Caching Python Packages

undertow-llm

Provider-agnostic LLM observability & reliability SDK — semantic caching, retries, rate limiting, and real-time dashboard via a single @track() decorator.

agentfuse-runtime

Intelligent LLM agent cost optimization runtime.

lensllm

Provider-agnostic LLM observability & reliability SDK — semantic caching, retries, rate limiting, and real-time dashboard via a single @track() decorator.

smartmemo

Semantic memory and caching for LLM agents with classifier-validated equivalence instead of naive cosine thresholds.

hyperion-ai

Ultra-low-latency LLM gateway with microsecond caching, dynamic routing, budgets, analytics, and forecasting.

semcache

Python library for the Semcache API