PyRank
  • Search
  • PyPI
  • GitHub
  • Insights
  • Compare
  • Advisories
  • Ecosystem
  • About

Llm Caching Python Packages

Python packages with the GitHub topic llm-caching. Sorted by monthly downloads, with stars and forks.
2796gaurav
shad0wllm

System 0 for your LLM: learns the fixed-choice answers your LLM gives (intents, routing, moderation) and serves the repeats in microseconds from a small CPU table, at most 5% disagreement, certified. Zero-code proxy for the OpenAI Decisions API, System One and chat models.

15K 0 0
imxinchengyou
cacheverifier

Verification layer for semantic caches: a REST API and Python client that checks gray-zone cache hits with a cross-encoder before they are served.

963 1 0
smartass-4ever
mnemon-ai

Biological nervous systems don't recompute known workflows from scratch. Mnemon gives LLM agents the same primitive — execution memory that caches plans, not responses. 93% token reduction, 2.66ms vs 20s, zero tokens on repeat runs. LangChain, CrewAI, AutoGen.

324 6 2
    • Original data from PyPI, GitHub, and OSV; downloads served through ClickHouse
    • Sources & methodology