PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Llm Infernece Python Packages

Python packages with the GitHub topic llm-infernece. Sorted by relevance, with stars and monthly downloads.
konjoai
squish-ai

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

9K 10 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery