PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Llm Gateway Python Packages

Python packages with the GitHub topic llm-gateway. Sorted by relevance, with stars and monthly downloads.
BerriAI
litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

464.4M 48K 8K
BerriAI
litellm-proxy-extras

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

1.5M 48K 8K
BerriAI
litellm-enterprise

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

1.3M 48K 8K
llimona-org
llimona

Llimona is an open and modular Python framework for building production-ready LLM gateways

9K 1 0
Oaklight
llm-rosetta

Production-ready LLM API translation layer for Python — bidirectional conversion between OpenAI, Anthropic & Google formats via hub-and-spoke IR. Optional API gateway. Streaming & non-streaming. Zero core deps. Contributions welcome!

6K 21 1
thatAverageGuy
llmgate

Lightweight, provider-agnostic Python LLM library — one API for OpenAI, Gemini, Anthropic, Groq, Mistral, Cohere, Azure, Bedrock & Ollama. Vision, streaming, tools, batch.

3K 2 0
doramirdor
nadirclaw

Open-source LLM router & AI cost optimizer. Routes simple prompts to cheap/local models, complex ones to premium — automatically. Drop-in OpenAI-compatible proxy for Claude Code, Codex, Cursor, OpenClaw. Saves 40-70% on AI API costs. Self-hosted, no middleman.

2K 497 58
Nayjest
ai-proxy-server

AI Proxy Server is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

2K 127 12
Nayjest
lm-proxy

OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.

2K 127 12
Nayjest
llm-proxy-server

OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.

967 127 12
Nayjest
openai-http-proxy

OpenAI HTTP Proxy is an OpenAI-compatible http proxy server for inferencing various LLMs capable of working with Google, Anthropic, OpenAI APIs, local PyTorch inference, etc.

807 127 12
Nayjest
inference-proxy

OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.

792 127 12
Nayjest
lm-proxy-server

OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.

711 127 12
Nayjest
oai-proxy

OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.

637 127 12
shin-bot-litellm
litellm-mcp

MCP server giving AI agents access to 100+ LLMs through LiteLLM

597 38K 6K
BerriAI
litellm-dev

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

528 48K 8K
Tanantor
langgate

Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.

401 6 0
kesiee
llmgt

Plug-and-play LLM connector via YAML config — 21 providers, async, tool calling, structured outputs, embeddings, middleware. Zero bloat.

380 3 0
adwantg
kvfleet

KV-cache-aware intelligent routing for self-hosted and hybrid LLM fleets. Route requests using model quality, latency, cost, policy, and live GPU state.

319 0 0
BerriAI
litellm-ad

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

296 48K 8K
kikuai-lab
reliapi-sdk

Python SDK for reliapi - Reliability layer for API calls

244 8 0
ruslanmv
ollabridge

OllaBridge transforms your laptop or workstation into a production-grade, OpenAI-compatible LLM provider. It features self-healing setup, built-in security, and automatic tunneling—making your local hardware ready for real applications in 60 seconds.

221 0 0
BerriAI
litellm-plus

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

196 48K 8K
BerriAI
litellm-slipstream-fork

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

194 48K 8K
    • Data from PyPI, GitHub, ClickHouse, and BigQuery