Ai Gateway Python Packages

litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-proxy-extras

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-enterprise

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

portkey-ai

Build reliable, secure, and production-ready AI apps easily.

modelmeld

OpenAI- and Anthropic-compatible AI gateway. Capability-based routing. Streaming. BYOK passthrough. No key custody.

agent-magnet

Give your AI a memory. Drop-in learning layer for any LLM — no retraining, no RAG setup.

litellm-dev

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

blockrun-litellm

LiteLLM adapter for BlockRun — call x402-paid AI models through LiteLLM (custom provider or local OpenAI-compatible proxy)

privaite

Drop-in self-hosted LLM proxy that reversibly redacts PII before OpenAI, ChatGPT and Ollama calls, including inside tool-call arguments and multimodal content. OpenAI-compatible, zero telemetry.

gavio

The open standard AI gateway — PII protection, audit, reliability & cost control as composable interceptors. Same API in Python, Java & JavaScript.

freelm

Free, always-up LLM client & gateway for Python and TypeScript — one OpenAI-compatible API over free-tier OpenRouter, Google Gemini, NVIDIA NIM, Groq, Cerebras & Mistral with automatic failover, key rotation, streaming and live model discovery.

amp-instrumentation

WSO2 AI Agent Manager is an open control plane designed for enterprises to deploy, manage, and govern AI agents at scale.

arbr-client

Open-source LLM control plane: observe every call, recommend savings, then apply only human-approved, reversible routing.

freeride-gateway

One free AI endpoint, every free tier behind it. Local OpenAI-compatible gateway routing across OpenRouter, Groq, NVIDIA NIM, Cloudflare Workers AI, and HuggingFace with automatic failover.

ollama-herd

Local AI load balancer for Ollama fleets — auto-discovery, smart routing, OpenAI-compatible API, zero config. Perfect for Mac Minis & Studios.

ai-stats-py-sdk

Open-source, OpenAI-compatible AI gateway with health-aware routing, failover, and live telemetry (cost, latency, errors). Built for production.

cost-katana

Cost Katana Python

litellm-mcp

MCP server giving AI agents access to 100+ LLMs through LiteLLM

ai-devsec-gateway

Open-source DevSecOps gateway for AI traffic. Intercept, audit, and route LLM requests to prevent data leaks and enforce zero-trust security policies.

llmkit-sdk

Know what your AI agents cost. API gateway with budget enforcement, session tracking, and MCP tools.

langgate

Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.

vllm-sr-sim

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

ferrolabsai

Ferro Labs AI Gateway Python SDK - Instrument your LLM app, Works with any LLM or framework

freemygpt

FreeMyGPT — HTTP gateway that lets ChatGPT (and any LLM) drive any local MCP server via simple GET requests. Hybrid: MCP stdio backends by default, Codex CLI and other runtimes via bundled adapters.