Ai Gateway Python Packages

litellm

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-proxy-extras

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

litellm-enterprise

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

portkey-ai

Build reliable, secure, and production-ready AI apps easily.

freeride-gateway

One free AI endpoint, every free tier behind it. Local OpenAI-compatible gateway routing across OpenRouter, Groq, NVIDIA NIM, Cloudflare Workers AI, and HuggingFace with automatic failover.

ai-stats-py-sdk

Open-source, OpenAI-compatible AI gateway with health-aware routing, failover, and live telemetry (cost, latency, errors). Built for production.

blockrun-litellm

LiteLLM adapter for BlockRun — call x402-paid AI models through LiteLLM (custom provider or local OpenAI-compatible proxy)

ollama-herd

Local AI load balancer for Ollama fleets — auto-discovery, smart routing, OpenAI-compatible API, zero config. Perfect for Mac Minis & Studios.

cost-katana

Cost Katana Python

amp-instrumentation

WSO2 AI Agent Manager is an open control plane designed for enterprises to deploy, manage, and govern AI agents at scale.

llmkit-sdk

Know what your AI agents cost. API gateway with budget enforcement, session tracking, and MCP tools.

sagify

LLMs and Machine Learning done easily

ferrolabsai

Ferro Labs AI Gateway Python SDK - Instrument your LLM app, Works with any LLM or framework

freemygpt

FreeMyGPT — HTTP gateway that lets ChatGPT (and any LLM) drive any local MCP server via simple GET requests. Hybrid: MCP stdio backends by default, Codex CLI and other runtimes via bundled adapters.

litellm-mcp

MCP server giving AI agents access to 100+ LLMs through LiteLLM

rubeus

Build reliable, secure, and production-ready AI apps easily.

rosetta-llm

Multi-format LLM proxy — translates OpenAI Chat, OpenAI Responses & Anthropic Messages. Works as a Claude Code LLM gateway with full model picker support.

fastapi-ai-router

FastAPI middleware that turns your existing routes into a natural-language-callable surface using LLM function-calling. Drop-in. Zero new metadata.

litellm-dev

Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM]

vllm-sr-sim

System Level Intelligent Router for Mixture-of-Models at Cloud, Data Center and Edge

llama-index-llms-ferrolabsai

Ferro Labs AI Gateway Python SDK - Instrument your LLM app, Works with any LLM or framework

langgate

Lightweight AI inference gateway - local model registry & parameter transformer (Python SDK) - with optional Envoy proxy processor and FastAPI registry server deployment options.

schmitech-orbit-client

An MCP Python client for interacting with Orbit chat servers

langchain-ferrolabsai

Ferro Labs AI Gateway Python SDK - Instrument your LLM app, Works with any LLM or framework