Ai Gateway Python Packages

litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

litellm-proxy-extras

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

litellm-enterprise

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

portkey-ai

Build reliable, secure, and production-ready AI apps easily.

my-claude-code

My Claude Code (MCC) is a local multi-provider LLM proxy, model router and control plane for Claude Code, Codex, OpenCode, Gemini CLI and other AI coding agents — 57 providers, fallback routing, credential health, a native web-search tool proxy, and exhaustive analytics with cost provenance in one local dashboard.

rosetta-llm

Multi-format LLM proxy — translates OpenAI Chat, OpenAI Responses & Anthropic Messages. Works as a Claude Code LLM gateway with full model picker support.

sandhi-gateway

Sandhi — the metering layer for AI agents. An open-source AI usage gateway: meter, attribute, and budget every model call across shared keys.

inferrail

Self-hosted OpenAI/Anthropic-compatible gateway: give each AI agent run a dollar budget, enforced before the provider, with a payload-free per-run cost record.

ferrogw

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

litellm-dev

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

privaite

Self-hosted PII redaction proxy for LLM APIs and agent CLIs. Scrubs names, emails and secrets from Claude Code, Codex and any OpenAI-compatible app before requests leave your machine, tool-call arguments included, then restores them in the reply. Local detection, zero telemetry.

codai-sdk

Official Python SDK for codai - OpenAI-compatible client for the codai gateway. PyPI: codai-sdk

acruxcore

Open-source LLMOps platform with prompt management, an OpenAI-compatible AI gateway, OpenTelemetry tracing, a versioned tool catalog, and LLM evaluation. Use hosted or self-host with TypeScript and Python SDKs.

ollama-herd

Local AI load balancer for Ollama and MLX fleets. Auto-discovery, smart routing, OpenAI-compatible API, zero config. Perfect for Mac Minis & Studios.

undertow-llm

Provider-agnostic LLM observability & reliability SDK — semantic caching, retries, rate limiting, and real-time dashboard via a single @track() decorator.

agent-magnet

Remembers your project so you don't have to explain it again. Memory across Claude, Codex, Cursor.

a3m-router

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel ensemble merge. npm: adaptive-memory-multi-model-router

blockrun-litellm

LiteLLM adapter for BlockRun — call x402-paid AI models through LiteLLM (custom provider or local OpenAI-compatible proxy)

rlcd-gateway

Self-hosted gateway for LLMs and decision models. Claude Code, Codex, OpenCode and any OpenAI/Anthropic SDK app reach any provider (OpenRouter, Groq, Ollama…) with context pruning; Jev and open-rlcd System One decisions get audit and calibration. Live dashboard, single Go binary.

freelm

Free LLM API for Python & Node.js: a free-only, lightweight LiteLLM/OpenRouter alternative. One OpenAI-compatible client and local /v1 endpoint over Gemini, Groq, OpenRouter, Cloudflare, Mistral, NIM, Z.ai, Cohere & Kilo, with LLM fallback that races slow providers and key rotation. Works with no API keys.

amp-instrumentation

WSO2 AI Agent Manager is an open control plane designed for enterprises to deploy, manage, and govern AI agents at scale.

freeride-gateway

One free AI endpoint, every free tier behind it. Local OpenAI-compatible gateway routing across OpenRouter, Groq, NVIDIA NIM, Cloudflare Workers AI, and HuggingFace with automatic failover.

leanroute

A fast decision layer in front of any LLM: blocks attacks, sends easy requests to cheap models, hard ones to strong models, and shows the money saved. OpenAI-compatible, self-hostable, powered by Laya.

nrouter-sdk

Official nRouter SDKs & Examples — Universal AI gateway client for TypeScript, Python, Go, Rust, Ruby, PHP, Java, .NET, Swift, and cURL with smart routing, streaming, and guardrails.