Llm Gateway Python Packages

litellm

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

litellm-proxy-extras

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

litellm-enterprise

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

ddg-agent-services-mcp

Pay-per-call x402 gateway for AI agents — MCP server + OpenAI-compatible API for agent tools, market data, RPC, and MCP security audits. Pay in USDC on Base, no account.

inferrail

Self-hosted OpenAI/Anthropic-compatible gateway: give each AI agent run a dollar budget, enforced before the provider, with a payload-free per-run cost record.

g8e

Verify, then execute.

litellm-mysubs

Serve your Claude Max, ChatGPT Plus and Google Antigravity subscriptions as LiteLLM models. One command to install.

hubrelay

Self-hosted LLM gateway — from LAN to the public internet (tunnel-friendly): multi-upstream key pool, downstream tokens, quota accounting, MCP & A2A routing, E2E envelope encryption. 自托管大模型中转站，局域网与公网两用，兼容 OpenAI/Anthropic。

llm-shield-proxy

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-boundary leaks), mcp-ssrf-check, and a self-hosted redacting gateway.

llm-rosetta

Production-ready LLM API translation layer — bidirectional conversion between OpenAI, Anthropic & Google formats via hub-and-spoke IR. Optional API gateway. Streaming & non-streaming. Zero core deps. Contributions welcome!

ferrogw

Unified AI Gateway for 30+ LLMs (OpenAI, Anthropic, Bedrock, Azure etc) with Caching, Guardrails, A/B test & cost controls. Go-native Fastest & Scalable AI Gateway LiteLLM & Kong AI Gateway alternative.

litellm-dev

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

pii-leak-benchmark

Streaming privacy for LLM apps: pii-leak-benchmark (does your gateway leak PII in streams?), chunk-invariance (one-line test for split-boundary leaks), mcp-ssrf-check, and a self-hosted redacting gateway.

radlab-llm-router

LLM Router is a service that can be deployed on‑premises or in the cloud. It adds a layer between any application and the LLM provider. In real time it controls traffic, distributes a load among providers of a specific LLM, and enables analysis of outgoing requests from a security perspective (masking, anonymization, prohibited content).

neutral-llm-gateway

Neutral LLM gateway: typed contracts, provider adapters, explicit retry/fallback/usage/cost accounting.

codai-sdk

Official Python SDK for codai - OpenAI-compatible client for the codai gateway. PyPI: codai-sdk

acruxcore

Open-source LLMOps platform with prompt management, an OpenAI-compatible AI gateway, OpenTelemetry tracing, a versioned tool catalog, and LLM evaluation. Use hosted or self-host with TypeScript and Python SDKs.

llimona

Llimona is an open and modular Python framework for building production-ready LLM gateways

a3m-router

⚡ Adaptive multi-model LLM router — 80+ providers, Jev System One single-pass routing (model=jev-auto), pheromone-trail failover, parallel ensemble merge. npm: adaptive-memory-multi-model-router

freellmpool

Free LLM gateway: 22 LLM providers, 178 enabled chat routes, 431 cataloged chat models; keyless start when available.

paritok

Non-destructive compression gateway for AI coding agents. Cuts token bills 25% on turn 1 to past 85% in long or saturated sessions, and fits ~3× more turns in the same context window. Powered by our open-source code-native 4B model. Drop-in for Claude Code, Cursor, Codex, OpenHands, and any BASE_URL agent.

rlcd-gateway

Self-hosted gateway for LLMs and decision models. Claude Code, Codex, OpenCode and any OpenAI/Anthropic SDK app reach any provider (OpenRouter, Groq, Ollama…) with context pruning; Jev and open-rlcd System One decisions get audit and calibration. Live dashboard, single Go binary.

aegis-latent-sdk

Self-hosted evidence gateway for AI systems: fail-closed policy, WAF, egress controls, signed durable MMR proofs, and offline verification across LLM providers.

freelm

Free LLM API for Python & Node.js: a free-only, lightweight LiteLLM/OpenRouter alternative. One OpenAI-compatible client and local /v1 endpoint over Gemini, Groq, OpenRouter, Cloudflare, Mistral, NIM, Z.ai, Cohere & Kilo, with LLM fallback that races slow providers and key rotation. Works with no API keys.