Tokenspeed Python Packages

tokenspeed-mla

TokenSpeed is a speed-of-light LLM inference engine.

smg-grpc-proto

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

smg-grpc-servicer

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

mooncake-transfer-engine-cuda13

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-non-cuda

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

tokenspeed-smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

tokenspeed-smg-grpc-proto

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

tokenspeed-smg-grpc-servicer

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

smg

Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.

tokenspeed-mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-rocm

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-efa-cuda13

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

tokenspeed-cutedsl-kda

TokenSpeed is a speed-of-light LLM inference engine.

mooncake-transfer-engine-npu

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

tokenspeed-kernel-amd

TokenSpeed is a speed-of-light LLM inference engine.

mooncake-transfer-engine-efa

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-efa-non-cuda

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-musa

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

tokenrl

TokenSpeed is a speed-of-light LLM inference engine.

tokenspeed-kernel-npu

TokenSpeed is a speed-of-light LLM inference engine.

tokenspeed-situ

TokenSpeed is a speed-of-light LLM inference engine.

tokenspeed-trtllm-kda

TokenSpeed is a speed-of-light LLM inference engine.