Kvcache Python Packages

mooncake-transfer-engine-cuda13

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-non-cuda

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

tokenspeed-mooncake

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-rocm

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-efa-cuda13

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-npu

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

uc-manager-cuda-cu129

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann910-a3

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann910-a2

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cuda-cu130

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a2

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a3

Persist and reuse KV Cache to speedup your LLM.

mooncake-transfer-engine-efa

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

mooncake-transfer-engine-efa-non-cuda

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

uc-manager-cann910-a5

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a5

Persist and reuse KV Cache to speedup your LLM.

mooncake-transfer-engine-musa

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

uccl

UCCL is an efficient communication library for GPUs, covering collectives, P2P (e.g., KV cache transfer, RL weight transfer), and EP (e.g., GPU-driven)

kvpacket

Recomputation-free, context-independent KV caching for large language models