Consumer Gpu Python Packages

ultracompress

Lossless 5-bit transformer compression. 22 architectures shipped (0.6B-405B incl. dense + MoE + SSM), 14 PPL-verified. Hermes-3-405B 1.0066x, Mistral-7B 1.00548x, Mixtral-8x7B 1.00368x. SHA-256-verifiable bit-identical reconstruction. OpenAI-compatible API at api.sipsalabs.com. pip install ultracompress

turboquant-vllm

TurboQuant KV cache compression plugin for vLLM — asymmetric K/V, 8 models validated, consumer GPUs

zorac

Interactive CLI chat client for vLLM inference servers with persistent sessions and automatic context management