Vram Python Packages

whichllm

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

turboquant

First open-source TurboQuant KV cache compression for LLM inference. Drop-in for HuggingFace. pip install turboquant.

llm-cal

LLM inference hardware calculator — architecture-aware, engine-version-aware, honest-labeled.

gemma4-adaptive-router

Adaptive dual-tier serving for Gemma 4 on consumer 16GB GPUs. Complexity + real-time VRAM routing between vLLM E4B and llama.cpp 27B. Production stack with OpenWebUI, monitoring, and more.

llm-neofetch-plus

LLM-Neofetch++ is an advanced system information tool designed specifically for local LLM (Large Language Model) usage. It provides detailed hardware detection with personalized recommendations for running AI models on your system.

quantsim-bench

Which quantization should I use? One command benchmarks every quant level on YOUR GPU.

spectral-kv

Up to 28x KV cache compression for LLMs via spectral SVD projection

gpu-memory-guard

CLI tool to check GPU VRAM before loading AI models. Prevent OOM crashes.

kvcache-bench

Benchmark every KV cache compression method on your GPU. One command, real numbers. Supports Ollama + llama.cpp.

hcgk-kernel

Hardware Control GateKeeper Kernels for AI inference within frameworks.