inference-benchmark
Measure local LLM speed, first-token latency, and throughput. Reproducible benchmarks across GPUs and Apple Silicon.
Vram budgeting and tail-latency simulation for three models sharing one triton gpu
