A quick and optimized solution to manage llama based gguf quantized models, download gguf files, retreive messege formatting, add more models from hf repos and more. It's super easy to use and comes prepacked with best preconfigured open source models: dolphin phi-2 2.7b, mistral 7b v0.2, mixtral 8x7b v0.1, solar 10.7b and zephyr 3b
🦙 Put your LLM server through a marathon — one command to benchmark any OpenAI-compatible endpoint (Ollama, vLLM, llama.cpp), with model auto-detection, real-workload sizing, and self-contained HTML reports.
