Gguf Python Packages

auto-round

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包

xllamacpp

xllamacpp - a Python wrapper of llama.cpp

auto-round-nightly

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包

auto-round-lib

A simple and effective post training quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的后训练量化工具包

llama-roofline

Is your llama.cpp decode memory-bandwidth-bound? Find out in one command.

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

mlx-kquant

Native K-quant support for MLX, with a quantization and fine-tuning toolchain for Apple Silicon

soup-cli

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

gguf-connector

gguf (GPT-Generated Unified Format) connector

termux-diffusion

AmfyUI (ComfyUI Mobile DAG Studio) & Z-Image Turbo (6.0B DiT) On-Device Diffusion Runtime for Android Termux

quantfit

Quantize an LLM and check it still refuses what it should — a GPU-aware quantization CLI that measures the safety drift of the quantization it just performed.

lilbee

The whole local AI stack in one executable: it runs and manages local AI models across every GPU, and it's a search engine you can talk to, with cited answers from your files, code, and the web. MCP server for coding agents, web crawler, TUI, CLI, REST API, Python library. No Ollama or LM Studio needed, works with both.

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

transcribe-cpp-native

ggml speech-to-text inference for 16+ model families

gmlx

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

termux-llamacpp

Production-grade, prebuilt GGUF LLM runtime, model manager & OpenAI-compatible server for Android Termux & ARM64.

opendecider

Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVIDIA, Apple Silicon, LM Studio, Ollama and vLLM.

hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

inferhost

Self-hosted multi-modal AI server: chat/vision LLMs (Qwen, Llama, Gemma, DeepSeek), text-to-speech (OuteTTS, Qwen3-TTS) and image generation (SDXL, Flux, Z-Image, Qwen-Image) behind one OpenAI-compatible endpoint. Wraps llama.cpp + stable-diffusion.cpp, auto-fetches GGUF models from Hugging Face, VRAM hot-swap, speculative decoding, no compiling.

transcribe-cpp

ggml speech-to-text inference for 16+ model families

hfl

Download, run and try any Hugging Face model on your own machine — OpenAI-, Ollama- and Anthropic-compatible API. No account, no cloud.

quantcpp

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.