Gguf Python Packages

auto-round

PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度

xllamacpp

xllamacpp - a Python wrapper of llama.cpp

auto-round-nightly

PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度

auto-round-lib

PTQ-Level Efficiency, QAT-Level Accuracy|训练后量化的效率，量化感知训练的精度

llama-roofline

Is your llama.cpp decode memory-bandwidth-bound? Measure it, in one command.

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

mlx-kquant

Native K-quant support for MLX, with a quantization and fine-tuning toolchain for Apple Silicon

quantfit

Quantize an LLM and check it still refuses what it should — a GPU-aware quantization CLI that measures the safety drift of the quantization it just performed.

soup-cli

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

gguf-connector

gguf (GPT-Generated Unified Format) connector

termux-diffusion

Production On-Device AI Image Generation Framework for Android Termux & Samsung Galaxy. Dual-Engine (Python & Node.js), Bionic ARM DotProd/FP16 SIMD, Vulkan GPU acceleration.

transcribe-cpp-native

ggml speech-to-text inference for 16+ model families

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

lilbee

The whole local AI stack in one executable: it runs and manages local AI models across every GPU, and it's a search engine you can talk to, with cited answers from your files, code, and the web. MCP server for coding agents, web crawler, TUI, CLI, REST API, Python library. No Ollama or LM Studio needed, works with both.

gmlx

Local inference platform for K/IQ-quant GGUF models on Apple Silicon

termux-llamacpp

Production-grade, prebuilt GGUF LLM runtime, model manager & OpenAI-compatible server for Android Termux & ARM64.

opendecider

Open, calibrated System 1 decision models: typed choice / score / yes-no answers with a probability for every option. 400M to 80B; CPU, NVIDIA, Apple Silicon, LM Studio, Ollama and vLLM.

transcribe-cpp

ggml speech-to-text inference for 16+ model families

hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

hfl

Download, run and try any Hugging Face model on your own machine — OpenAI-, Ollama- and Anthropic-compatible API. No account, no cloud.

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

quantcpp

Single-header LLM inference engine with KV cache compression (7× compression at fp32 parity)

inferhost

Self-hosted multi-modal AI server: chat/vision LLMs (Qwen, Llama, Gemma, DeepSeek), text-to-speech (OuteTTS, Qwen3-TTS) and image generation (SDXL, Flux, Z-Image, Qwen-Image) behind one OpenAI-compatible endpoint. Wraps llama.cpp + stable-diffusion.cpp, auto-fetches GGUF models from Hugging Face, VRAM hot-swap, speculative decoding, no compiling.

make-localllm-easier

One command runs the best local LLM your GPU can handle: model picked from measured multilingual benchmarks, llama.cpp tuned for AMD, NVIDIA, Intel and Apple GPUs.