Gguf Python Packages

auto-round

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

whichllm

Find the local LLM that actually runs and performs best on your hardware. Ranked by real, recency-aware benchmarks, not parameter count. One command, run it instantly.

lilbee

A local AI search engine: it runs and manages local AI models, searches your files and code, and crawls the web, all in one program. Cited answers, local-first, with an MCP server for your coding agent. TUI, CLI, REST API, and Python library. Works with Ollama and LM Studio.

llmfit

Hundreds of models & providers. One command to find what runs on your hardware.

auto-round-lib

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

auto-round-nightly

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

gguf-connector

gguf (GPT-Generated Unified Format) connector

soup-cli

Soup turns the pain of LLM fine-tuning into a simple workflow. One config, one command, done.

hf-mem

A CLI to estimate inference memory requirements for Hugging Face models, written in Python.

transcribe-cpp-native

ggml speech-to-text inference for 16+ model families

omi-med-stt

On-device English medical speech-to-text — CLI for Omi Med STT v1 (MLX / NeMo / parakeet.cpp)

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

sawyer-core

Distributed MoE inference network — the load is split, friends help.

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

quantcpp

LLM inference with 7x longer context. Pure C, zero dependencies. Lossless KV cache compression + single-header library.

outetts

Interface for OuteTTS models.

transcribe-cpp

ggml speech-to-text inference for 16+ model families

opencode-llmstack

Cursor-Auto / Claude-tier-style serving for local GGUF models on Mac (M4 Max, 64 GB). FastAPI router fronts llama-swap + llama.cpp, classifying each request into a coder, planner, or uncensored-planner tier. OpenAI-compatible API, opencode integration, per-project subshell, one `llmstack` console-script.

ovos-gguf-embeddings-plugin

DEPRECATED — folded into OpenVoiceOS/ovos-gguf-plugin (keeps plugin name 'ovos-gguf-embeddings-plugin'). Install ovos-gguf-plugin.

ctransformer-core

solo connector core built on ctransformers

gguf-core

a simple way to interact llama with gguf

llama-core

solo connector core built on llama.cpp

auto-round-hpu

A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.

gguf-node

gguf node for comfyui