Llama Cpp Python Packages

xllamacpp

xllamacpp - a Python wrapper of llama.cpp

llama-roofline

Is your llama.cpp decode memory-bandwidth-bound? Find out in one command.

spark-pulse

Spark Pulse is a web control plane for spark-vllm-docker

hugpy-video

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

hugpy

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

cyllama

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

lilbee

The whole local AI stack in one executable: it runs and manages local AI models across every GPU, and it's a search engine you can talk to, with cited answers from your files, code, and the web. MCP server for coding agents, web crawler, TUI, CLI, REST API, Python library. No Ollama or LM Studio needed, works with both.

localharness

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harness (detailed event steams, logs, and performance data) or quickly replace closed source one (featuring out of the box plugins for web search, image gen, memory, auto research and more in dev).

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

hubrelay

Self-hosted LLM gateway — from LAN to the public internet (tunnel-friendly): multi-upstream key pool, downstream tokens, quota accounting, MCP & A2A routing, E2E envelope encryption. 自托管大模型中转站，局域网与公网两用，兼容 OpenAI/Anthropic。

cyllama-vulkan

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

hugpy-engine

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

llama-mdl

Run local llama.cpp servers from a config file, with an optional terminal dashboard

dflash-console

Local Windows control panel for llama-server and DFlash speculative-decoding engines.

hugpy-oracle

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

hugpy-storage

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

cyllama-cuda12

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

hugpy-platform

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

hugpy-media

Self-hosted LLM console, OpenAI-compatible API and GPU worker fleet: 13 hugpy-* Python distributions and the React consoles.

mcgyvr

Offload scoped coding work from your agent to a configurable worker ladder — deterministic tools, local models, API models. A skill for TUI/CLI agent harnesses.

inferhost

Self-hosted multi-modal AI server: chat/vision LLMs (Qwen, Llama, Gemma, DeepSeek), text-to-speech (OuteTTS, Qwen3-TTS) and image generation (SDXL, Flux, Z-Image, Qwen-Image) behind one OpenAI-compatible endpoint. Wraps llama.cpp + stable-diffusion.cpp, auto-fetches GGUF models from Hugging Face, VRAM hot-swap, speculative decoding, no compiling.

hfl

Download, run and try any Hugging Face model on your own machine — OpenAI-, Ollama- and Anthropic-compatible API. No account, no cloud.

inferna

An experimental nanobind wrapper around llama.cpp, whisper.cpp, and stable-diffusion.cpp