Llama Cpp Python Packages

lilbee

A local AI search engine: it runs and manages local AI models, searches your files and code, and crawls the web, all in one program. Cited answers, local-first, with an MCP server for your coding agent. TUI, CLI, REST API, and Python library. Works with Ollama and LM Studio.

forge-guardrails

A Python framework for self-hosted LLM tool-calling and multi-step agentic workflows

cyllama

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

scribe-llm

Universal TUI & web agent that connects to any llama.cpp server and uses RAG + semantic memory to research, write, and remember across sessions.

llama-cpp-py-sync

Auto-synced CFFI ABI python bindings for llama.cpp with prebuilt wheels (CPU/CUDA/Vulkan/Metal).

sawyer-core

Distributed MoE inference network — the load is split, friends help.

hermes-router

Local AI control center and routing/control plane for OpenAI-compatible agents, local runtimes, receipts, telemetry, and safety policy.

cyllama-vulkan

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

mycellm

Distributed LLM inference across heterogeneous hardware. Pool GPUs into a P2P network with QUIC transport, Ed25519 identity, and an OpenAI-compatible API.

cyllama-cuda12

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-rocm

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

cyllama-sycl

A thin cython wrapper around llama.cpp, whisper.cpp and stable-diffusion.cpp

opencode-llmstack

Cursor-Auto / Claude-tier-style serving for local GGUF models on Mac (M4 Max, 64 GB). FastAPI router fronts llama-swap + llama.cpp, classifying each request into a coder, planner, or uncensored-planner tier. OpenAI-compatible API, opencode integration, per-project subshell, one `llmstack` console-script.

sollol

Super Ollama Load Balancer - Performance-aware routing for distributed Ollama deployments with Ray, Dask, and adaptive metrics

rikis

Lightweight agent that pulls and runs LLM benchmarks from Protorikis Bench

llmesh-mcp

Secure LLM Mesh over MCP — multi-LLM swarm + industrial IoT + privacy gateway

arc-llama

Plug-and-play llama.cpp runtime for Intel Arc GPUs. Auto-detects your card, picks safe SYCL defaults, and exposes an OpenAI-compatible API.

llama-buddy

CLI wrapper for llama.cpp providing an ollama-like experience

hfl

CLI + API server to download, manage, and run 500K+ HuggingFace models locally with Ollama & OpenAI compatibility

inzen-bot

Multi-provider AI chatbot for the terminal. Chat with Claude, GPT-4, Ollama or llama.cpp without leaving your shell. Full conversation history. pip install inzen-bot

notolog

Notolog Markdown Editor

llama-server

LLaMA Server combines the power of LLaMA C++ with the beauty of Chatbot UI.

tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

llm-gpu-lab

One GPU. Full LLM workflow. Real benchmarks. No cloud required.