Llamacpp Python Packages

spark-nlp

State of the Art Natural Language Processing

khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

xllamacpp

xllamacpp - a Python wrapper of llama.cpp

khoj-assistant

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

drydock-cli

Local-first TUI coding agent for Gemma 4 + llama.cpp. Built-in GraphRAG retrieval and cookbook of domain recipes. Install: pip install drydock-cli.

ameva-runtime

Unified Next-Gen Hardware Orchestration & AI Acceleration Runtime for Mobile & Edge

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

pyllamacpp

Python bindings for llama.cpp

termux-llamacpp

Production-grade, prebuilt GGUF LLM runtime, model manager & OpenAI-compatible server for Android Termux & ARM64.

ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

lmdk

Language Model Development Kit a.k.a. glorified ChatGPT wrapper

llmware

Unified framework for building enterprise RAG pipelines with small, specialized models

easy-llama

Python package wrapping llama.cpp for on-device LLM inference

cactus-compute

Quantization, kernels, runtime and inference engine for mobiles, wearables, smart home and robots.

r105

Rust-native AI harness for local and cloud OpenAI-compatible providers. Binaries for Linux x86_64/aarch64, Ubuntu/Debian, Arch, Fedora/RHEL, Alpine, FreeBSD amd64, macOS x86_64/Apple silicon, and Windows x86_64/ARM64.

otaku

LLM frontend for creative writing and roleplay

llama-cpp-agent

The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.

promptmask

Never give AI companies your secrets! A local LLM-based privacy filter for LLM users. Seamless integration with your existing AI tools as a Python library / OpenAI SDK replacement / API Gatetway / Web Server.

cappr

Completion After Prompt Probability. Make your LLM make a choice

exxa

Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and minimal learning curve.

ameva-vulkan-runtime

Unified Cross-Modal Vulkan GPU Acceleration Runtime & HAL for Mobile Android

claude-codex-local

Hit your limit? Need privacy? Just swap the model, everything else stays

translate-shell

Translate text by google, bing, youdaozhiyun, haici, stardict, openai, large language model of local machine, etc at same time from CLI, GUI (GNU/Linux, Android, macOS and Windows), REPL, python, shell and vim.