Llamacpp Python Packages

spark-nlp

State of the Art Natural Language Processing

xinference

Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.

khoj

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

gptme

Your agent in your terminal, equipped with local tools: writes code, uses the terminal, browses the web. Make your own persistent autonomous agent on top!

khoj-assistant

Your AI second brain. Self-hostable. Get answers from the web or your docs. Build custom agents, schedule automations, do deep research. Turn any online or local LLM into your personal, autonomous AI (gpt, claude, gemini, llama, qwen, mistral). Get started - free.

jang

JANG — GGUF for MLX. YOU MUST USE JANG_Q RUNTIME. Adaptive Mixed-Precision Quantization + Runtime for Apple Silicon

llama-cpp-agent

The llama-cpp-agent framework is a tool designed for easy interaction with Large Language Models (LLMs). Allowing users to chat with LLM models, execute structured function calls and get structured output. Works also with models not fine-tuned to JSON output and function calls.

ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

webscout

Webscout is the all-in-one search and AI toolkit you need. Discover insights with Yep.com, DuckDuckGo, and Phind; access cutting-edge AI models; transcribe YouTube videos; generate temporary emails and phone numbers; perform text-to-speech conversions; and much more!

pyllamacpp

Python bindings for llama.cpp

toolmate

ToolMate AI, developed by Eliran Wong, is a cutting-edge AI companion that seamlessly integrates agents, tools, and plugins to excel in conversations, generative work, and task execution. Supports custom workflow and plugins to automate multi-step actions.

llmware

Unified framework for building enterprise RAG pipelines with small, specialized models

claude-codex-local

Hit your limit? Need privacy? Just swap the model, everything else stays

easy-llama

Python package wrapping llama.cpp for on-device LLM inference

i4-0-client-py

Fully modular AI server and client

toolmate-lite

ToolMate AI, developed by Eliran Wong, is a cutting-edge AI companion that seamlessly integrates agents, tools, and plugins to excel in conversations, generative work, and task execution. Supports custom workflow and plugins to automate multi-step actions.

toolmate-android

ToolMate AI, developed by Eliran Wong, is a cutting-edge AI companion that seamlessly integrates agents, tools, and plugins to excel in conversations, generative work, and task execution. Supports custom workflow and plugins to automate multi-step actions.

translate-shell

Translate text by google, bing, youdaozhiyun, haici, stardict, openai, large language model of local machine, etc at same time from CLI, GUI (GNU/Linux, Android, macOS and Windows), REPL, python, shell and vim.

lexicont

Lightweight LLM agent for text moderation with deterministic pipeline: profanity rules → ML → LLM. Production-grade, YAML configurable. Uses glin-profanity + rapidfuzz + detoxify with Qwen LLM agent (llamacpp/Ollama) + RAG. Early-stop logic. Pure Python, CLI/API/SDK.

urban-worm

Workflow of reproducible multimodal inference for urban environment evaluation.

llama-optimus

Lightweight Python tool using Optuna for tuning llama.cpp flags: towards optimal tok/s for your machine

fastapi-gen

Build LLM-enabled FastAPI applications without build configuration.

exxa

Exa - Pytorch

developergpt

DeveloperGPT is a LLM-powered command line tool that enables natural language to terminal commands and in-terminal chat.