Local Ai Python Packages

mlx-vlm

MLX-VLM is a package for inference and fine-tuning of Vision Language Models (VLMs) on your Mac using MLX.

foundry-local-sdk

Foundry Local Python SDK (v2): in-process Python bindings for the Foundry Local native runtime.

laya-mlx

Native MLX runtime for Laya typed decision models — 7–14 ms short decisions on M3 Max. No text generation, PyTorch, or cloud API.

rapid-mlx

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

knowledge-rag

Local RAG MCP server for Claude Code — hybrid search (semantic + BM25), cross-encoder reranking, 13 MCP tools, 20 format parsers. Zero external servers, zero API keys.

soup-cli

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

lilbee

The whole local AI stack in one executable: it runs and manages local AI models across every GPU, and it's a search engine you can talk to, with cited answers from your files, code, and the web. MCP server for coding agents, web crawler, TUI, CLI, REST API, Python library. No Ollama or LM Studio needed, works with both.

localharness

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harness (detailed event steams, logs, and performance data) or quickly replace closed source one (featuring out of the box plugins for web search, image gen, memory, auto research and more in dev).

mtplx

The fastest way to run Qwen 3.8 Flash Next, Qwen 3.8 27B and Ternary Bonsai 2 27B on a Mac: 125 tok/s in OpenCode on an M5 Max, and a 27B model on 16 GB Macs. Native MTP speculative decoding on Apple Silicon, exact at any temperature. OpenAI and Anthropic compatible local server.

imap-cleanup-tool

NOW AVAILABLE on MICROSOFT STORE - Bulk-delete or move IMAP emails by AI Cleanup (NEW), sender, domain, or nested rules - CLI + Web GUI, stdlib-only Python. Gmail support, scheduling. Windows/Linux/MacOS supported platforms.

dflash-console

Local Windows control panel for llama-server and DFlash speculative-decoding engines.

jaketts

Local, offline text-to-speech for macOS using Kokoro-82M — GUI + CLI, 54 voices, no cloud API.

inferhost

Self-hosted multi-modal AI server: chat/vision LLMs (Qwen, Llama, Gemma, DeepSeek), text-to-speech (OuteTTS, Qwen3-TTS) and image generation (SDXL, Flux, Z-Image, Qwen-Image) behind one OpenAI-compatible endpoint. Wraps llama.cpp + stable-diffusion.cpp, auto-fetches GGUF models from Hugging Face, VRAM hot-swap, speculative decoding, no compiling.

laya-coreml

Local Laya typed decisions on Apple Core ML and Neural Engine. Validated ports, ~5 ms short decisions on M3 Max, reproducible speed and energy benchmarks.

luar

LUAR (Local Utility for Automated Reviews): drop a spreadsheet, ask choice/yes-no questions, get answers per row from a local decision model (Laya). No API keys, nothing leaves your machine.

ltcai

Local-first private AI memory layer / Digital Brain for conversations, documents, decisions, and model-agnostic knowledge.

l2s1-sdk

jev-style adaptor for llama.cpp interface

algo-cli-runtime

Agent runtime for tools, durable context, and verified work

foq

⚡ Foq — the FREE, local, open-source alternative to Jev. Typed System 1 decisions in ~25 ms — no waitlist, no cloud, no per-token cost. foq.fr

guaardvark

Self-hosted AI studio on your own GPU: chat with your files (RAG), screen and browser agents, coding swarms, LoRA training, an MCP server for Claude Code and Cursor, and local video, image, music and voice generation. No telemetry.

skillware

A Python framework for modular, self-contained skill management for machines.

nanorecall

100% Private, Zero-Cloud, Bare-Metal Desktop Memory & Screen Search Engine, Native MCP Server (nanorecall-mcp).

whisper-local

Free, open-source, 100% offline AI dictation for Windows & macOS. Wispr Flow / Dragon alternative. Push-to-talk hotkey, voice commands, transforms, sub-second latency. Powered by Whisper. No cloud, no subscription, no telemetry.

llmrig

Know what your rig can run. Open compatibility and performance intelligence for local AI.