Local Inference Python Packages

localharness

An open-source, coding agent harness designed specifically for local LLMs. Designed as everything a user needs to jumpstart their own harness (detailed event steams, logs, and performance data) or quickly replace closed source one (featuring out of the box plugins for web search, image gen, memory, auto research and more in dev).

edgejev

离线可用的本地类型化决策：4 核 CPU 单题 15.6ms。Local & offline Jev / System One inference on CPU — ONNX + INT8, no torch at runtime. 支持 laya / kev / PlayJev

stuntd

Local proxy that learns your app's typed LLM decisions and answers them with a Laya head. Jev and OpenAI compatible.

projectdavid-platform

A single pip installed package will orchestrate a production ready instance of the AI stack in any environment

kiapi

Unified local inference API server for LLM agents on Apple Silicon (MLX).

microloop

Behavior JIT for production AI. Turn verified repeated decisions into qualified local software.

notolog

Notolog Markdown Editor

mlx-serve

Local inference server for Apple Silicon — hot-swaps MLX models (LLM, vision, embeddings, TTS, STT) via OpenAI API

alcove-search

Search your own documents locally. No server, no sign-up, no data leaves your machine.

hippo-llm

Local LLM inference + embedding & search in one package. Run 30B on consumer hardware, RAG without ChromaDB.

kiapi-relay

Unified local inference API server for LLM agents on Apple Silicon (MLX).

kiapi-proxy

Unified local inference API server for LLM agents on Apple Silicon (MLX).