Ollama Alternative Python Packages

rapid-mlx

Rapid-MLX is an open-source (Apache 2.0) OpenAI- and Anthropic-compatible LLM inference server for Apple Silicon, built on MLX, focused on reliable tool calling for coding agents. Up to 4× faster than Apple's MLX (mlx-lm) on the same weights.

make-localllm-easier

One command runs the best local LLM your GPU can handle: model picked from measured multilingual benchmarks, llama.cpp tuned for AMD, NVIDIA, Intel and Apple GPUs.

squish-ai

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

hippo-llm

Local LLM inference + embedding & search in one package. Run 30B on consumer hardware, RAG without ChromaDB.