On Device Inference Python Packages

foundry-local-sdk

Foundry Local Python SDK (v2): in-process Python bindings for the Foundry Local native runtime.

foundry-local-sdk-winml

Foundry Local Manager Python SDK: Control-plane SDK for Foundry Local.

squish-ai

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

aneforge

Python runtime for the Apple Neural Engine

quantcost

pip install quantcost — measure what quantization actually costs on your own hardware: speed, size, memory and perplexity for fp32/int8/int4. No PyTorch, no GPU. Submit your machine to the leaderboard.

metro-asr

Metro-ASR is a novel speech recognition architecture designed specifically for Egyptian Arabic with native English code-switching support. It uses a CTC-based non-autoregressive approach for fast, accurate, on-device inference — no decoder, no beam search required.