Speculative Decoding Python Packages

mtplx

2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

angelslim

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

aphrodite-engine

Large-scale LLM inference engine

intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

m5-infer

Extraordinary speed, extraordinary quality — MLX-based inference engine for Apple Silicon.

eagle-llm

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

opencac

Multi-agent orchestration CLI for AI coding tools. Chain Claude Code, Antigravity, and Codex with validated handoffs, JSONL audit logging, hybrid cloud/local routing, and speculative decoding for local LLMs.

angelslim-fork

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

specstream

Fast LLM inference with 2.8x speedup using speculative decoding