Speculative Decoding Python Packages

angelslim

Model compression toolkit engineered for enhanced usability, comprehensiveness, and efficiency.

mtplx

2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

intel-extension-for-transformers

⚡ Build your chatbot within minutes on your favorite device; offer SOTA compression techniques for LLMs; run LLMs efficiently on Intel Platforms⚡

aphrodite-engine

Large-scale LLM inference engine

flashspec

Adaptive speculative-decoding inference engine with Triton-optimised verification and online bandit draft selection.

tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

eagle-llm

Official Implementation of EAGLE-1 (ICML'24), EAGLE-2 (EMNLP'24), and EAGLE-3 (NeurIPS'25).

m5-infer

Extraordinary speed, extraordinary quality — MLX-based inference engine for Apple Silicon.

angelslim-fork

A toolkit for compress llm model.

opencac

OpenCAC: Claude Code, Antigravity, and Codex working together across cloud APIs and local models.

specstream

Fast LLM inference with 2.8x speedup using speculative decoding