Inference Acceleration Python Packages

sageattention

[ICLR2025, ICML2025, NeurIPS2025 Spotlight] Quantized Attention achieves speedup of 2-5x compared to FlashAttention, without losing end-to-end metrics across language, image, and video models.

mlx-teacache

TeaCache step-skipping for FLUX, Qwen-Image & Z-Image diffusion on Apple Silicon, in pure MLX

turbodiffusion

TurboDiffusion: 100–200× Acceleration for Video Diffusion Models

torch-nos

⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.

autonomi-nos

⚡️ A fast and flexible PyTorch inference server that runs locally, on any cloud or AI HW.