Sparse Attention Python Packages

natten

Fast Multi-dimensional Sparse Attention

hypercube-hopfield

Modern Hopfield associative memory on a hypercube graph. Sparse local-attention retrieval with exponential capacity. C++23 and Python SDKs.

native-sparse-attention-pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

flash-sparse-attn

Trainable fast and memory-efficient sparse attention

flash-dmattn

Trainable fast and memory-efficient sparse attention

nano-deepseek-v4

A compact, readable PyTorch reference implementation of the DeepSeek-V4 architecture (Flash and Pro), in the spirit of nanoGPT. ~4,300 LOC.

sparsepr

Training-free sparse-attention acceleration for video diffusion and world models

shadowkv

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference