Sparse Attention Python Packages

natten

Fast Multi-dimensional Sparse Attention

native-sparse-attention-pytorch

Implementation of the sparse attention pattern proposed by the Deepseek team in their "Native Sparse Attention" paper

hypercube-hopfield

Modern Hopfield associative memory on a hypercube graph. Sparse local-attention retrieval with exponential capacity. C++23 and Python SDKs.

flash-sparse-attn

Trainable fast and memory-efficient sparse attention

flash-dmattn

Trainable fast and memory-efficient sparse attention

shadowkv

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference