Flash Attention Python Packages

nvidia-cudnn-frontend

cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it

ffpa-attn

🤖FFPA: Extends FlashAttention-2 (forward & backward) via Split-D for large headdim, delivering 1.5~3×↑🎉 speedup over SDPA.

flash-sparse-attn

Trainable fast and memory-efficient sparse attention

easywheels

Smart GPU wheel installer. Auto-detects CUDA, GPU, torch, and Python.

flashmha

An simple pytorch implementation of Flash MultiHead Attention

inf-cl

[CVPR 2025 Highlight] The official CLIP training codebase of Inf-CL: "Breaking the Memory Barrier: Near Infinite Batch Size Scaling for Contrastive Loss". A super memory-efficiency CLIP training scheme.

flash-sinkhorn

Sinkhorn optimal transport kernels in PyTorch + Triton (squared Euclidean, no cost matrix materialization).

gpkg

GPU package manager — find prebuilt CUDA wheels, build missing ones

jax-flash-attn2

A flexible and efficient implementation of Flash Attention 2.0 for JAX, supporting multiple backends (GPU/TPU/CPU) and platforms (Triton/Pallas/JAX).

flash-attention-triton

Cross-platform FlashAttention-2 Triton implementation for Turing+ GPUs with custom configuration mode

flash-dmattn

Trainable fast and memory-efficient sparse attention