Residuals Python Packages

hyper-connections

Attempt to make multiple residual streams from Bytedance's Hyper-Connections paper accessible to the public

deep-cross-attention

Deep Cross Attention Language Model

attn-res

A clean, single-file PyTorch implementation of Attention Residuals (Kimi Team, MoonshotAI, 2026), integrated with Grouped Query Attention (GQA), SwiGLU feed-forward networks, and Rotary Position Embeddings (RoPE).

openfoam-residuals

A Python interface to parse and plot OpenFOAM residuals