open-implementation
Single-file PyTorch implementation of Attention Residuals (Kimi Team, 2026) with GQA, SwiGLU, and RoPE.
Paper - Pytorch