Long Context Python Packages

megabyte-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

infini-transformer-pytorch

Implementation of Infini-Transformer in Pytorch

nexusquant-kv

Training-free KV cache compression via E8 lattice quantization; fits longer context in the same VRAM

kakeyalattice

Discrete Kakeya cover for LLM KV cache: D4/E8 nested-lattice quantisation realising a Kakeya-style tube-cover over the direction sphere. 2.4x-2.8x compression at <1% perplexity loss on Qwen3, Llama-3, DeepSeek, GLM-4, Gemma. Drop-in transformers.DynamicCache. pip install kakeyalattice.

perceiver-ar-pytorch

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

tokenpack-rag

Query-aware semantic chunk selection under LLM context-window budgets.

titans-memory

PyTorch implementation of Titans: Learning to Memorize at Test Time (Behrouz, Zhong & Mirrokni, 2024)

mcp-long-context-reader

A tool to help agents read and query long documents.

ringattention

RingAttention for Transformers with Arbitrarily Large Context.

compactbench

Open benchmark for LLM context compaction methods — measures what survives when you replace conversation history with a compacted artifact. Multi-cycle drift, hidden ranked set.

memory-garden

记忆花园 Memory Garden: local-first, auditable memory layer for AI agents, also usable as a memory skill

replm

Recursive Language Models — process arbitrarily long prompts by offloading context into a REPL and enabling symbolic recursion via sub-LLM calls

pageindex-condb

KV-cache native context database with reasoning-based tree retrieval

hard-needle

Semantically hard multi-needle long-context data generator

liah

needle in a haystack for LLMs

shadowkv

[ICML 2025 Spotlight] ShadowKV: KV Cache in Shadows for High-Throughput Long-Context LLM Inference