Long Context Python Packages

megabyte-pytorch

Implementation of MEGABYTE, Predicting Million-byte Sequences with Multiscale Transformers, in Pytorch

loop-memory

Local-first, agent-agnostic memory system that closes the agent loop on any AI agent. Watches any agent's transcript directory on disk and distils long sessions into a curated wiki that re-injects into the next turn. Out of the box: hooks for Codex / Claude / Hermes / OpenClaw; SDK + generic watcher CLI for the rest.

ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

mlx-train-perf

Fused, logit-free linear-cross-entropy loss, RAM-fit planner, and benchmark harness for MLX fine-tuning on Apple Silicon

laya-finetune

Fine-tune Laya, the open System One decision model, for any typed-decision task: synthetic data, RLCD, calibration and long context.

aether-context

Give any local LLM a billion-token memory. Open-source, local-first context + memory engine for Ollama and AI agents — runs on your own machine, free. 🧠⚡

kvfit

Will a Hugging Face LLM fit your GPU or DGX? Plan weights, KV cache or recurrent state, context, TP/DP, and measured calibration

recurrent-memory-transformer-pytorch

Implementation of Recurrent Memory Transformer, Neurips 2022 paper, in Pytorch

infini-transformer-pytorch

Implementation of Infini-Transformer in Pytorch

mcp-long-context-reader

An MCP server for overcoming context window limitations when processing extensive documents.

perceiver-ar-pytorch

Implementation of Perceiver AR, Deepmind's new long-context attention network based on Perceiver architecture, in Pytorch

tina4-groundwire

Retrieval + verification in front of an LLM — ground a weak reader in your own source, off-GPU. Part of the tina4stack.

ringattention

Large Context Attention

megalodon-jax

jax rewrite for CEMA support + other training speedups.

nearlossless-context

Near-lossless long-context KV compression under fixed VRAM, critical-span retention, query-unknown discovery, and measured peak/quality Pareto.

tokenpack-rag

TokenPack packs long documents, codebases, PDFs, and folders into compact, evidence-dense LLM context using local embeddings, evidence scoring, and budget-aware selection.

three-layer-agent-memory

A filesystem-native, version-controllable long-memory paradigm for AI agents. Surface/Middle/Deep layers + recall/writeback/consolidate protocol. Pure Markdown, zero infra, drives agent self-evolution.

titans-memory

Titans in PyTorch: Infinite Context models that learn to remember. Faster than Transformers, smarter than RNNs, and fully HuggingFace compatible.

wlpworld

A filesystem-native, version-controllable long-memory paradigm for AI agents. Surface/Middle/Deep layers + recall/writeback/consolidate protocol. Pure Markdown, zero infra, drives agent self-evolution.

compactbench

Open benchmark for LLM context compaction methods — measures what survives when you replace conversation history with a compacted artifact. Multi-cycle drift, hidden ranked set.

hard-needle

Semantically hard multi-needle long-context data generator. Stop testing LLMs with random-password needles.

memory-garden

Local-first memory layer and memory skill for AI agents that mitigates long-context attention dilution with traceable retrieval, policy-aware memory lifecycle, and auditable forget. 面向 AI Agent 的本地优先记忆层与记忆 Skill，在应用层缓解长上下文注意力稀释与上下文遗忘，支持可追溯检索、策略化记忆生命周期与可审计遗忘

kakeyalattice

Discrete Kakeya cover for LLM KV cache: D4/E8 nested-lattice quantisation realising a Kakeya-style tube-cover over the direction sphere. 2.4x-2.8x compression at <1% perplexity loss on Qwen3, Llama-3, DeepSeek, GLM-4, Gemma. Drop-in transformers.DynamicCache. pip install kakeyalattice.

pageindex-condb

ConDB: The KV-Cache Native Context Database