Memory Optimization Python Packages

mlx-flash

Run AI models too large for your Mac's memory — at near-full speed. Intelligent expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon.

mlx-train-perf

Fused, logit-free linear-cross-entropy loss, RAM-fit planner, and benchmark harness for MLX fine-tuning on Apple Silicon

block-streamer

Stream PyTorch model blocks from NVMe or pinned CPU memory with a bounded GPU residency budget. LoRA-finetune models far larger than host RAM on one GPU - Qwen3-VL-32B and gpt-oss-120b train in under 6 GB of VRAM.

mlx-dfloat

Run DFloat11 losslessly compressed BF16 checkpoints on Apple Silicon with MLX and a Metal decode kernel; today that runs FLUX.1 on a 32 GB Mac with weights about 30% smaller

dfreduce

Automatically reduce the memory size of any pandas dataframe based on downcasting bit types efficiently

diet-pandas

Automated memory optimization for Pandas DataFrames. Reduces memory by 50-80% and loads CSV 5-10x faster. Drop-in replacement that returns standard DataFrames - works with sklearn, matplotlib, and all your favorite libraries. Safe, fast, and zero refactoring required.

foldpipe

Bounded-memory asynchronous streaming for native PyTorch/PyG molecular-data shards, with remote prefetch from Hugging Face and Google Drive.

langchain-turboquant

LangChain VectorStore with TurboQuant compression (ICLR 2026) - 6x memory reduction, training-free, no GPU required. The first LangChain integration for Google Research's TurboQuant algorithm.

ultralytics-autoimport

Lightweight Python lazy imports that defer module loading to reduce startup time and initial memory use.

tqai

TurboQuant KV cache compression for local LLM inference — 80% memory savings, near-zero quality loss on 8B+ models. PyTorch + MLX (Apple Silicon). Based on arXiv:2504.19874 (Google Research, ICLR 2026).

fastflash-attention

Drop-in exact bf16 flash-attention for CUDA with a deterministic backward, tuned for Blackwell (sm_120 / RTX 5090).

superstring

A fast and memory-optimized string library for heavy-text manipulation in Python

sysplug

GPU-aware hyperparameter advisor: predict memory & throughput, then get an OOM-safe batch size, LR, precision & parallelism