Dataloader Python Packages

conecta

SQL to Arrow made easy.

cellmap-data

Utility for loading CellMap data for machine learning training, utilizing PyTorch, TensorStore, Xarray, and PyDantic.

jax-dataloader

Pytorch-like dataloaders for JAX.

continuum

A clean and simple data loading library for Continual Learning

datago

A natively parallel dataloader for Python, written in Rust. Serving data at GB/s speeds, while covering aspect ratio bucketing, crop and resize for image ML workloads.

scdataloader

a dataloader to work with large single cell datasets from lamindb

fewsoldataloader

Pytorch Dataloader for FewSOL

dynabatch

PyTorch/Hugging Face batching utility that sorts variable-length text by difficulty, then dynamically increases batch size on easier samples using a pre-trained VRAM predictor to improve GPU utilization and throughput while reducing OOM risk with fallback handling.

bucketsampler

Aspect ratio bucketing for diffusion training. PyTorch native, DDP correct, no training-framework lock-in.

bruker

A modular toolkit for Bruker MRI raw-data handling.

torchvtk

PyTorch volume toolkit. Efficient data loading, dataset conversions, visualization tools

lavender-data

Load & manage evolving datasets efficiently

torch-dataloader-utils

Efficient Data Loader Utils for loading data from structured sources into Pytorch

torch-waymo

PyTorch dataloader for Waymo Open Dataset

vdataset

Load video datasets to PyTorch DataLoader. (Custom Video Data set for PyTorch DataLoader)

datatap

Focus on Algorithm Design, Not on Data Wrangling

ukv

Multi-Modal Database replacing MongoDB, Neo4J, and Elastic with 1 faster ACID solution, with NetworkX and Pandas interfaces, and bindings for C 99, C++ 17, Python 3, Java, GoLang 🗄️

audiotree

Audio data loading and augmentations in JAX

ffcv-pl

[FFCV-PL] manage fast data loading with ffcv and pytorch lightning

light-dataloader

Much faster dataloader for datasets that are fully loaded into memory

lrp-pf-auc

Explain Neural Networks using Layer-Wise Relevance Propagation and evaluate the explanations using Pixel-Flipping and Area Under the Curve.

hydrax

A zero-copy multiprocess dataloader for Jax.

dloader

DataLoader implementation for Python asyncio

async-dataloader

Asyncio Dataloader for GraphQL.