Sharding Python Packages

substrax

JAX/Flax NNX training infrastructure: device meshes and SPMD sharding, an Orbax checkpoint store, early stopping and callbacks, W&B and MLflow logging.

torchrec

Pytorch domain library for recommendation systems

torchrec-nightly

Pytorch domain library for recommendation systems

edgeproc-core

Decides which partition a vector belongs in when a vector index is split across tenants, users, or time windows: three routing strategies plus result merge-back, one runtime dependency (pydantic), no index of its own. 243 tests, 98% coverage.

torchrec-nightly-cpu

Pytorch domain library for recommendation systems

doorway

🚪 Utilities for working with files. Including easy downloading, fast hashing, stale file detection, dataset sharding, proxy downloading, file renaming

djanquiltdb

Multi-schema, multi-cluster, flexible horizontal database sharding for Django applications.

sharded-queue

strict order sharded job queue

djanquiltdb-plugin-postgres-objects

Multi-schema, multi-cluster, flexible horizontal database sharding for Django applications.

dbkit-sql

SQLAlchemy Core made enjoyable — sync + async, bounded pools, idempotency-gated retries, sharding, streaming, and observability. ORM-free, PostgreSQL-first.

django-sharding

A sharding library for Django

django-horizon

Simple database sharding (horizontal partitioning) library for Django applications.

torchrec-cpu

Pytorch domain library for recommendation systems

sharding

Sharding manager contract, and related software and tests

iginx

IginX -- An open-source clustering system for multi-dimensional scaling of standalone time series databases through generalized sharding.

melisa

MelisaPy is a cache-optimized Discord microframework for Python 3

redis-cluster-tools

Python CLI for Redis Cluster administration