Distributed Training Python Packages

timm

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

skypilot

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

paddlepaddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

metaflow

Build, Manage and Deploy AI/ML Systems

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

skypilot-nightly

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

metaflow-stubs

Build, Manage and Deploy AI/ML Systems

torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

paddlepaddle-gpu

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

substrax

JAX/Flax NNX training infrastructure: device meshes and SPMD sharding, an Orbax checkpoint store, early stopping and callbacks, W&B and MLflow logging.

trainy-skypilot-nightly

The AI Compute Platform for frontier teams. SkyPilot turns fragmented AI compute into one AI supercomputer, so frontier AI teams build custom intelligence faster.

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

tool-helpers

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

fast-dataindex

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

tensorcircuit-ng

Next-gen AI-native tensor-network-based quantum software framework

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

neuroshard-ai

Open-source native blockchain for verified neural training. Experimental CPU testnet, public ledger, and locally controlled keys.

flagquantum

Sharded, differentiable quantum simulation and training in PyTorch, reaching domestic accelerators through FlagOS.

basetrainer

Pytorch分布式训练框架

adaptdl

Resource-adaptive cluster scheduler for deep learning training.

telescope-ui

Real-time observability dashboard for the Telescope RL post-training framework. Monitor metrics, rollouts, traces, GPU infrastructure, and evals at scale.

fast-tokenizer-python

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

oat-llm

🌾 OAT: A research-friendly framework for LLM online alignment, including reinforcement learning, preference learning, etc.