Distributed Training Python Packages

timm

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

skypilot

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

paddlepaddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

metaflow

Build, Manage and Deploy AI/ML Systems

skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

tensorcircuit-nightly

The next-gen AI-native tensor-network-based quantum software framework

metaflow-stubs

Build, Manage and Deploy AI/ML Systems

paddlepaddle-gpu

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

trainy-skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

tool-helpers

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

fast-dataindex

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

basetrainer

Pytorch分布式训练框架

codeflare-torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

telescope-ui

Real-time observability dashboard for the Telescope RL post-training framework. Monitor metrics, rollouts, traces, GPU infrastructure, and evals at scale.

adaptdl

Dynamic-resource trainer and scheduler for deep learning

adaptdl-sched

Dynamic-resource trainer and scheduler for deep learning

fast-tokenizer-python

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

openembedding

OpenEmbedding is an open source framework for Tensorflow distributed training acceleration.

atorch

DLRover: An Automatic Distributed Deep Learning System