Distributed Training Python Packages

timm

The largest collection of PyTorch image encoders / backbones. Including train, eval, inference, export scripts, and pretrained weights -- ResNet, ResNeXT, EfficientNet, NFNet, Vision Transformer (ViT), MobileNetV4, MobileNet-V3 & V2, RegNet, DPN, CSPNet, Swin Transformer, MaxViT, CoAtNet, ConvNeXt, and more

paddlepaddle

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

skypilot

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

metaflow

Build, Manage and Deploy AI/ML Systems

torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

skypilot-nightly

Run, manage, and scale AI workloads on any AI infrastructure. Use one system to access & manage all AI compute (Kubernetes, Slurm, 20+ clouds, on-prem).

metaflow-stubs

Build, Manage and Deploy AI/ML Systems

paddlepaddle-gpu

PArallel Distributed Deep LEarning: Machine Learning Framework from Industrial Practice （『飞桨』核心框架，深度学习&机器学习高性能单机、分布式训练和跨平台部署）

torchx-nightly

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

paddlenlp

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

tool-helpers

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

fast-dataindex

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

trainy-skypilot-nightly

SkyPilot: An intercloud broker for the clouds

hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

codeflare-torchx

TorchX is a universal job launcher for PyTorch applications. TorchX is designed to have fast iteration time for training/research and support for E2E production ML pipelines when you're ready.

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

ddpw

A lightweight wrapper that bootstraps PyTorch's Distributed (Data) Parallel.

primus-lens-wandb-exporter

Primus-SaFE(Stability and Fault Endurance)

tsugi-mend

tsugiai-mend-sdk. Maximum-uplift cross-rack distributed-training reducer built on Decoupled DiLoCo + DES-LOC + async tensor parallelism + FALCON fail-slow mitigation. Patent-independent by deliberate construction; see LICENSE preamble.

fast-tokenizer-python

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

faster-tokenizer

Easy-to-use and powerful LLM and SLM library with awesome model zoo.

basetrainer

Pytorch分布式训练框架

tensorcircuit-ng

Next-gen AI-native tensor-network-based quantum software framework

telescope-ui

Real-time visualization dashboard for LLM training runs