Model Parallelism Python Packages

deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

colossalai

Making large AI models cheaper, faster and more accessible

colossalai-nightly

Making large AI models cheaper, faster and more accessible

torchgpipe

A GPipe implementation in PyTorch

torchshard

Slicing a PyTorch Tensor Into Parallel Shards

redco

No description available

custom-colossalai

An integrated large-scale model training system with efficient parallelization techniques

fleet-x

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。

fleet-lightning

No description available

adeepspeed

DeepSpeed library

veloce

WIP. Veloce is a low-code Ray-based parallelization library that makes machine learning computation novel, efficient, and heterogeneous.

paddle-fleet

飞桨大模型开发套件，提供大语言模型、跨模态大模型、生物计算大模型等领域的全流程开发工具链。