PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Mixture Of Experts Python Packages

Python packages with the GitHub topic mixture-of-experts. Sorted by relevance, with stars and monthly downloads.
NVIDIA
nvidia-cudnn-frontend

cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it

3.5M 723 153
deepspeedai
deepspeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

1.2M 42K 5K
lucidrains
mixture-of-experts

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

47K 860 71
relf
egobox

Efficient global optimization toolbox in Rust: bayesian optimization, mixture of gaussian processes, sampling methods

37K 173 10
SMTorg
smt

Surrogate Modeling Toolbox

29K 876 227
codelion
optillm

Optimizing inference proxy for LLMs

7K 4K 319
theoddden
terradev-cli

NUMA-aware GPU provisioning and orchestration for stateless MoE workloads of all sizes

4K 11 2
learning-at-home
hivemind

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

3K 2K 227
lucidrains
peer-pytorch

PEER - Pytorch

3K 136 7
brontoguana
krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

3K 452 26
lucidrains
st-moe-pytorch

Implementation of ST-Moe, the latest incarnation of MoE after years of research at Brain, in Pytorch

2K 384 33
wuwangzhang1216
abliterix

Automated alignment adjustment for LLMs — direct steering, LoRA, and MoE expert-granular abliteration, optimized via multi-objective Optuna TPE.

2K 220 40
PR0CK0
dissenter

Multi-LLM debate engine for complex questions — surface disagreement, synthesize decisions

2K 1 0
kyegomez
switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

1K 141 18
szibis
mlx-flash

Run AI models too large for your Mac's memory — expert caching, speculative execution, and 15+ research techniques for MoE inference on Apple Silicon

1K 2 0
jaisidhsingh
pytorch-mixtures

The one-stop solution to easily integrate MoE & MoD layers into custom PyTorch code.

1K 27 1
lucidrains
soft-moe-pytorch

Soft MoE - Pytorch

978 345 10
lucidrains
sinkhorn-router-pytorch

Self contained pytorch implementation of a sinkhorn based router, for mixture of experts or otherwise

769 40 0
lucidrains
mixture-of-attention

Mixture of Attention

495 122 4
scouzi1966
mlxlmprobe

Universal probing and interpretability tool for MLX language models on Apple Silicon

281 3 0
michaelellis003
lmxlab

Transformer language models on Apple Silicon with MLX

279 1 0
cgrtml
neural-trees

sklearn-compatible PyTorch implementations of Soft Decision Trees, HMoE, and classifier comparison tests (5×2cv F-test). pip install neural-trees

235 23 3
Leeroo-AI
mergoo

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.

227 513 33
andriygav
mixturelib

The implementation of mixtures for different tasks.

213 2 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery