Mixture Of Models Python Packages

switch-transformers

Implementation of Switch Transformers from the paper: "Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity"

vllm-sr-sim

An open, programmable decision layer for models and compute.

vllm-semantic-router-bench

An open, programmable decision layer for models and compute.

mixturelib

The implementation of mixtures for different tasks.