mixture-of-depths
The one-stop solution to easily integrate MoE & MoD layers into custom PyTorch code.