fsdp
Fault tolerance for PyTorch (HSDP, LocalSGD, DiLoCo, Streaming DiLoCo)
Train speculative decoding models effortlessly and port them smoothly to SGLang serving.