Gradient Clipping Python Packages

zclip

Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".

zcls

Object Classification Training Framework

freegrad

PyTorch extension for alternative backward rules and gradient transforms (STE, gradient jamming, non-standard activations).

gradientnorm-autoclipper

Allows automatic gradient norm clipping. This feature can help to stabilize training in certain situations by limiting the magnitude of gradient updates. The implementation is inspired by the paper "AutoClip: Adaptive Gradient Clipping for Source Separation Networks"