Gradient Clipping Python Packages

zclip

Official implementation of the paper: "ZClip: Adaptive Spike Mitigation for LLM Pre-Training".

zcls

Object Classification Training Framework

gradientnorm-autoclipper

Allows automatic gradient norm clipping. This feature can help to stabilize training in certain situations by limiting the magnitude of gradient updates. The implementation is inspired by the paper "AutoClip: Adaptive Gradient Clipping for Source Separation Networks"

freegrad

PyTorch extension for alternative backward rules and gradient transforms (STE, gradient jamming, non-standard activations).