Optimizers Python Packages

prodigy-plus-schedule-free

Prodigy and Schedule-Free, together at last.

torch-optimi

Fast, Modern, and Low Precision PyTorch Optimizers

opytimizer

🐦 Opytimizer is a Python library consisting of meta-heuristic optimization algorithms.

adafactor8bit

Memory-efficient PyTorch optimizers with Adaptive Log-Space quantization, fused CUDA kernels, and Adafactor, CAME, APOLLO support.

submodlib-py

Summarize Massive Datasets using Submodular Optimization

rose-opt

🌹 Rose: Range-Of-Slice Equilibration PyTorch optimizer. Stateless optimization through range-normalized gradient updates.

labml-nn

🧑‍🏫 60+ Implementations/tutorials of deep learning papers with side-by-side notes 📝; including transformers (original, xl, switch, feedback, vit, ...), optimizers (adam, adabelief, sophia, ...), gans(cyclegan, stylegan2, ...), 🎮 reinforcement learning (ppo, dqn), capsnet, distillation, ... 🧠

cgds

Package for CGD and ACGD optimizers

keras-radam

RAdam implemented in Keras & TensorFlow

keras-rectified-adam

RAdam implemented in Keras & TensorFlow

torchzero

Modular optimization library for PyTorch (work-in-progress).

gefen-x

Gefen-X: A Low Memory Optimized Stochastic Optimizer (Fork)

keras-adamw

Keras/TF implementation of AdamW, SGDW, NadamW, Warm Restarts, and Learning Rate multipliers

rangerlite

A pytorch implementation of the Lookahead optimizer with RAdam as inner optimizer

optiml

Optimizers for/and sklearn compatible Machine Learning models

tensorflow-radam

RAdam implemented in Keras & TensorFlow

gradient-centralization-tf

Instantly improve your optimizer with just 2 lines of code!

keras-lookahead

Lookahead mechanism for optimizers in Keras.

zij

A canon of deep learning optimizers.

lion-tf

Lion - EvoLved Sign Momentum w/ New Optimizer API in TensorFlow 2.11+