Gradient Descent Python Packages

agentdescent

Gradient descent, but the parameters are agents — a parallel, asynchronous framework for self-evolving agents (skills, prompts, harnesses). Diffs are the gradients; the aggregator is the optimizer.

genoxide

Optimization for Rust and Python: genetic algorithms, CMA-ES, differential evolution, multi-objective, L-BFGS-B, Adam, MMA and more. The same results on every platform.

fastcpd

Fast Change Point Detection

networks

Library which can be used to build feed forward NN, Convolutional Nets, Linear Regression, and Logistic Regression Models.

rose-opt

🌹 Rose: Range-Of-Slice Equilibration PyTorch optimizer. Stateless optimization through range-normalized gradient updates.

tangent-ad

Source-to-Source Debuggable Derivatives in Pure Python

neural-tangents

Fast and Easy Infinite Neural Networks in Python

nets

Recreating PyTorch from scratch, using Numpy. Supports FCN, CNN, RNN layers.

grande

(ICLR 2024) GRANDE: Gradient-Based Decision Tree Ensembles

torchzero

Modular optimization library for PyTorch (work-in-progress).

ml-python

Easy to use high level python library for popular machine learning algorithms. Has in-built support for graphing and optimizers based in C++.

spsa

A simple implementation of SPSA with automatic learning rate tuning

descent

First-order optimization tools

mlektic

A simple machine learning python library.

yodf

'Hello, World!' Forward Mode Autdiff library with Tensorflow 1 like interface.

torchimize

numerical optimization in pytorch

simpliml

SimpliML is a versatile machine learning library designed to be a one-stop solution for the entire data lifecycle. Whether you're preparing raw data or deploying advanced predictive models, SimpliML simplifies every step of the machine learning process

mlr-gd

Multiple linear regression by gradient descent.

mangograd

🥭 A PyTorch-shaped, NumPy-powered autograd engine you can actually read. Born from watching Karpathy's micrograd while eating a mango.

dlfs

Implementation from scratch (using numpy arrays) of a framework based on keras interface which allows to build and train Fully Connected Networks and Convolutional Neural Networks (CNNs).

gradient-equilibrum

An implementation of an Gradient Algorithm that fines the equilibrium instead of the minimum or maximum

psd-optimizer

Perturbed Saddle-escape Descent (PSD): a first-order optimizer that escapes strict saddle points in nonconvex problems.

waveletml

WaveletML: A Scalable and Extensible Wavelet Neural Network Framework

zij

A canon of deep learning optimizers.