Activation Functions Python Packages

activations

AReLU: Attention-based-Rectified-Linear-Unit

siren-pytorch

Pytorch implementation of SIREN - Implicit Neural Representations with Periodic Activation Function

torchact

TorchAct, collection of activation function for PyTorch. https://pypi.org/project/torchact/

siren-torch

PyTorch implementation of Sinusodial Representation networks (SIREN)

pytorch-logit-logic

Logit-space logical activation functions for pytorch

freegrad

PyTorch extension for alternative backward rules and gradient transforms (STE, gradient jamming, non-standard activations).

glu-tf

An easy-to-use library for GLU (Gated Linear Units) and GLU variants in TensorFlow.

acttensor-tf

Activation Functions for TensorFlow

jaxdiversity

Jax implementation of metalearning diversity paper

sigmoidcolon

:poop: Sigmoid Colon: The biologically inspired activation function.