Self Attention Python Packages

morphottention

Mathematical Morphology-based self-attention module for PyTorch (CUDA) using Flash-style kernel fusion.

deberta

The implementation of DeBERTa

mambavision

[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

fastervit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

crabnet

Predict materials properties using only the composition information!

keras-vision

Porting vision models to Keras 3 for easily accessibility. Contains MobileViT v1, MobileViT v2, fastvit

gcvit

Tensorflow 2.0 Implementation of GCViT: Global Context Vision Transformer

gsa-pytorch

A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks

keyatten

Attention-based keyword extraction with ordered semantic word weights

perceiver-io-pytorch

Unofficial implementation of Perceiver IO

self-attention-cv

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

transformerx

Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

trphysx

Transformers for modeling physical systems

arenets

Tensorflow-based framework which lists attentive implementation of the conventional neural network models (CNN, RNN-based), applicable for Relation Extraction classification tasks as well as API for custom model implementation

fixedvit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

graphattentionnetworks

A Graph Attention Framework for extracting Graph Attention embeddings and implementing Multihead Graph Attention Networks

correctedvit

fixed version of fastervit implementation https://github.com/NVlabs/FasterViT

fastervitfixed

fixed version of fastervit implementation https://github.com/NVlabs/FasterViT

rnn4ie

Chinese Information Extraction Toolkit。中文信息抽取工具