Self Attention Python Packages

mambavision

[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

keras-vision

Porting vision models to Keras 3 for easily accessibility. Contains MobileViT v1, MobileViT v2, fastvit

crabnet

Predict materials properties using only the composition information!

deberta

The implementation of DeBERTa

fastervit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

gcvit

Tensorflow 2.0 Implementation of GCViT: Global Context Vision Transformer. https://github.com/awsaf49/gcvit-tf

gsa-pytorch

Global Self-attention Network (GSA) - Pytorch

perceiver-io-pytorch

Unofficial Perceiver IO implementation

transformerx

TransformerX is a python library for building transformer-based models using ready-to-use layers.

self-attention-cv

Self-attention building blocks for computer vision applications in PyTorch

trphysx

Transformers for modeling physical systems

fixedvit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

arenets

Tensorflow-based framework which lists implementation of conventional neural network models (CNN, RNN-based) for Relation Extraction classification tasks as well as API for custom model implementation

keyatten

KeyAtten: Attention-based Zero-Shot Keyword & Keyphrase Extraction

rnn4ie

Chinese Information Extraction Toolkit。中文信息抽取工具

correctedvit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

fastervitfixed

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

graphattentionnetworks

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.