Self Attention Python Packages

crabnet

Predict materials properties using only the composition information!

mambavision

[CVPR 2025] Official PyTorch Implementation of MambaVision: A Hybrid Mamba-Transformer Vision Backbone

fastervit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

morphottention

Mathematical Morphology-based self-attention module for PyTorch (CUDA) using Flash-style kernel fusion.

gcvit

Tensorflow 2.0 Implementation of GCViT: Global Context Vision Transformer

keras-vision

Porting vision models to Keras 3 for easily accessibility. Contains MobileViT v1, MobileViT v2, fastvit

self-attention-cv

Implementation of various self-attention mechanisms focused on computer vision. Ongoing repository.

deberta

The implementation of DeBERTa

gsa-pytorch

A Pytorch implementation of Global Self-Attention Network, a fully-attention backbone for vision tasks

keyatten

KeyAtten: Attention-based Zero-Shot Keyword & Keyphrase Extraction

trphysx

Transformers for modeling physical systems

perceiver-io-pytorch

Unofficial implementation of Perceiver IO

arenets

Tensorflow-based framework which lists attentive implementation of the conventional neural network models (CNN, RNN-based), applicable for Relation Extraction classification tasks as well as API for custom model implementation

fixedvit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

graphattentionnetworks

This package is a Tensorflow2/Keras implementation for Graph Attention Network embeddings and also provides a Trainable layer for Multihead Graph Attention.

correctedvit

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

transformerx

Flexible Python library providing building blocks (layers) for reproducible Transformers research (Tensorflow ✅, Pytorch 🔜, and Jax 🔜)

rnn4ie

Chinese Information Extraction Toolkit。中文信息抽取工具

fastervitfixed

[ICLR 2024] Official PyTorch implementation of FasterViT: Fast Vision Transformers with Hierarchical Attention

ssma

Structured State Matrix Architecture (SSMA) is a high-performance framework designed for efficient sequence modeling, combining structured state space models with adaptive attention mechanisms.