PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Attention Mechanism Python Packages

Python packages with the GitHub topic attention-mechanism. Sorted by relevance, with stars and monthly downloads.
lucidrains
x-transformers

A concise but complete full-attention transformer with a set of promising experimental features from various papers

1.5M 6K 507
lucidrains
vit-pytorch

Implementation of Vision Transformer, a simple way to achieve SOTA in vision classification with only a single transformer encoder, in Pytorch

118K 25K 3K
linto-ai
whisper-timestamped

Multilingual Automatic Speech Recognition with word-level timestamps and confidence

96K 3K 211
lucidrains
linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

67K 832 77
lucidrains
linformer

Implementation of Linformer for Pytorch

66K 307 27
lucidrains
flamingo-pytorch

Implementation of 🦩 Flamingo, state-of-the-art few-shot visual question answering attention net out of Deepmind, in Pytorch

25K 1K 66
lucidrains
reformer-pytorch

Reformer, the efficient Transformer, in Pytorch

23K 2K 252
lucidrains
tab-transformer-pytorch

Implementation of TabTransformer, attention network for tabular data, in Pytorch

20K 1K 131
kyegomez
zetascale

Build high-performance AI models with modular building blocks

18K 591 58
CyberZHG
keras-self-attention

Attention mechanism for processing sequential data that considers the context for each timestamp.

11K 657 117
lucidrains
enformer-pytorch

Implementation of Enformer, Deepmind's attention network for predicting gene expression, in Pytorch

11K 564 101
lucidrains
performer-pytorch

An implementation of Performer, a linear attention-based transformer, in Pytorch

9K 1K 150
lucidrains
en-transformer

Implementation of E(n)-Transformer, which incorporates attention mechanisms into Welling's E(n)-Equivariant Graph Neural Network

8K 226 31
lucidrains
alphafold2-pytorch

To eventually become an unofficial Pytorch implementation / replication of Alphafold2, as details of the architecture get released

8K 2K 266
densechen
activations

AReLU: Attention-based-Rectified-Linear-Unit

5K 62 8
lucidrains
x-transformers-rl

Implementation of a transformer for reinforcement learning using `x-transformers`

5K 73 6
lucidrains
dalle-pytorch

Implementation / replication of DALL-E, OpenAI's Text to Image Transformer, in Pytorch

5K 6K 643
lucidrains
tf-bind-transformer

A repository with exploration into using transformers to predict DNA ↔ transcription factor binding

4K 88 9
lucidrains
ring-attention-pytorch

Implementation of 💍 Ring Attention, from Liu et al. at Berkeley AI, in Pytorch

4K 547 36
lucidrains
se3-transformer-pytorch

Implementation of SE3-Transformers for Equivariant Self-Attention, in Pytorch. This specific repository is geared towards integration with eventual Alphafold2 replication.

4K 328 30
lucidrains
improving-transformers-world-model

Implementation of the new SOTA for model based RL, from the paper "Improving Transformer World Models for Data-Efficient RL", in Pytorch

4K 154 6
lucidrains
nystrom-attention

Implementation of Nyström Self-attention, from the paper Nyströmformer

4K 145 18
lucidrains
memorizing-transformers-pytorch

Implementation of Memorizing Transformers (ICLR 2022), attention net augmented with indexing and retrieval of memories using approximate nearest neighbors, in Pytorch

4K 644 47
lucidrains
coca-pytorch

Implementation of CoCa, Contrastive Captioners are Image-Text Foundation Models, in Pytorch

4K 1K 90
    • Data from PyPI, GitHub, ClickHouse, and BigQuery