Transformer Architecture Python Packages

zetascale

Build high-performance AI models with modular building blocks

sockeye

Sequence-to-sequence framework with a focus on Neural Machine Translation based on PyTorch

dantinox

DantinoX: A modular, memory-efficient Transformer implementation in JAX/Flax NNX. Includes Sparse MoE, GQA, Sliding Window Attention, Gradient Accumulation and Checkpointing

ernie

Simple State-of-the-Art BERT-Based Sentence Classification with Keras / TensorFlow 2. Built with HuggingFace's Transformers.

gptmed

pip install gptmed

joeynmt

Minimalist NMT for educational purposes

gpt3-torch

An implementation of the base GPT-3 Model architecture from the paper by OPENAI "Language Models are Few-Shot Learners"

seq2seqsharp

Seq2SeqSharp is a tensor based fast & flexible deep neural network framework written by .NET (C#). It has many highlighted features, such as automatic differentiation, different network types (Transformer, LSTM, BiLSTM and so on), multi-GPUs supported, cross-platforms (Windows, Linux, MacOS), multimodal model for text and images and so on.

triformer

Transformers components but in Triton

openlanguagemodel

Readable, composable PyTorch library for language models: build Llama, Qwen, Gemma, DeepSeek, Kimi and 20+ other architectures from plain nn.Modules, then train them on CPU, one GPU, or multi-GPU.

pyromusa-ai

PyRomusa AI is a lightweight Python framework for building and training chatbots easily and fast, using your own examples or prepared datasets. 🤖⭐

gpt4-torch

An implementation of the base GPT-3 Model architecture from the paper by OPENAI "Language Models are Few-Shot Learners"

bangla-ocr

Implementation of the paper 'Towards Full page Offline Bangla Handwritten Text Recognition using Image-to-Sequence Architecture'. For details, please read the README section.

mmm-zeta

A novel implementation of fusing ViT with Mamba into a fast, agile, and high performance Multi-Modal Model. Powered by Zeta, the simplest AI framework ever.

padie-extended

An open-source Python package that uses AI to predict Nigerian languages, including English, Pidgin, Yoruba, Hausa, and Igbo.

aot-x

My implementation of "Algorithm of Thoughts: Enhancing Exploration of Ideas in Large Language Models"

free-transformer

Free Transformer Architecture - an extension of the decoder Transformer that conditions its generative process on random latent variables which are learned without supervision

swin-transformer-pytorch

Implementation of the Swin Transformer in PyTorch.

minimal-slt

Minimal Sign Language Translation package based on Hugging Face

transformer-builder

This package provides builder-like API to create really flexible transformers using PyTorch