Transformer Python Packages

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

faster-whisper

Faster Whisper transcription with CTranslate2

nvidia-cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

torchao

PyTorch native quantization for training and inference

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

lm-eval

A framework for few-shot evaluation of language models.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers

neuralforecast

Scalable and user friendly neural :brain: forecasting algorithms.

mmdet

OpenMMLab Detection Toolbox and Benchmark

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

mnn

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

text-generation

Large Language Model Text Generation Inference

multilspy

multilspy is a lsp client library in Python intended to be used to build applications around language servers.

pytorch-pretrained-bert

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

linformer

Implementation of Linformer for Pytorch

ptflops

Flops counter for neural networks in pytorch framework

mmsegmentation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.

flash-ansr

Fast Amortized Neural Symbolic Regression with Transformers and SimpliPy

fastt5

⚡ boost inference speed of T5 models by 5x & reduce the model size by 3x.