Transformer Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

faster-whisper

Faster Whisper transcription with CTranslate2

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

torchao

PyTorch native quantization and sparsity for training and inference

nvidia-cudnn-frontend

cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it

lm-eval

A framework for few-shot evaluation of language models.

onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

mmdet

OpenMMLab Detection Toolbox and Benchmark

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers

neuralforecast

Scalable and user friendly neural :brain: forecasting algorithms.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

mnn

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

text-generation

Large Language Model Text Generation Inference

pytorch-pretrained-bert

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

ptflops

Flops counter for neural networks in pytorch framework

torchscale

Foundation Architecture for (M)LLMs

auto-gptq

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

linear-attention-transformer

Transformer based on a variant of attention that is linear complexity in respect to sequence length

linformer

Implementation of Linformer for Pytorch

mmsegmentation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.