Transformer Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

transformers

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

faster-whisper

Faster Whisper transcription with CTranslate2

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

nvidia-cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

torchao

PyTorch native quantization and sparsity for training and inference

lm-eval

A framework for few-shot evaluation of language models.

onnx2tf

A tool for converting ONNX files to LiteRT/TFLite/TensorFlow, PyTorch native code (nn.Module), TorchScript (.pt), state_dict (.pt), Exported Program (.pt2), and Dynamo ONNX. It also supports direct conversion from LiteRT to PyTorch.

curated-transformers

🤖 A PyTorch library of curated Transformer models and their composable components

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

conformer

Implementation of the convolutional module from the Conformer paper, for use in Transformers

mmdet

OpenMMLab Detection Toolbox and Benchmark

neuralforecast

Scalable and user friendly neural :brain: forecasting algorithms.

text-generation

Large Language Model Text Generation Inference

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

mnn

MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.

pytorch-pretrained-bert

🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.

soup2dict

Transforms BeautifulSoup soup to python dict or json

torchscale

Foundation Architecture for (M)LLMs

ptflops

Flops counter for neural networks in pytorch framework

auto-gptq

An easy-to-use LLMs quantization package with user-friendly apis, based on GPTQ algorithm.

mblt-model-zoo

Mobilint Model Zoo Project

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

mmsegmentation

OpenMMLab Semantic Segmentation Toolbox and Benchmark.