Blackwell Python Packages

nvidia-cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

tokenspeed-mla

TokenSpeed is a speed-of-light LLM inference engine.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-cpu-nightly

A high-throughput and memory-efficient inference and serving engine for LLMs

tensorrt-llm

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

tokenspeed-cutedsl-kda

TokenSpeed is a speed-of-light LLM inference engine.

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

lsglang

SGLang is a high-performance serving framework for large language models and multimodal models.

pygpukit

Minimal GPU runtime for Python - high-performance CUDA kernels, memory management, and LLM inference without heavy dependencies

gridbook

Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.

tokenspeed

TokenSpeed is a speed-of-light LLM inference engine.

tokenspeed-kernel-amd

TokenSpeed is a speed-of-light LLM inference engine.

dblcsgen

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-xft

A high-throughput and memory-efficient inference and serving engine for LLMs

ai-dynamo-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

pyptx

A Python DSL to write Nvidia PTX for Hopper and Blackwell in JAX and PyTorch

vllm-dgxspark-psychoscales

A high-throughput and memory-efficient inference and serving engine for LLMs

cutile-gpt

A complete GPT built entirely on NVIDIA cuTile: declarative GPU kernels in ~20 lines instead of ~150 hand-tuned CUDA, matching PyTorch speed in a ~10MB footprint

vllm-npu

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-musa

A high-throughput and memory-efficient inference and serving engine for LLMs