Blackwell Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

nvidia-cudnn-frontend

cudnn_frontend provides a c++ wrapper for the cudnn backend API and samples on how to use it

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

tokenspeed-mla

TokenSpeed is a speed-of-light LLM inference engine.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

tensorrt-llm

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

penguin-burner

Nvidia ultimate undervolting companion on Linux. Now with a nice UI. Supports MSI Afterburner profile imports and LACT profile exports. Can automatically scan for the most optimal GPU VF curve and generate silent fan curves.

tokenspeed-smg

TokenSpeed is a speed-of-light LLM inference engine.

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

pyptx

A Python DSL to write Nvidia PTX for Hopper and Blackwell in JAX and PyTorch

pygpukit

Minimal GPU runtime for Python - high-performance CUDA kernels, memory management, and LLM inference without heavy dependencies

taiwan-asr-toolkit

Production-grade Traditional Chinese / Taiwan Mandarin speech-to-text. Qwen3-ASR + MediaTek Breeze-ASR-25, hot-word injection, LLM polish, speaker diarization. RTF up to 1554x on RTX 5090, 56 TDD tests.

ai-dynamo-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

dblcsgen

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-acc

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-xft

A high-throughput and memory-efficient inference and serving engine for LLMs

wxy-test

A high-throughput and memory-efficient inference and serving engine for LLMs

nextai-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-consul

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-musa

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-npu

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-hust

A high-throughput and memory-efficient inference and serving engine for LLMs