Blackwell Python Packages

sglang

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

nvidia-cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

tokenspeed-mla

TokenSpeed is a speed-of-light LLM inference engine.

sglang-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

sgl-kernel

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-tpu

A high-throughput and memory-efficient inference and serving engine for LLMs

tokenspeed-smg

TokenSpeed is a speed-of-light LLM inference engine.

vllm-cpu-nightly

A high-throughput and memory-efficient inference and serving engine for LLMs

tensorrt-llm

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

sglang-kt

SGLang is a high-performance serving framework for large language models and multimodal models.

pygpukit

Minimal GPU runtime for Python - high-performance CUDA kernels, memory management, and LLM inference without heavy dependencies

tokenspeed-kernel-amd

TokenSpeed AMD-specific high-performance kernels.

lsglang

SGLang is a fast serving framework for large language models and vision language models.

taiwan-asr-toolkit

Production-grade Traditional Chinese / Taiwan Mandarin speech-to-text. Qwen3-ASR + MediaTek Breeze-ASR-25, hot-word injection, LLM polish, speaker diarization. RTF up to 1554x on RTX 5090, 56 TDD tests.

pyptx

A Python DSL to write Nvidia PTX for Hopper and Blackwell in JAX and PyTorch

ai-dynamo-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-acc

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-xft

A high-throughput and memory-efficient inference and serving engine for LLMs

dblcsgen

SGLang is a high-performance serving framework for large language models and multimodal models.

vllm-musa

vLLM platform plugin for Moore Threads MUSA GPUs

vllm-consul

A high-throughput and memory-efficient inference and serving engine for LLMs

vllm-npu

A high-throughput and memory-efficient inference and serving engine for LLMs

nextai-vllm

A high-throughput and memory-efficient inference and serving engine for LLMs