Tensorrt Llm Python Packages

ai-dynamo-runtime

A Datacenter Scale Distributed Inference Serving Framework

ai-dynamo

A Datacenter Scale Distributed Inference Serving Framework

optimum-benchmark

🏋️ A unified multi-backend utility for benchmarking Transformers, Timm, PEFT, Diffusers and Sentence-Transformers with full support of Optimum's hardware optimizations & quantization schemes.

kvbm

A Datacenter Scale Distributed Inference Serving Framework

whisper-live

A nearly-live implementation of OpenAI's Whisper.

whisper-s2t

An Optimized Speech-to-Text Pipeline for the Whisper Model Supporting Multiple Inference Engine

modelsig

Compare LLM architectures without downloading weights — structural fingerprint & proxy-test advisor for vLLM, TensorRT-LLM, SGLang, ONNX Runtime