Fp8 Python Packages

nvidia-cudnn-frontend

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

experts4bit-qlora

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

grouped-nf4-gemm

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.

ffpa-attn

Kernel Library for Large Headdim Attention (64~1024, BF16/FP8/FP4), 1.5x~15x↑ vs PyTorch SDPA.

neural-weight-compression

Neural Weight Compression: lossless LLM weight compression decoded inside the CUDA matvec. BF16 31 % smaller and faster than cuBLAS; fp8 13 % smaller at native fp8 speed. Bit-identical weights.

bigsmall

Lossless AI model compression - ~34% smaller with bit-identical weights; the autopilot profiles your machine, picks the highest fidelity that runs, and streams models bigger than your RAM.

gridbook

Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.

whisper-blaze

High-throughput batched Whisper large-v3 serving on NVIDIA Hopper GPUs — OpenAI-compatible API, dynamic batching, VRAM capping, fused Hopper LayerNorm kernel

crival

Floating-point results that are the same bits on every machine: correctly rounded sums and matrix products, neural-network primitives with one answer, the normal distribution, verified linear algebra, correctly rounded fixed-point functions, FP8 and MX math, interval functions, and tools that find machine-dependent results

neural-speed

An innovative library for efficient LLM inference via low-bit quantization

mantissa-core

Low-precision numerics core for neural networks, in C. Custom float formats (tekin8/tekin32), narrow-store/wide-accumulate, Python bindings. By Tekin Ertekin.

inferpilot

Evidence-first vLLM screening and controlled capacity analysis. Screen live metrics, verify config changes, and abstain when evidence is weak.

breccia

Block-scaled FP8 / FP4 / INT4 tensor primitive with Triton scaled-matmul at FP32 parity on H100. NumPy / PyTorch / MLX / JAX backends.

e4b

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

experts4bit

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

e4b-qlora

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

experts-mxfp4

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

rwkv-quant-nv

RWKV-7 FP8 quantized inference - 6.4x decode speedup on Blackwell GPUs with <0.3% accuracy loss. Full FP8 E4M3 weight quantization with fused Triton kernels.

ssblast

First open-source FP8 linear solver for consumer NVIDIA GPUs — 2-3x faster than cuBLAS FP64. pip install ssblast

grouped-mxfp4-gemm

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.

expertsnbit

Fine-tune and serve large Mixture-of-Experts models on one GPU, with 4-bit experts in GPU memory, RAM or SSD.

jax-scalify

JAX Scalify: end-to-end scaled arithmetics

gnf4

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.

nf4gemm

4-bit GPU kernels that compute directly on packed Mixture-of-Experts weights, for training and inference.