Auto Tuning Python Packages

neural-compressor

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

isat-tuner

Inference Stack Auto-Tuner — one command to find the fastest ONNX inference config for any model on any GPU. Bayesian optimization, Pareto analysis, thermal-aware benchmarking, multi-provider (ROCm/CUDA/TensorRT/OpenVINO).

neural-compressor-pt

Repository of Intel® Neural Compressor

neural-compressor-tf

Repository of Intel® Neural Compressor

kernel-tuner

Kernel Tuner

alchemy-cat

Alchemy Cat —— 🔥Config System for SOTA

smartloop

AI orchestration on your device

autotuning-methodology

This software package accompanies the paper "A Methodology for Comparing Auto-Tuning Optimization Algorithms" (https://doi.org/10.1016/j.future.2024.05.021), making the guidelines in the methodology easy to apply.

neural-compressor-full

Repository of Intel® Neural Compressor

neural-solution

Repository of Intel® Neural Compressor

neural-insights

Repository of Intel® Neural Compressor

lpot

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

tuning-metrics

Library to compute auto-tuning and performance metrics.

ilit

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

neural-compressor-3x-pt

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

neural-compressor-3x-ort

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime

neural-compressor-3x-tf

SOTA low-bit LLM quantization (INT8/FP8/MXFP8/INT4/MXFP4/NVFP4) & sparsity; leading model compression techniques on PyTorch, TensorFlow, and ONNX Runtime