Model Compression Python Packages

tensorflow-model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

torch-pruning

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

tf-model-optimization-nightly

A suite of tools that users, both novice and advanced can use to optimize machine learning models for deployment and execution.

model-quantizer

Effortlessly quantize, benchmark, and publish Hugging Face models with cross-platform support for CPU/GPU. Reduce model size by 75% while maintaining performance.

kxy

A toolkit to boost the productivity of machine learning engineers.

picollm

On-device LLM Inference Powered by X-Bit Quantization

fasterai

FasterAI: Prune and Distill your models with FastAI and PyTorch

picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

torch-optim

PyTorch models optimization by neural network pruning

aquvitae

Knowledge Distillation Toolkit

nni-daily

Neural Network Intelligence package

uni-layer

A Universal Framework for Layer Contribution Analysis

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.

kd-lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

fused-turboquant

Fused Triton kernels for TurboQuant KV cache compression — 2-4 bit quantization with RHT rotation. Drop-in HuggingFace & vLLM integration. Up to 4.9x KV cache compression for Llama, Qwen, Mistral, and more.

only-train-once

Only Train Once (OTO): Automatic One-Shot General DNN Training and Compression Framework

musco-pytorch

MUSCO: MUlti-Stage COmpression of neural networks

prismkv

3-D stacked-plane KV cache quantization + RAG framework. Defensive prior-art publication extending TurboQuant to conditional 3-D polar cells with adaptive bit allocation.

archai

Accelerate your Neural Architecture Search (NAS) through fast, reproducible and modular research.

bfas

Brute Force Architecture Search

qsparse

train neural networks with joint quantization and pruning on both weights and activations using any pytorch modules