Model Compression Python Packages

tensorflow-model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

veloxquant-mlx

Fast KV-cache quantization for Apple Silicon (MLX) — 43 research-adapted compression methods with Metal kernels

tigris-ml

Ahead-of-time compiler that fits ML models on embedded devices with hard memory budgets.

nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

torch-pruning

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

kxy

A toolkit to boost the productivity of machine learning engineers.

axquant

Not another flat 4-bit quantizer. AXQuant mixes precision layer by layer for Apple Silicon—smaller MLX models, protected quality-critical layers, production-ready plans.

tf-model-optimization-nightly

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

fasterai

FasterAI: Prune and Distill your models with FastAI and PyTorch

bigsmall

Lossless AI model compression - ~34% smaller with bit-identical weights; the autopilot profiles your machine, picks the highest fidelity that runs, and streams models bigger than your RAM.

torch-kirigami

Structured pruning and sparse training for PyTorch models.

dnaty

Compress PyTorch models for edge devices — CPU-only, no GPU, no retraining. One function call.

picollm

On-device LLM Inference Powered by X-Bit Quantization

micronet

micronet, a model compression and deploy lib. compression: 1、quantization: quantization-aware-training(QAT), High-Bit(>2b)(DoReFa/Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference)、Low-Bit(≤2b)/Ternary and Binary(TWN/BNN/XNOR-Net); post-training-quantization(PTQ), 8-bit(tensorrt); 2、 pruning: normal、regular and group convolutional channel pruning; 3、 group convolution structure; 4、batch-normalization fuse for quantization. deploy: tensorrt, fp32/fp16/int8(ptq-calibration)、op-adapt(upsample)、dynamic_shape

mlx-dfloat

Run DFloat11 losslessly compressed BF16 checkpoints on Apple Silicon with MLX and a Metal decode kernel; today that runs FLUX.1 on a 32 GB Mac with weights about 30% smaller

picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

bnn-lab

Extreme low-bit inference lab: pack weights to 1–1.58 bits and run real XNOR/popcount kernels on CPU/edge — honest STE training, not fake GPU 32× from sign().

fracast

Fracast-0 is an 85K-parameter pretrained Time Series Foundation Model, small enough to load directly in a web page.

nni-daily

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

silverspoon-kd

A general-purpose toolkit for knowledge distillation of transformer models

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.

lmzip

Fast lossless compression for large model weights: 34% off BF16 checkpoints, byte-for-byte identical, with a CUDA decoder at 111 GB/s

kd-lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.