Model Compression Python Packages

tensorflow-model-optimization

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

nni

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

torch-pruning

[CVPR 2023] DepGraph: Towards Any Structural Pruning; LLMs, Vision Foundation Models, etc.

tf-model-optimization-nightly

A toolkit to optimize ML models for deployment for Keras and TensorFlow, including quantization and pruning.

kxy

A toolkit to boost the productivity of machine learning engineers.

dnaty

Compress PyTorch models for edge devices — CPU-only, no GPU, no retraining. One function call.

bigsmall

Lossless AI model compression - ~34% smaller with bit-identical weights; the autopilot profiles your machine, picks the highest fidelity that runs, and streams models bigger than your RAM.

picollm

On-device LLM Inference Powered by X-Bit Quantization

picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

model-quantizer

Effortlessly quantize, benchmark, and publish Hugging Face models with cross-platform support for CPU/GPU. Reduce model size by 75% while maintaining performance.

aquvitae

Knowledge Distillation Toolkit

micronet

A model compression and deploy lib.

torch-optim

PyTorch models optimization by neural network pruning

fasterai

FasterAI: Prune and Distill your models with FastAI and PyTorch

distill-anything

Distill any AI model into one you own — a teacher (Claude/GPT/HF/Ollama) generates your dataset, a student trains on its logits or its words, a judge scores it blind, a benchmark prices it. Runs on a MacBook.

only-train-once

OTOv1-v3, NeurIPS, ICLR, TMLR, DNN Training, Compression, Structured Pruning, Erasing Operators, CNN, Diffusion, LLM

uni-layer

A Universal Framework for Layer Contribution Analysis

nni-daily

An open source AutoML toolkit for automate machine learning lifecycle, including feature engineering, neural architecture search, model compression and hyper-parameter tuning.

musco-pytorch

MUSCO: MUlti-Stage COmpression of neural networks

kd-lib

A Pytorch Knowledge Distillation library for benchmarking and extending works in the domains of Knowledge Distillation, Pruning, and Quantization.

prismkv

3-D stacked-plane KV cache quantization + RAG framework. Defensive prior-art publication extending TurboQuant to conditional 3-D polar cells with adaptive bit allocation.

fused-turboquant

Fused Triton kernels for TurboQuant KV cache compression — 2-4 bit quantization with RHT rotation. Drop-in HuggingFace & vLLM integration. Up to 4.9x KV cache compression for Llama, Qwen, Mistral, and more.

octopus-ml

A collection of handy ML and data visualization and validation tools. Go ahead and train, evaluate and validate your ML models and data with minimal effort.