PyRank

Efficient Inference Python Packages

Python packages with the GitHub topic efficient-inference. Sorted by monthly downloads, with stars and forks.
horseee
deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

24K 968 51
Picovoice
picollm

On-device LLM Inference Powered by X-Bit Quantization

720 318 27
Picovoice
picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

530 318 27
lucidrains
speculative-decoding

Explorations into some recent techniques surrounding speculative decoding

317 308 25