PyRank

Efficient Inference Python Packages

Python packages with the GitHub topic efficient-inference. Sorted by relevance, with stars and monthly downloads.
horseee
deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

55K 967 52
Picovoice
picollm

On-device LLM Inference Powered by X-Bit Quantization

1K 312 25
Picovoice
picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

921 312 25
lucidrains
speculative-decoding

Explorations into some recent techniques surrounding speculative decoding

470 302 24