PyRank

Efficient Inference Python Packages

Python packages with the GitHub topic efficient-inference. Sorted by relevance, with stars and monthly downloads.
horseee
deepcache

[CVPR 2024] DeepCache: Accelerating Diffusion Models for Free

53K 969 54
Picovoice
picollm

On-device LLM Inference Powered by X-Bit Quantization

2K 313 26
Picovoice
picollmdemo

On-device LLM Inference Powered by X-Bit Quantization

2K 313 26
lucidrains
speculative-decoding

Speculative Decoding

303 307 24