Npu Python Packages

optimum-rbln

⚡ A seamless integration of HuggingFace Transformers & Diffusers with RBLN SDK for efficient inference on RBLN NPUs.

device-smi

Self-contained Python lib with zero-dependencies that give you a unified device properties for gpu, cpu, and npu. No more calling separate tools such as nvidia-smi or /proc/cpuinfo and parsing it yourself.

vllm-mblt

An out-of-tree vLLM plugin for Mobilint NPU runtime integration.

mblt-vision-python

Python toolkit for preprocessing, inference, and postprocessing of computer vision models on Mobilint NPUs and ONNX Runtime.

mblt-tracker

A lightweight Python library for tracking hardware metrics (Power, Utilization, Memory) across CPU, GPU, and NPU.

vinowhisper

NPU-accelerated local voice typing & auto-captioning (with translations) on Linux via OpenVINO GenAI Whisper

transformers-mblt

Python toolkit for running pre-quantized Hugging Face Transformers models (LLMs, VLMs, speech recognition) on Mobilint NPUs.

ovkit

One line of Python for 20 AI capabilities on Intel CPU/GPU/NPU — detection, OCR, text-to-speech, pose. Offline, Apache-2.0, no AGPL.

uc-manager-cuda-cu129

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann910-a3

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann910-a2

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cuda-cu130

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a2

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a3

Persist and reuse KV Cache to speedup your LLM.

torch-rechub

A Lighting Pytorch Framework for Recommendation Models, Easy-to-use and Easy-to-extend.

hal0ai

Open-source self-hosted home AI inference platform for AMD Strix Halo — multi-backend slots, OpenAI-compatible gateway, Vue 3 + FastAPI + systemd.

ascend-nputop

An interactive Ascend-NPU process viewer

maixtool

C/C++ development kit for Sipeed Maix ecosystem boards

melotts-mblt

Python toolkit for MeloTTS text-to-speech (English and Korean) on Mobilint NPUs, with a CLI and Gradio WebUI.

uc-manager-cann910-a5

Persist and reuse KV Cache to speedup your LLM.

uc-manager-cann901-a5

Persist and reuse KV Cache to speedup your LLM.

rk-transformers

Export and Run Hugging Face Transformers Models on Rockchip NPUs

lemonade-python-sdk

A production-grade Python SDK for the Lemonade LLM backend. Featuring auto-discovery, port scanning, and a low-overhead client architecture. Powering the Sorana AI workspace.

rockchip-stats

Rockchip Device Monitoring Tool