Inference Engine Python Packages

astroid

A common base representation of python source code for pylint and other projects

qai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

mtplx

2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

experta

Expert Systems for Python

qai-hub-models-cli

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

aphrodite-engine

Large-scale LLM inference engine

friendli-client

Friendli Suite Client

onediff

an out-of-the-box acceleration library for diffusion models

nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

onediffx

OneDiff: An out-of-the-box acceleration library for diffusion models.

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

nocturnusai

Python SDK for NocturnusAI — a logic-based inference engine and knowledge database

m5-infer

Extraordinary speed, extraordinary quality — MLX-based inference engine for Apple Silicon.

nef2

NEF2: A high-performance, unified multi-backend AI infrastructure stack. Native support for CUDA, ROCm, Metal, and Distributed Fabric.

neurobrix

Universal AI Runtime — Execute any model on any hardware

exxa

Exa - Pytorch

kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

yolomosaic

A Python library for visualizing YOLO detections and segmented instances on large orthomosaic images, with the ability to generate shapefiles for GIS integration

para-attn

https://wavespeed.ai/ Context parallel attention that accelerates DiT model inference with dynamic caching

zllm-zse

The inference engine the open-source world built for itself.

quaynor

Local GGUF inference, tool calling, and streaming chat for Python (Quaynor bindings).

opencv-python-inference-engine

Wrapper package for OpenCV with Inference Engine python bindings.

qai-hub-apps

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.