Inference Engine Python Packages

astroid

A common base representation of python source code for pylint and other projects

qai-hub-models

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

tessera-hypernetwork

Generate per-session LoRA adapters in <1s for agentic inference efficiency.

qai-hub-models-cli

Qualcomm® AI Hub Models is our collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.

experta

Expert Systems for Python

squish-ai

⚡️ The fastest way to run local LLMs on Apple Silicon — sub-second model loads, beats Ollama on throughput, tail latency, and full-response time. OpenAI/Ollama-compatible. No cloud, no API keys.

sigit-code

A local-Llm-first coding agent.

mtplx

2.24x decode TPS increase On Qwen 3.6 27B @ temp 0.6 | Native MTP Speculative Decoding On Apple Silicon With No External Drafter.

krasis

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

friendli-client

Friendli Suite Client

onediff

OneDiff: An out-of-the-box acceleration library for diffusion models.

onediffx

OneDiff: An out-of-the-box acceleration library for diffusion models.

ai4i-core

A comprehensive microservices based platform to handle language based AI models inferences at scale.

aphrodite-engine

Large-scale LLM inference engine

fedml

FEDML - The unified and scalable ML library for large-scale distributed training, model serving, and federated learning. FEDML Launch, a cross-cloud scheduler, further enables running any AI jobs on any GPU cloud or on-premise cluster. Built on this library, TensorOpera AI (https://TensorOpera.ai) is your generative AI platform at scale.

neurobrix

Universal AI Runtime — Execute any model on any hardware

nobodywho

NobodyWho is an inference engine that lets you run LLMs locally and efficiently on any device.

nocturnusai

Verified knowledge for AI agents. Compress context, extract and store facts, define rules, and ask questions — get deterministic answers with proof, not LLM guesses. Connect agents via MCP, Python SDK, TypeSc

tala-locale

Phone number → country, currency, and language. One call. Zero dependencies.

kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond

ai4icore-core

A comprehensive microservices based platform to handle language based AI models inferences at scale.

zllm-zse

The inference engine the open-source world built for itself.

exxa

Unleash the full potential of exascale LLMs on consumer-class GPUs, proven by extensive benchmarks, with no long-term adjustments and minimal learning curve.

qai-hub-apps

The Qualcomm® AI Hub apps are a collection of state-of-the-art machine learning models optimized for performance (latency, memory etc.) and ready to deploy on Qualcomm® devices.