Inference Server Python Packages

truss

The simplest way to serve AI/ML models in production

inference-gpu

Turn any computer or edge device into a command center for your computer vision projects.

inference-cli

Turn any computer or edge device into a command center for your computer vision projects.

baseten-performance-client

The simplest way to serve AI/ML models in production

truss-transfer

The simplest way to serve AI/ML models in production

inference-sdk

Turn any computer or edge device into a command center for your computer vision projects.

inference

Turn any computer or edge device into a command center for your computer vision projects.

ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

inference-cpu

Turn any computer or edge device into a command center for your computer vision projects.

inference-core

Turn any computer or edge device into a command center for your computer vision projects.

friendli-client

Friendli Suite Client

fastdeploy

Deploy DL/ ML inference pipelines with minimal extra code.

baseten-inference-client

The simplest way to serve AI/ML models in production

jesco-apple-silicon-engine

LLM inference server with continuous batching & SSD caching for Apple Silicon — managed from the macOS menu bar

geniusrise-vision

Vision and vision-multi-modal components for geniusrise framework

geniusrise-text

Text components powering LLMs & SLMs for geniusrise framework

fdclient

Deploy DL/ ML inference pipelines with minimal extra code.

geniusrise-audio

audio bolts for geniusrise

pipeless-ai-tf-models

An open-source computer vision framework to build and deploy apps in minutes

pinferencia

Python + Inference - Model Deployment library in Python. Simplest model inference server ever.

periflow-client

Client of PeriFlow, the fastest generative AI serving available.

pipeless-ai

An open-source computer vision framework to create and deploy computer vision applications that scale in minutes

kvwarden

Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.

pipeless-ai-cli

An open-source computer vision framework to create and deploy computer vision applications that scale in minutes