Inference Server Python Packages

inference-gpu

Turn any computer or edge device into a command center for your computer vision projects.

inference-cli

Turn any computer or edge device into a command center for your computer vision projects.

truss

The simplest way to serve AI/ML models in production

truss-transfer

The simplest way to serve AI/ML models in production

baseten-performance-client

The simplest way to serve AI/ML models in production

inference-sdk

Turn any computer or edge device into a command center for your computer vision projects.

inference

Turn any computer or edge device into a command center for your computer vision projects.

ramalama

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

inference-core

Turn any computer or edge device into a command center for your computer vision projects.

inference-cpu

Turn any computer or edge device into a command center for your computer vision projects.

friendli-client

Friendli Suite Client

fastdeploy

Deploy DL/ ML inference pipelines with minimal extra code.

baseten-inference-client

The simplest way to serve AI/ML models in production

kvwarden

Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.

geniusrise-vision

Vision and vision-multi-modal components for geniusrise framework

fdclient

Deploy DL/ ML inference pipelines with minimal extra code.

geniusrise-text

Text components powering LLMs & SLMs for geniusrise framework

pipeless-ai

An open-source computer vision framework to create and deploy computer vision applications that scale in minutes

pipeless-ai-tf-models

An open-source computer vision framework to build and deploy apps in minutes

geniusrise-audio

Audio components for geniusrise framework

pipeless-ai-cli

An open-source computer vision framework to create and deploy computer vision applications that scale in minutes

infergrid

Tenant-fair LLM inference orchestration on a single GPU. No Kubernetes.

pinferencia

Aims to be the Simplest Machine Learning Model Inference Server

periflow-client

[⛔️ DEPRECATED] Friendli: the fastest serving engine for generative AI