Distributed Inference Python Packages

flashinfer-python

FlashInfer: Kernel Library for LLM Serving

flashinfer-cubin

FlashInfer: Kernel Library for LLM Serving

terradev-cli

An imperative command-line-interface for AI workload orchestration

sawyer-core

Distributed MoE inference network — the load is split, friends help.

sollol

Intelligent Load Balancer for Ollama Clusters - Adaptive Parallelism + VRAM-Aware GPU Routing + Ray/Dask/llama.cpp Distribution

tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

forge-sdk

Python SDK for the Forge compute economy — where AI agents earn and spend Compute Units

mlx-sharding

Distributed Inference for mlx LLm

terradev-mcp

An imperative command-line-interface for AI workload orchestration

forge-cu-mcp

MCP server for Forge compute economy — lets Claude, Cursor, and AI agents interact with CU

vimin-core

Source-available local AI inference orchestration — broadcast dispatch and multi-step pipelines across up to 10 nodes