PyRank
  • Insights
  • PyPI
  • GitHub
  • Search
  • Compare
  • Advisories
  • Ecosystem
  • About

Distributed Inference Python Packages

Python packages with the GitHub topic distributed-inference. Sorted by relevance, with stars and monthly downloads.
flashinfer-ai
flashinfer-python

FlashInfer: Kernel Library for LLM Serving

5M 6K 977
flashinfer-ai
flashinfer-cubin

FlashInfer: Kernel Library for LLM Serving

3.5M 6K 977
youngharold
tightwad

Mixed-vendor GPU inference cluster manager with speculative decoding

2K 22 2
BenevolentJoker-JohnL
sollol

Super Ollama Load Balancer - Performance-aware routing for distributed Ollama deployments with Ray, Dask, and adaptive metrics

2K 4 2
mzbac
mlx-sharding

Distributed Inference for mlx LLm

220 101 11
clearclown
forge-sdk

Python SDK for the Forge compute economy — where AI agents earn and spend Compute Units

192 1 0
clearclown
forge-cu-mcp

MCP server for Forge compute economy — lets Claude, Cursor, and AI agents interact with CU

130 1 0
theoddden
terradev-mcp

An imperative command-line-interface for AI workload orchestration

118 13 2
pberlizov
vimin-core

Source-available local AI inference orchestration — broadcast dispatch and multi-step pipelines across up to 10 nodes

98 0 0
    • Data from PyPI, GitHub, ClickHouse, and BigQuery