semantic-routing
KV-cache-aware intelligent routing for self-hosted and hybrid LLM fleets. Route requests using model quality, latency, cost, policy, and live GPU state.
SemRoute is a semantic router that helps you route using the semantic meaning of the query
