devtools

semantic-router: Dynamic Routing for Multi‑Model AI Deployments

Name: semantic-router: Dynamic Routing for Multi‑Model AI Deployments
Author: vllm-project

Direct LLM requests to the optimal model across cloud, edge, or data‑center, boosting scalability and latency—perfect for founders building AI services. 6.1k+ GitHub stars.

From the repo · “A programmable Mixture-of-Models router for heterogeneous LLM inference”

intermediate⏱ 30 minutes💵 Free (self-hosted)

6,243 stars+74★ this week1,067 forksGoGuide quality 9/10 · reviewed Oct 9Updated 10/22/2026100% free · open source

What it is

Directs data to the right AI model with the semantic-router.

Real use case example

"A founder like Emily wants to create a chatbot that answers customer queries about product availability. Emily uses the semantic-router to direct customer data to the most accurate AI model, which returns an answer within seconds. The chatbot improves customer satisfaction and Emily can focus on growing the business."

Best insideClaude DesktopCursorCodex CLI

When to use it

•When deploying AI models at scale and needing efficient traffic routing
•In environments with diverse model types requiring smart routing decisions
•For optimizing resource utilization in cloud, data center, or edge computing

Ready-to-paste prompt

Run './semantic-router -config config.yaml -model-dir /path/to/models' to start the router with a specific model directory

Heads up: Ensure you have Go (version 1.17 or later) installed on your system, as semantic-router is built using Go and requires it to compile and run

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/semantic-router && curl -fsSL https://workflowstacks.com/api/skills/semantic-router/claude-skill -o ~/.claude/skills/semantic-router/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How semantic-router: Dynamic Routing for Multi‑Model AI Deployments works

Codeflow

Free to inspect

Semantic-router: Dynamic Routing for Multi‑Model AI Deployments is a very large Go project (~959k lines across 4786 code files, plus 722 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~959k lines · 4786 code files · days to read — use, don't read

Setup

Developer setup

A real software project. Use it via its install path; don't expect to read it all.

Runs on

No API keys detected

Go 48%Python 25%TypeScript 12%Rust 7%CSS 3%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
src/fleet-sim/README.md
Inside src/ — the main logic begins here

What's in each folder

src/Core code — the actual logic4811 files

website/Project website1370 files

dashboard/Folder1261 files

tools/Helper scripts358 files

deploy/Deployment / infrastructure315 files

config/Configuration254 files

candle-binding/Folder208 files

bench/Folder152 files

READMEHas testsDocumentedCI checksApache-2.0 licenseUpdated this month

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/semantic-router && curl -fsSL https://workflowstacks.com/api/skills/semantic-router/claude-skill -o ~/.claude/skills/semantic-router/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

vllm-project

Language

Do this with a working workflow

Pages we wrote and tested that put a skill like this to work — importable n8n templates, outcome guides and Claude Desktop configs.

Free n8n template

AI Support Agent (Chat)

A chat agent that answers customer questions from YOUR business info — with conversation memory. It never invents policies; anything it doesn't know, it hands to a human.