workflowstacks
voice-ai

Xinference: Swap LLMs Easily

Run various LLMs with one API.
intermediate⏱ 30 minutesπŸ’΅ Free (self-hosted)
9,500 stars859 forksPythonGuide quality 8/10Updated 8/17/2026100% free Β· open source
What it is

Run different AI models by changing a single line of code.

Real use case example

"A founder wants to use the latest version of a chatbot model with their customer support app. They can switch out the model by updating a line of code, then test and deploy the changes without affecting their users. With Xinference, they can run the new model on their cloud server instantly."

Best insideClaude DesktopClaude CodeCursor
When to use it
  • β€’When you need to compare the performance of different LLMs for your application
  • β€’When you want to deploy a single API that can handle multiple speech, open-source, and multimodal models
  • β€’When you need to test and integrate LLMs in different environments, such as cloud, on-prem, or local laptops
Ready-to-paste prompt
Modify the example code to run a specific LLM, such as swapping GPT for another model, and test the API with a prompt like 'What is the capital of France?'
Heads up: Make sure you have the necessary dependencies and runtime environment set up, as the inference API requires specific versions of Python and other libraries to function correctly
Saves to your device
Use with Claude
New

Skip the builder β€” one click puts this in Claude, Cursor, Antigravity and more.

πŸ› οΈ Technical setup

Expect 20–40 minutes in a terminal β€” or let your AI agent drive it.

Try it instantly β€” no install
Claude Code
mkdir -p ~/.claude/skills/inference && curl -fsSL https://workflowstacks.com/api/skills/inference/claude-skill -o ~/.claude/skills/inference/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go β€” no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How Xinference: Swap LLMs Easily works
Codeflow
Free to inspect

Xinference: Swap LLMs Easily is a very large Python project (~154k lines across 588 code files, plus 260 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~154k lines Β· 588 code files Β· days to read β€” use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
Python
No API keys detected
Python 82%TypeScript 17%
Where to start reading
  1. 1
    README.md
    Start here β€” what it does and how to install it
  2. 2
    AGENTS.md
    The instructions the AI actually follows
  3. 3
    pyproject.toml
    Dependencies and the commands it exposes
What's in each folder
frontend/Frontend / UI214 files
doc/Documentation1810 files
xinference/Folder663 files
monitor/Folder56 files
READMES/Folder9 files
benchmark/Evaluations & benchmarks8 files
.github/CI / automation (GitHub Actions)12 files
assets/Images & static assets6 files
READMEHas testsDocumentedCI checksApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
xorbitsai
Language
Python
Category
voice-ai
Published
6/14/2023

Are you the creator of this tool? Claim your listing β†’ and earn 85% of every sale.

Related skills

More voice-ai tools founders pair with this one.

voice-aiβ˜… 190,735
opencode: Code Faster
Get a coding agent with opencode.
voice-aiβ˜… 75,970
Unsloth: Train AI Models
Train and run open AI models locally with Unsloth Studio.
voice-aiβ˜… 47,937
Airi
Self hosted, you-owned Grok Companion, a container of souls of waifu, cyber livings to bring them into our worlds, wishing to achieve Neuro-sama's altitude. Capable of realtime voice chat, Minecraf…
voice-aiβ˜… 30,031
Meetily
Privacy first, AI meeting assistant with 4x faster Parakeet/Whisper live transcription, speaker diarization, and Ollama summarization built on Rust. 100% local processing. no cloud required. Meetil…
voice-aiβ˜… 26,304
Boost AI coding with cmux
Get a macOS terminal with vertical tabs and notifications for AI coding agents, ideal for founders using AI coding tools
voice-aiβ˜… 18,358
Speech: Build AI Voices
Get scalable speech AI with Speech, a framework for researchers and developers.