Run various LLMs with one API. For founders, by Xinference with 9.3k+ GitHub stars
intermediateโฑ 30 minutes๐ต Free (self-hosted)
9,418 stars845 forksPythonQuality 8/10Updated 7/10/2026100% free ยท open source
What it is
Run different AI models by changing a single line of code.
What you can make with it
Automations that integrate custom AI models with your apps or services, like using a different chatbot model with a single line of code.
How it helps
Swap out AI models without having to rewrite your entire app, and run them on any platform from cloud to your laptop.
Real use case example
"A founder wants to use the latest version of a chatbot model with their customer support app. They can switch out the model by updating a line of code, then test and deploy the changes without affecting their users. With Xinference, they can run the new model on their cloud server instantly."
If you're new
Picking this up first can give you a good foundation to build on when exploring AI models and APIs.
If you're senior
Senior engineers and professionals will want to reach for this to simplify the process of integrating and switching between AI models in their projects.
Common confusion cleared up
Xinference is not a specific AI model itself, but rather an API that allows you to run different models with a single line of code.
Best inside these AI tools
Claude DesktopClaude CodeCursor
Pairs with
Claude APIStripe webhookNotion database
Why we list it on WorkflowStacks: Xinference provides a unified API for running different AI models, which can save development time and increase flexibility for users.
What it does
Run various large language models (LLMs) with a single unified API, allowing you to easily switch between models by changing just one line of code.
โขWhen you need to compare the performance of different LLMs for your application
โขWhen you want to deploy a single API that can handle multiple speech, open-source, and multimodal models
โขWhen you need to test and integrate LLMs in different environments, such as cloud, on-prem, or local laptops
Quick start
1Clone the repository and navigate to the project directory: cd inference
2Review the README for specific configuration and model setup instructions
3Modify the model configuration as needed, for example by changing the model name in the code
4Run the inference API with a sample model, following the instructions in the README
5Use the API to test and integrate different LLMs, swapping them out by changing a single line of code
Ready-to-paste prompt
Modify the example code to run a specific LLM, such as swapping GPT for another model, and test the API with a prompt like 'What is the capital of France?'
Heads up: Make sure you have the necessary dependencies and runtime environment set up, as the inference API requires specific versions of Python and other libraries to function correctly
Saves to your device
Topics
artificial-intelligence
chatglm
deployment
flan-t5
gemma
ggml
glm4
inference
llama
llama3
llamacpp
llm
machine-learning
mistral
openai-api
pytorch
qwen
vllm
whisper
wizardlm
What's inside โ free to inspect
No purchase needed
Read the entire source before you build โ unlike paid marketplaces that hide it behind a buy button.