ai-agent

xllm – Accelerate AI Model Inference for Your Startup

Name: xllm – Accelerate AI Model Inference for Your Startup
Author: xLLM-AI

Leverage xllm’s high‑performance engine to run LLM, VLM, DiT and REC models on any AI accelerator. Ideal for founders building fast, scalable AI products. 1.6k+ GitHub stars.

From the repo · “A high-performance inference engine for LLM, VLM, DiT and REC models, optimized for diverse AI accelerators. It is hosted in OpenAtom Foundation.”

1,603 stars+3★ this week315 forksC++Guide quality 9/10 · reviewed Oct 14Updated 10/22/2026100% free · open source

What it does

xllm provides high-performance inference for large language models, enabling fast and efficient processing of AI workloads

When to use it

•You need to deploy large language models in a production environment with strict latency requirements
•Your startup relies on real-time natural language processing for tasks like sentiment analysis or text classification
•You want to optimize the performance of your AI models on diverse hardware accelerators

Ready-to-paste prompt

xllm_infer -c config.json -m your_model.xllm -i 'What is the capital of France?'

Heads up: Ensure you have the necessary dependencies installed, including CMake and a compatible C++ compiler, and that your model is compatible with the xllm format

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/xllm-ai-xllm && curl -fsSL https://workflowstacks.com/api/skills/xllm-ai-xllm/claude-skill -o ~/.claude/skills/xllm-ai-xllm/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How xllm – Accelerate AI Model Inference for Your Startup works

Codeflow

Free to inspect

Xllm – Accelerate AI Model Inference for Your Startup is a very large C++ project (~536k lines across 2206 code files, plus 464 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size

Very large codebase

~536k lines · 2206 code files · days to read — use, don't read

Setup

Developer setup

A real software project. Use it via its install path; don't expect to read it all.

Runs on

Python

No API keys detected

C++ 80%Python 16%Cuda 3%CMake 1%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
pyproject.toml
Dependencies and the commands it exposes
4
xllm/__init__.py
Inside xllm/ — the main logic begins here
5
examples/__init__.py
A worked example — copy this to get going

What's in each folder

xllm/Core code — the actual logic2293 files

docs/Documentation224 files

examples/Examples you can copy6 files

xllm-kernel/Folder38 files

cmake/Folder20 files

scripts/Helper scripts11 files

tools/Helper scripts7 files

cibuild/Folder3 files

READMEHas testsDocumentedCI checksExamples includedApache-2.0 licenseUpdated this month

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/xllm-ai-xllm && curl -fsSL https://workflowstacks.com/api/skills/xllm-ai-xllm/claude-skill -o ~/.claude/skills/xllm-ai-xllm/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

xLLM-AI

Language

C++