ai-agent

Scale LLM Deployments Efficiently with llm-scaler

llm-scaler lets founders rapidly expand large language model workloads in C++ without manual tuning, ideal for AI‑focused startups seeking reliable performance.

From the repo · “No description available”

574 stars+10★ this week86 forksC++Guide quality 8/10 · reviewed Oct 10Updated 10/20/2026100% free · open source
What it does

llm-scaler is a C++ tool that helps founders scale large language models for AI applications, enabling efficient and optimized performance

When to use it
  • •When you need to deploy large language models in a production environment
  • •When you want to optimize the performance of your AI models for better user experience
  • •When you are working with limited computational resources and need to scale your models efficiently
Ready-to-paste prompt
./llm_scaler -c config.json -m bert-base-uncased -b 32 -s 128
Heads up: Make sure you have the required C++ compiler and CMake version installed on your system, as specified in the README, to successfully build and run llm-scaler
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/llm-scaler && curl -fsSL https://workflowstacks.com/api/skills/llm-scaler/claude-skill -o ~/.claude/skills/llm-scaler/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

7
top-level files
4
folders
64.5M
source size
Apache-2.0
license
Key files
README.md
File tree
.github/
omni/
sglang/
vllm/
.gitignore
CODE_OF_CONDUCT.md
CONTRIBUTING.md
LICENSE
README.md
Releases.md
SECURITY.md
Quick Actions
Details
Creator
intel
Language
C++
Category
ai-agent
Published
6/19/2025

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/llm-scaler.svg)](https://workflowstacks.com/skills/llm-scaler?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.