ai-agent

sonar: Accelerate LLM Inference at Scale for AI‑Driven Products

Leverage sonar’s C++ engine with CUDA, TPU, and Intel support to run massive language models fast. Ideal for founders building AI services that need high‑throughput inference. 1.9k+ GitHub stars.

From the repo · “Large-scale LLM inference engine”

1,880 stars+6★ this week211 forksC++Guide quality 9/10 · reviewed Oct 6Updated 10/21/2026100% free · open source
What it does

Sonar is a large-scale LLM inference engine that allows founders and machine learning teams to run fast and efficient language model inference

When to use it
  • •When you need to deploy a language model for production use cases
  • •When you want to speed up your language model inference pipeline
  • •When you require a scalable solution for large-scale language model deployments
Ready-to-paste prompt
./sonar -c config.json -m your_model.sonar -i 'What is the meaning of life?'
Heads up: Sonar requires a compatible C++ compiler (e.g. GCC 10 or later) and a CPU with AVX-512 support for optimal performance
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/sonar && curl -fsSL https://workflowstacks.com/api/skills/sonar/claude-skill -o ~/.claude/skills/sonar/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

27
top-level files
18
folders
101.6M
source size
AGPL-3.0
license
Key files
.pre-commit-config.yaml
config.yaml
README.md
File tree
.buildkite/
.gemini/
.github/
.sync/
aphrodite/
assets/
benchmarks/
cmake/
csrc/
docker/
docs/
examples/
patches/
recipes/
requirements/
rust/
tests/
tools/
.clang-format
.dockerignore
.gitignore
.markdownlint.yaml
.pre-commit-config.yaml
amdpatch.sh
Quick Actions
Details
Creator
dphnAI
Language
C++
Category
ai-agent
Published
6/23/2023

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/sonar.svg)](https://workflowstacks.com/skills/sonar?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.