workflowstacks
local-ai

ggrun: Fast Multi‑GPU LLM Server for Startup AI Apps

Deploy GGUF models on llama.cpp with auto‑tuned flags, tensor‑split, and MoE placement. Ideal for founders needing a self‑hosted, OpenAI‑compatible inference server on multi‑GPU rigs.

From the repo · “Auto-tuned launcher for GGUF models on llama.cpp / ik_llama.cpp — OpenAI-compatible server with multi-GPU tensor-split, MoE expert placement, measured flag tuning (AI Tune), hardware-matched HuggingFace downloads, and crash recovery. An Ollama alternative for multi-GPU rigs.”

282 stars+2★ this week19 forksGoGuide quality 9/10 · reviewed Oct 18Updated 10/21/2026100% free · open source
What it does

Ggrun launches a self-hosted OpenAI-compatible server with auto-tuned settings for local AI solutions, especially suited for multi-GPU environments.

When to use it
  • •You need to deploy AI models on your own hardware for privacy or security reasons.
  • •Your application requires low-latency AI responses that cloud services cannot guarantee.
  • •You want to leverage the power of multiple GPUs for demanding AI workloads without relying on cloud providers.
Ready-to-paste prompt
go run main.go -model /path/to/your/llama.model -gpu 0,1 to start the server with a specific model and utilize two GPUs.
Heads up: Ensure you have the necessary Go environment set up and that your system meets the hardware requirements for the models you intend to run, especially considering the need for compatible GPUs.
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/ggrun && curl -fsSL https://workflowstacks.com/api/skills/ggrun/claude-skill -o ~/.claude/skills/ggrun/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How ggrun: Fast Multi‑GPU LLM Server for Startup AI Apps works
Codeflow
Free to inspect

Ggrun: Fast Multi‑GPU LLM Server for Startup AI Apps is a very large Go project (~121k lines across 365 code files, plus 23 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, MIT license, has a test suite.

Size
Very large codebase
~121k lines · 365 code files · days to read — use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
Docker
No API keys detected
Go 88%Python 6%Shell 4%PowerShell 1%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    AGENTS.md
    The instructions the AI actually follows
  3. 3
    go/pkg/advisor/artifact_test.go
    Inside go/pkg/ — the main logic begins here
  4. 4
    examples/systemd/ggrun.service
    A worked example — copy this to get going
What's in each folder
go/pkg/Core code — the actual logic234 files
go/cmd/Command-line entry points110 files
docs/Documentation30 files
examples/Examples you can copy1 files
scripts/Helper scripts21 files
docker/Deployment / infrastructure5 files
native/Folder5 files
tools/Helper scripts5 files
READMEHas testsDocumentedCI checksDocker readyExamples includedMIT licenseUpdated this month
Quick Actions
Details
Creator
raketenkater
Language
Go
Category
local-ai
Published
3/11/2026

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/ggrun.svg)](https://workflowstacks.com/skills/ggrun?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Related skills

More local-ai tools founders pair with this one.

local-ai★ 132,005
Run Powerful LLMs Locally with llama.cpp – Fast C++ Inference
Empower your product with on‑device LLM inference using llama.cpp’s efficient C++ engine. Ideal for founders who need privacy, speed, and no cloud costs. 130k+ GitHub stars.
local-ai★ 94,719
Stirling-PDF: Edit PDFs Anywhere
Edit PDFs on any device with Stirling-PDF, a tool for startup founders, backed by 80k+ GitHub stars
local-ai★ 38,207
Run AI Locally with llmfit
Find compatible AI models for your hardware with llmfit, a tool for founders, with 29k+ GitHub stars.
local-ai★ 765
sno-station
Sno Station — your Claude Code and Codex working as one squad on your own machine. Shared encrypted memory, agent-to-agent messaging (Reach), squad skills for handoff and cross-vendor review, and a nightly loop that rewrites the agents' own skills with your approval. Open source, no daemon, no cloud required. Assembled in public.
local-ai★ 285
TurboLLM
Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine in one command. No Electron, no Python, offline-first.
local-ai★ 664
argosy-launcher: Launch Games
Sync, install, and launch games on mobile devices with argosy-launcher, a tool for founders and mobile gamers.