workflowstacks
ai-agent

Lucebox: Fast Local AI

Get fast LLM inference on consumer hardware with lucebox, ideal for local AI startup founders
2,697 stars253 forksC++Guide quality 9/10Updated 7/30/2026100% free · open source
What it does

Lucebox is a fast LLM speculative inference server that allows you to run large language models on consumer-grade hardware for AI-related tasks.

When to use it
  • •When you need to deploy AI models on consumer hardware with limited resources
  • •When you want to test and prototype AI applications quickly without relying on cloud services
  • •When you require a self-hosted solution for AI inference with a small footprint
Ready-to-paste prompt
curl -X POST -H 'Content-Type: application/json' -d '{"prompt": "Generate a short story about a character who discovers a hidden world"}' http://localhost:8080/infer
Heads up: Ensure you have a compatible C++ compiler (such as GCC 10 or higher) and CMake installed on your system before attempting to build and run Lucebox
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/lucebox && curl -fsSL https://workflowstacks.com/api/skills/lucebox/claude-skill -o ~/.claude/skills/lucebox/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How Lucebox: Fast Local AI works
Codeflow
Free to inspect

Lucebox: Fast Local AI is a very large C++ project (~362k lines across 833 code files, plus 122 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~362k lines · 833 code files · days to read — use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
Python · Docker
No API keys detected
C++ 60%C 14%Cuda 12%Python 7%Metal 2%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    pyproject.toml
    Dependencies and the commands it exposes
  3. 3
    server/src/cuda_cross_device_copy.cpp
    Inside server/src/ — the main logic begins here
What's in each folder
server/src/Core code — the actual logic309 files
server/docs/Documentation15 files
docs/Documentation6 files
server/examples/Examples you can copy1 files
server/deps/Folder1099 files
optimizations/Folder60 files
harness/Folder55 files
server/scripts/Helper scripts42 files
READMEHas testsDocumentedCI checksDocker readyApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
Luce-Org
Language
C++
Category
ai-agent
Published
4/3/2026

Are you the creator of this tool? Claim your listing → and earn 85% of every sale.

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 276,806
Track GitHub Stars with 996.ICU
Get a popular GitHub star counter for your startup, backed by 276k+ GitHub stars, ideal for founders.
ai-agent★ 264,191
Superpowers
An agentic skills framework & software development methodology that works.
ai-agent★ 244,424
Linux: Open Source Kernel
Get the Linux kernel source tree for custom development, ideal for startup founders in need of a flexible OS foundation with 236k+ GitHub stars
ai-agent★ 210,750
Vue
This is the repo for Vue 2. For Vue 3, go to https://github.com/vuejs/core
ai-agent★ 204,069
Improve Code with andrej-karpathy-skills
Get better code behavior with andrej-karpathy-skills, 165k+ GitHub stars, for founders using LLMs.
ai-agent★ 197,010
FreeDomain: Free Website Domain
Get a free domain with FreeDomain. For startup founders.