workflowstacks
local-ai

MNN: Accelerate On‑Device AI with a 16k+ Star Inference Engine

Deploy high‑performance LLMs and edge AI models on embedded hardware fast. Ideal for founders building C++‑based AI products for mobile, IoT, or edge devices.

From the repo · “MNN: A blazing-fast, lightweight inference engine battle-tested by Alibaba, powering high-performance on-device LLMs and Edge AI.”

intermediate⏱ 30 minutes💵 Free (self-hosted)
16,276 stars+49★ this week2,485 forksC++Guide quality 8/10 · reviewed Oct 5Updated 10/23/2026100% free · open source
What it is

Run highly optimized machine learning on small devices, like phones or robots, using MNN.

Real use case example

"A founder of a robotics startup wants to create a self-driving wheelchair. First, they connect a camera to their prototype. Then, they write code that uses MNN to detect obstacles and adapt the path. Finally, they test the device in different environments, refining its AI."

Best insideCursorClaude DesktopCodex CLI
When to use it
  • •When you need to deploy AI models on edge devices with limited computational resources
  • •When you want to improve the performance and efficiency of your on-device AI applications
  • •When you need a battle-tested and widely adopted inference engine for your startup's AI projects
Ready-to-paste prompt
mnn::Interpreter interpreter('model.mnn'); mnn::Session session(interpreter); session.run({'input': input_data}, {'output': output_data});
Heads up: MNN requires a compatible C++ compiler (e.g. GCC 5.4 or later) and a compatible OpenCV version (e.g. OpenCV 3.4 or later) to build and run successfully
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/mnn && curl -fsSL https://workflowstacks.com/api/skills/mnn/claude-skill -o ~/.claude/skills/mnn/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How MNN: Accelerate On‑Device AI with a 16k+ Star Inference Engine works
Codeflow
Free to inspect

MNN: Accelerate On‑Device AI with a 16k+ Star Inference Engine is a very large C++ project (~1.2M lines across 5457 code files, plus 572 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~1.2M lines · 5457 code files · days to read — use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
C++
No API keys detected
C++ 89%C 3%Python 2%Kotlin 2%Assembly 2%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    AGENTS.md
    The instructions the AI actually follows
  3. 3
    source/backend/hexagon/README.md
    Inside source/ — the main logic begins here
  4. 4
    demo/exec/CMakeLists.txt
    A worked example — copy this to get going
What's in each folder
source/Core code — the actual logic2641 files
skills/Prompts, skills & agent definitions53 files
apps/Sub-packages / plugins2733 files
docs/Documentation163 files
doc/Documentation154 files
demo/Examples you can copy18 files
3rd_party/Folder947 files
tools/Helper scripts842 files
READMEHas testsDocumentedCI checksExamples includedApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
alibaba
Language
C++
Category
local-ai
Published
4/15/2019

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/mnn.svg)](https://workflowstacks.com/skills/mnn?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Related skills

More local-ai tools founders pair with this one.

local-ai★ 132,105
Run Powerful LLMs Locally with llama.cpp – Fast C++ Inference
Empower your product with on‑device LLM inference using llama.cpp’s efficient C++ engine. Ideal for founders who need privacy, speed, and no cloud costs. 130k+ GitHub stars.
local-ai★ 94,806
Stirling-PDF: Edit PDFs Anywhere
Edit PDFs on any device with Stirling-PDF, a tool for startup founders, backed by 80k+ GitHub stars
local-ai★ 38,284
Run AI Locally with llmfit
Find compatible AI models for your hardware with llmfit, a tool for founders, with 29k+ GitHub stars.
local-ai★ 1,686
auto-round
A SOTA quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的量化工具包
local-ai★ 8,039
Secure Email with Tutanota
Encrypt emails, contacts, and calendar entries for enhanced privacy. For founders needing secure communication, backed by 7.7k+ GitHub stars.
local-ai★ 5,867
Maka‑Agent: Boost Productivity with a Local‑First AI Desktop Assistant
Maka‑Agent brings AI-powered assistance directly to your desktop, running offline for privacy and speed—ideal for founders who need instant, secure help without cloud dependencies.