workflowstacks
ai-agent

FlashRT: Accelerate Low‑Latency AI Inference on Jetson Edge Devices

FlashRT delivers real‑time inference for small‑batch, latency‑critical AI models, perfect for founders deploying VLA control or LLMs on Jetson platforms.

From the repo · “FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B”

751 stars+86★ this week97 forksC++Guide quality 8/10 · reviewed Oct 7Updated 10/22/2026100% free · open source
What it does

FlashRT is a high-performance realtime inference engine for running low-latency AI models, especially for small-batch workloads and production VLA control.

When to use it
  • •When you need to deploy AI models that require fast response times, such as real-time control systems.
  • •When working with small-batch AI workloads that require high-performance inference.
  • •When integrating with specific models like Pi0, Pi0.5, GROOT N1.6, Pi0-FAST, or large language models like qwen3.6-27B.
Ready-to-paste prompt
./flashrt -m examples/pi0/model -i examples/pi0/input -b 1 -v
Heads up: Ensure you have a compatible C++ compiler and necessary dependencies installed on your system, as FlashRT is built using C++ and may require specific runtime versions or libraries to be installed.
Saves to your device
Use with Claude
New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

🛠️ Technical setup

Expect 20–40 minutes in a terminal — or let your AI agent drive it.

Try it instantly — no install
Claude Code
mkdir -p ~/.claude/skills/flashrt && curl -fsSL https://workflowstacks.com/api/skills/flashrt/claude-skill -o ~/.claude/skills/flashrt/SKILL.md
Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)
claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

How FlashRT: Accelerate Low‑Latency AI Inference on Jetson Edge Devices works
Codeflow
Free to inspect

FlashRT: Accelerate Low‑Latency AI Inference on Jetson Edge Devices is a very large C++ project (~621k lines across 1556 code files, plus 301 test files). It is a full software project: use it through its install path rather than reading it end to end. Last commit this month, Apache-2.0 license, has a test suite.

Size
Very large codebase
~621k lines · 1556 code files · days to read — use, don't read
Setup
Developer setup
A real software project. Use it via its install path; don't expect to read it all.
Runs on
Python · Docker
No API keys detected
C++ 57%Python 30%Cuda 11%HIP 1%
Where to start reading
  1. 1
    README.md
    Start here — what it does and how to install it
  2. 2
    pyproject.toml
    Dependencies and the commands it exposes
  3. 3
    flash_rt/__init__.py
    Inside flash_rt/ — the main logic begins here
  4. 4
    examples/README.md
    A worked example — copy this to get going
What's in each folder
flash_rt/Core code — the actual logic534 files
docs/Documentation95 files
examples/Examples you can copy36 files
csrc/Folder1351 files
cpp/Folder171 files
training/Folder54 files
serving/Folder38 files
scripts/Helper scripts22 files
READMEHas testsDocumentedNo CIDocker readyExamples includedApache-2.0 licenseUpdated this month
Quick Actions
Details
Creator
flashrt-project
Language
C++
Category
ai-agent
Published
4/28/2026

Are you the creator of this tool? and earn 85% of every sale.

Show it off in your README:

Featured on WorkflowStacks
[![Featured on WorkflowStacks](https://workflowstacks.com/api/badge/flashrt.svg)](https://workflowstacks.com/skills/flashrt?utm_source=github&utm_medium=badge)
🔥 Hot this week, in your inbox
Skills like this, every Monday.

The five fastest-growing open-source AI skills, ranked by GitHub star growth. One email a week, unsubscribe anytime.

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 303,974
superpowers
An agentic skills framework & software development methodology that works.
ai-agent★ 277,153
Track GitHub Stars with 996.ICU
Get a popular GitHub star counter for your startup, backed by 276k+ GitHub stars, ideal for founders.
ai-agent★ 253,255
Linux – 251k+ GitHub stars for building custom kernel solutions
Get the complete Linux kernel source in C, perfect for founders needing deep system control or tailored OS components.
ai-agent★ 27,533
agenticSeek
Fully Local Manus AI. No APIs, No $200 monthly bills. Enjoy an autonomous agent that thinks, browses the web, and code for the sole cost of electricity.
ai-agent★ 27,401
PicGo: Fast Image Uploading
Upload images easily with PicGo, supporting 60+ services. For founders needing efficient image management.
ai-agent★ 3,004
vllm-ascend: Accelerate LLM Inference on Ascend Hardware
Leverage vllm-ascend to run large language models efficiently on Ascend GPUs, cutting latency and cost. Ideal for AI startups needing high‑performance serving (2.9k+ GitHub stars).