local-ai

memra

Name: memra
Author: avifenesh

memra is a Rust + CUDA inference engine for serving GGUF models on NVIDIA GPUs. It is optimized for Blackwell (sm_120a), with a separately gated Hopper/H100

From the repo · “Rust + CUDA inference engine for NVIDIA RTX PRO 6000 Blackwell and RTX 5090. Serves safetensors and GGUF over an OpenAI-compatible API, with per-device tuned defaults and speculative decode gated byte-identical to plain decode. Hosted instance: inference.tiyuvta.ai”

4 stars+2★ this week1 forksRustUpdated 10/6/2026100% free · open source

What it does

Memra is a Rust + CUDA inference engine that serves GGUF models on NVIDIA GPUs, optimized for Blackwell and Hopper/H100 architectures.

When to use it

•You need to deploy GGUF models on NVIDIA GPUs for high-performance inference.
•Your startup requires a customized inference engine for specific GPU architectures.
•You want to leverage Rust and CUDA for building a high-speed AI application.

Ready-to-paste prompt

cargo run --release -- -m path/to/model.gguf -i path/to/input.data

Heads up: Ensure you have the CUDA toolkit installed and configured properly on your system, as memra relies on it for GPU acceleration.

Saves to your device

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/memra && curl -fsSL https://workflowstacks.com/api/skills/memra/claude-skill -o ~/.claude/skills/memra/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

What's inside — free to inspect

No purchase needed

Read the entire source before you build — unlike paid marketplaces that hide it behind a buy button.

top-level files

folders

1322.5M

source size

Other

license

Key files

AGENTS.md

README.md

File tree

.cargo/

.github/

agent-knowledge/

crates/

deploy/

docs/

probe/

research/

tools/

.gitattributes

.gitignore

.ignore

AGENTS.md

ARCHITECTURE-H100.md

ARCHITECTURE.md

ASR-MODALITY-PLAN.md

bench_vllm.py

Cargo.lock

Cargo.toml

CITATION.cff

CONTRIBUTING.md

LICENSE

README.md

rust-toolchain.toml

Use with Claude

New

Skip the builder — one click puts this in Claude, Cursor, Antigravity and more.

✅ Light setup

Installs with a command or two; your AI agent can do it for you.

Try it instantly — no install

Claude Code

mkdir -p ~/.claude/skills/memra && curl -fsSL https://workflowstacks.com/api/skills/memra/claude-skill -o ~/.claude/skills/memra/SKILL.md

Open in another AI app

Opens the app with this repo with the prompt ready to go — no copy-paste needed.

Connect the whole catalog (MCP)

claude mcp add --transport http workflowstacks https://workflowstacks.com/api/mcp

Adds a WorkflowStacks connector to Claude Code: search and load any skill here by chatting.

Quick Actions

Details

Creator

avifenesh

Language

Rust

Related skills

More local-ai tools founders pair with this one.

local-ai★ 132,005

Run Powerful LLMs Locally with llama.cpp – Fast C++ Inference

Empower your product with on‑device LLM inference using llama.cpp’s efficient C++ engine. Ideal for founders who need privacy, speed, and no cloud costs. 130k+ GitHub stars.

local-ai★ 94,719

Stirling-PDF: Edit PDFs Anywhere

Edit PDFs on any device with Stirling-PDF, a tool for startup founders, backed by 80k+ GitHub stars

local-ai★ 38,243

Run AI Locally with llmfit

Find compatible AI models for your hardware with llmfit, a tool for founders, with 29k+ GitHub stars.

local-ai★ 1,238

crit: Streamline Your AI Agent Feedback Loop

crit lets founders quickly iterate with AI agents via a simple CLI, turning feedback into actionable code improvements. Ideal for Go‑based dev teams. 1.2k+ GitHub stars

local-ai★ 16,264

MNN: Accelerate On‑Device AI with a 16k+ Star Inference Engine

Deploy high‑performance LLMs and edge AI models on embedded hardware fast. Ideal for founders building C++‑based AI products for mobile, IoT, or edge devices.

local-ai★ 765

sno-station

Sno Station — your Claude Code and Codex working as one squad on your own machine. Shared encrypted memory, agent-to-agent messaging (Reach), squad skills for handoff and cross-vendor review, and a nightly loop that rewrites the agents' own skills with your approval. Open source, no daemon, no cloud required. Assembled in public.