local-ai

Run AI Locally with llmfit

Name: Run AI Locally with llmfit
Author: AlexsJones

Find compatible AI models for your hardware with llmfit, a tool for founders, with 29k+ GitHub stars.

31,165 stars1,912 forksRustHealth Score 9/10Updated 8/5/2026100% free · open source

What it does

Llmfit helps founders find compatible AI models that can run on their specific hardware with a single command, simplifying the process of selecting appropriate models for their projects.

Install / run

cargo install llmfit

When to use it

•When you need to quickly determine which AI models can run on your local machine without extensive research.
•When you're working with limited hardware resources and need to optimize AI model selection for performance.
•When you want to explore and compare various AI models from different providers to find the best fit for your startup's needs.

Quick start

1Run `llmfit discover` to find compatible models for your hardware.
2Use `llmfit list` to view a list of available models and providers.
3Utilize `llmfit info <model_name>` to get detailed information about a specific model.
4Filter results by specifying a provider with `llmfit discover --provider <provider_name>`.
5Save your favorite models for easy reference with `llmfit favorites add <model_name>`.

Ready-to-paste prompt

llmfit discover --provider huggingface

Heads up: Ensure you have Rust and Cargo installed on your system, as llmfit is built with Rust and relies on Cargo for installation and management.

Saves to your device

Topics

gguf

llm

localai

mlx

skill

unsloth

How Run AI Locally with llmfit works

Codeflow

Free to inspect

Run AI Locally with llmfit is a large Rust project (~54k lines across 68 code files, plus 4 test files). Setup is light: installs like a normal app. Reading the code is optional. Last commit this month, MIT license, has a test suite.

Size

Large codebase

~54k lines · 68 code files · ~8 h to skim

Setup

One-command install

Installs like a normal app. Reading the code is optional.

Runs on

Rust · Docker

No API keys detected

Rust 82%Python 9%JavaScript 6%CSS 2%Shell 1%

Where to start reading

1
README.md
Start here — what it does and how to install it
2
AGENTS.md
The instructions the AI actually follows
3
Cargo.toml
Dependencies and the commands it exposes
4
skills/llmfit-advisor/SKILL.md
Inside skills/ — an example of what the AI is told to do

What's in each folder

skills/Prompts, skills & agent definitions1 files

docs/Documentation9 files

llmfit-core/Folder110 files

llmfit-web/Folder27 files

llmfit-desktop/Folder15 files

llmfit-tui/Folder15 files

scripts/Helper scripts9 files

llmfit-python/Folder8 files