ai-agent

llama.cpp

Name: llama.cpp
Author: ggml-org

LLM inference in C/C++

114,247 stars19,052 forksC++Updated 6/2/2026100% free · open source

What it does

Llama.cpp enables fast and efficient inference of large language models in C/C++ applications, allowing developers to integrate AI capabilities into their projects

When to use it

•When you need to deploy a language model in a resource-constrained environment
•When you want to integrate a language model into a C/C++ application
•When you need to achieve high-performance inference for large language models

Quick start

1Compile the project using the command 'mkdir build && cd build && cmake .. && cmake --build .'
2Download the pre-trained model weights using the provided script 'download-weights.sh'
3Create a C++ application that includes the 'llama.cpp' header file and links against the 'libllama.so' library
4Initialize the model using the 'LlamaModel' class and load the pre-trained weights
5Use the 'generate' function to generate text based on a given prompt

Ready-to-paste prompt

cout << LlamaModel::generate("Tell me a story about a character who", 100) << endl;

Topics

ggml

Quick Actions

Details

Creator

ggml-org

Language

C++

Related skills

More ai-agent tools founders pair with this one.

ai-agent★ 172,909

Ollama: Run AI Models Locally on Your Own Machine

Instantly deploy top open-source AI models without cloud dependencies. Perfect for founders building AI applications with complete privacy and control.

ai-agent★ 165,332

Karpathy Skills: AI Coding Wisdom for Smarter Development

Unlock advanced LLM coding strategies from a leading AI researcher. Essential reference for founders building intelligent code agents and AI products.

ai-agent★ 161,190

Transformers: Ship AI Models in Minutes, Not Months

Quickly define, train, and deploy state-of-the-art machine learning models across text, vision, and audio. Perfect for AI founders building intelligent applications.

ai-agent★ 81,719

RAGFlow: Build Context-Aware AI Agents 10x Faster

Python toolkit for startup founders wanting powerful retrieval-augmented generation with advanced agent capabilities. Supercharge your LLM context management with open-source AI infrastructure.

ai-agent★ 75,645

OpenHands: AI-Driven Dev

Get AI-driven development tools with OpenHands. For founders and developers.

ai-agent★ 46,753

cherry-studio: Boost AI Productivity

Get unified access to frontier LLMs with cherry-studio, for founders. 47k+ GitHub stars