Home/Inference & Serving/chatllm.cpp

chatllm.cpp

foldl/chatllm.cpp

C++ real-time chat models for CPU and GPU

GitHub

GraphCanon updated today · GitHub synced today

917 stars72 forksLast push 2d C++ MIT

Decision brief

This C++ library aims to deploy language models for real-time chatting on local systems with support for CPU and GPU.

Good fit when

When you need a C++ framework that can integrate tightly into existing C++ applications requiring fast chat responses.
If your project requires inference capabilities solely in C++, avoiding the use of Python or other languages often associated with AI frameworks like TensorFlow or PyTorch.

Avoid when

Avoid if your preferred development environment is centered around high-level languages such as Python, where alternatives like Transformers are robust and well-supported.
Not suitable for projects that require a wide array of pre-trained models not provided by chatllm.cpp itself, since it does not include model training functionalities.

Observed Jul 15, 2026 · Source: enrich:decision_facts

chatllm.cpp alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Very active (2d since push); As of today
Provenance: Not a fork · Personal account; As of today
Security (OSV): No lockfile; As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

git clone https://github.com/foldl/chatllm.cpp

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

llama.cpp

LLM inference in C/C++

123k★

whisper.cpp

Port of OpenAI's Whisper model in C/C++ for speech-to-text inference

53k★

ChuanhuChatGPT

GUI for ChatGPT and LLMs with features like agents and file-based QA.

15k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Implementation in C++ of various language model inference engines designed for real-time chatting on local systems with support for both CPU and GPU.

Capability facts

Languages: c++
Source: github.language · Aug 25, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Aug 25, 2026)

All Python scripts are going to be rewritten in [Nim](https://nim-lang.org/), with followin

Source link

README

Quick Start

As simple as main_nim -i -m :model_id. Check it out.

Plan Nim

All Python scripts are going to be rewritten in Nim, with following exceptions:

when pickle is used

For agents

This page has a .md twin and JSON over the API.

/tools/foldl-chatllm-cpp.md GET /api/graphcanon/tools/foldl-chatllm-cpp GET /api/graphcanon/graph?tool=foldl-chatllm-cpp