Home/Inference & Serving/chatllm.cpp

chatllm.cpp

foldl/chatllm.cpp

C++ real-time chat models for CPU and GPU

GitHub

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

58views this month

929 stars75 forksLast push Sep 7, 2026 C++ MIT

Decision brief

This C++ library aims to deploy language models for real-time chatting on local systems with support for CPU and GPU.

Good fit when

When you need a C++ framework that can integrate tightly into existing C++ applications requiring fast chat responses.
If your project requires inference capabilities solely in C++, avoiding the use of Python or other languages often associated with AI frameworks like TensorFlow or PyTorch.

Avoid when

Avoid if your preferred development environment is centered around high-level languages such as Python, where alternatives like Transformers are robust and well-supported.
Not suitable for projects that require a wide array of pre-trained models not provided by chatllm.cpp itself, since it does not include model training functionalities.

Observed Jul 15, 2026 · Source: enrich:decision_facts

chatllm.cpp alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Active (11d since push); As of Sep 19, 2026
Provenance: Not a fork · Personal account; As of Sep 19, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

git clone https://github.com/foldl/chatllm.cpp

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

llama.cpp

LLM inference in C/C++

129k★

whisper.cpp

C++ port of OpenAI's Whisper model for speech-to-text

54k★

ChuanhuChatGPT

GUI for ChatGPT and LLMs with features like agents and file-based QA.

15k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Implementation in C++ of various language model inference engines designed for real-time chatting on local systems with support for both CPU and GPU.

Capability facts

Languages: c++
Source: github.language · Sep 19, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 19, 2026)

All Python scripts are going to be rewritten in [Nim](https://nim-lang.org/), with followin

Source link

README

Quick Start As simple as . Check it out. Plan Nim All Python scripts are going to be rewritten in Nim, with following exceptions: when is used

For agents

This page has a .md twin and JSON over the API.

/tools/foldl-chatllm-cpp.md GET /api/graphcanon/tools/foldl-chatllm-cpp GET /api/graphcanon/graph?tool=foldl-chatllm-cpp