{"data":{"slug":"foldl-chatllm-cpp","name":"chatllm.cpp","tagline":"C++ real-time chat models for CPU and GPU","github_url":"https://github.com/foldl/chatllm.cpp","owner":"foldl","repo":"chatllm.cpp","owner_avatar_url":"https://avatars.githubusercontent.com/u/4046440?v=4","primary_language":"C++","stars":917,"forks":72,"topics":["llm","llm-inference"],"archived":false,"github_pushed_at":"2026-08-22T08:30:42+00:00","maintenance_label":"Very active","stars_delta_30d":5,"url":"https://www.graphcanon.com/tools/foldl-chatllm-cpp","markdown_url":"https://www.graphcanon.com/tools/foldl-chatllm-cpp.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/foldl-chatllm-cpp","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=foldl-chatllm-cpp","description":"Pure C++ implementation of several models for real-time chatting on your computer (CPU & GPU)","homepage_url":null,"license":"MIT","open_issues":11,"watchers":22,"ai_summary":"Implementation in C++ of various language model inference engines designed for real-time chatting on local systems with support for both CPU and GPU.","readme_excerpt":"## Quick Start\n\nAs simple as `main_nim -i -m :model_id`. [Check it out](./docs/quick_start.md).\n\n---\n\n## Plan Nim\n\nAll Python scripts are going to be rewritten in [Nim](https://nim-lang.org/), with following exceptions:\n\n* when `pickle` is used","github_created_at":"2023-12-02T08:48:18+00:00","created_at":"2026-07-11T11:44:07.232816+00:00","updated_at":"2026-08-25T00:01:22.968305+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"cpu-support","name":"cpu-support"},{"slug":"gpu-support","name":"gpu-support"},{"slug":"llm","name":"llm"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"real-time-chatting","name":"real-time-chatting"}],"trust":{"provenance":{"is_fork":false,"github_id":726390185,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T00:01:22.267Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":2,"days_since_push":2,"last_release_at":"2026-08-15T09:50:24Z","stars_delta_30d":5,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:44:08.360Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T00:01:22.705Z"},"languages":{"value":["c++"],"source":"github.language","observed_at":"2026-08-25T00:01:22.705Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-25T00:01:22.705Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need a C++ framework that can integrate tightly into existing C++ applications requiring fast chat responses.","If your project requires inference capabilities solely in C++, avoiding the use of Python or other languages often associated with AI frameworks like TensorFlow or PyTorch."],"when_not_to_use":["Avoid if your preferred development environment is centered around high-level languages such as Python, where alternatives like Transformers are robust and well-supported.","Not suitable for projects that require a wide array of pre-trained models not provided by chatllm.cpp itself, since it does not include model training functionalities."],"source":"enrich:decision_facts","observed_at":"2026-07-15T08:34:14.905Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"This C++ library aims to deploy language models for real-time chatting on local systems with support for CPU and GPU."}]}}