Home/Inference & Serving/yalm

yalm

andrewkchan/yalm

LLM inference engine in C++/CUDA without dependency on external libraries except for I/O

GitHub

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

57views this month

600 stars65 forksLast push Sep 13, 2025 C++

Decision brief

YALM offers a no-frills LLM inference engine in C++/CUDA, optimized for tasks requiring minimal external dependencies beyond I/O and no reliance on heavyweight ML libraries.

Good fit when

When your project's stack is primarily based on C++ and CUDA, allowing seamless integration without additional dependencies
In scenarios where performance optimization is paramount and you require a lightweight solution that directly leverages hardware capabilities via CUDA

Avoid when

If extensive functionality or ease of use from other ML libraries is required, as YALM does not support dependencies beyond I/O needs
For developers who prefer tools with broader community support and more comprehensive feature sets, given that YALM specializes in a narrow scope

Observed Jul 16, 2026 · Source: enrich:decision_facts

yalm alternatives Trust report

yalm vs whisper.cpp

Verify the decision

Maintenance and security

Full trust report

Maintenance: Dormant (371d since push); As of Sep 20, 2026
Provenance: Not a fork · Personal account; As of Sep 20, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

git clone https://github.com/andrewkchan/yalm

How it fits your stack(1)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Alternative

whisper.cpp

YALM focuses on LLM inference in C++/CUDA, similar to whisper.cpp which also focuses on efficient execution of models with C/C++.

alternative

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

Similar tools

Same-category neighbours not already linked as typed edges.

llama.cpp

LLM inference in C/C++

129k★

litgpt

High-performance LLMs for pretraining, finetuning, and deployment

14k★

bitsandbytes

Large language model quantization toolkit for PyTorch.

8.5k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

A lightweight LLM inference solution implemented in C++ and CUDA without the need for additional machine learning libraries, focusing solely on LLM inference tasks.

Capability facts

Languages: c++
Source: github.language · Sep 20, 2026

README

install git LFS curl s https://packagecloud.io/install/repositories/github/git lfs/script.deb.sh sudo bash sudo apt get y install git lfs

For agents

This page has a .md twin and JSON over the API.

/tools/andrewkchan-yalm.md GET /api/graphcanon/tools/andrewkchan-yalm GET /api/graphcanon/graph?tool=andrewkchan-yalm