Home/Inference & Serving/KuiperLLama

KuiperLLama

zjhellofss/KuiperLLama

LLama2/3 and Qwen2.5 large-model inference framework

GitHub

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

32views this month

575 stars145 forksLast push Oct 28, 2025 C++

Decision brief

KuiperLLama is an inference framework project for LLama2/3 and Qwen2.5 models using C++ and CUDA. It provides guidance to implement a custom inference engine from scratch.

Good fit when

To build personalized inference engines tailored explicitly for LLama2/3 or Qwen2.5
For developers with strong C++ and CUDA expertise seeking precision control

Avoid when

If aiming to support a broader range of models beyond LLama2/3 and Qwen2.5
Without substantial experience in C++ and CUDA programming frameworks

Observed Jul 16, 2026 · Source: enrich:decision_facts

KuiperLLama alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Slowing (327d since push); As of Sep 20, 2026
Provenance: Not a fork · Personal account; As of Sep 20, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

git clone https://github.com/zjhellofss/KuiperLLama

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

181k★

transformers

🤗 Transformers: state-of-the-art machine learning models for text, vision, audio, and multimodal models

166k★

open-webui

User-friendly AI Interface

152k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

A project that guides users to implement an inference engine for Llama2/3 and Qwen2.5 from scratch using C++ and CUDA.

Capability facts

Languages: c++
Source: github.language · Sep 20, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 20, 2026)

python export.py llama2_7b.bin --meta-llama path/to/llama/model/7B

Source link

README

KuiperLLama 动手自制大模型推理框架，支持LLama2/3和Qwen2.5 News：新课发布，《动手自制大模型推理框架》，全手写cuda算子，课程框架支持LLama2和3.x以及Qwen2.5模型 Hi，各位朋友们好！我是 KuiperInfer 的作者。KuiperInfer 作为一门开源课程，迄今已经在 GitHub 上已斩获 2.5k star。如今在原课程的基础上，我们全新推出了《动手自制大模型推理框架》，新课程支持Llama系列大模型（包括最新的LLama3.2）以及Qwen2.5系列大模型，同时支持 Cuda 加速和 Int...

For agents

This page has a .md twin and JSON over the API.

/tools/zjhellofss-kuiperllama.md GET /api/graphcanon/tools/zjhellofss-kuiperllama GET /api/graphcanon/graph?tool=zjhellofss-kuiperllama