Home/Inference & Serving/rkllama

rkllama

NotPunchnox/rkllama

Ollama alternative for Rockchip NPU with optimized AI and Deep learning model inference

GitHub

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

43views this month

605 stars99 forksLast push Aug 25, 2026 Python GPL-3.0

Decision brief

Ollama alternative for Rockchip NPU: optimized AI and deep learning inference on Rockchip devices

Good fit when

You need to run models specifically optimized for Rockchip Neural Processing Unit (NPU)
Your deployment involves Rockchip hardware like rk3576, rk3588, Orange Pi 5 Pro

Avoid when

Your hardware does not include a Rockchip NPU
You are looking for an AI solution that works across multiple non-Rockchip platforms

Observed Jul 16, 2026 · Source: enrich:decision_facts

rkllama alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Active (25d since push); As of Sep 20, 2026
Provenance: Not a fork · Personal account; As of Sep 20, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install rkllama

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

181k★

oumi

Easily fine-tune, evaluate and deploy open source LLMs/VLMs

9.4k★

whichllm

Command-line tool to find and benchmark local LLM performance

6.7k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

This repository provides an efficient solution for running AI and deep learning models on Rockchip devices, featuring enhanced support for Rockchip Neural Processing Unit (NPU) through rkllm.

Capability facts

Deploy: Self-host
Source: dockerfile:Dockerfile · Sep 20, 2026
Docker: Dockerfile present
Source: dockerfile:Dockerfile · Sep 20, 2026
CLI: CLI entrypoint
Source: pyproject.toml:[project.scripts] · Sep 20, 2026
Languages: python
Source: github.language+pyproject.toml · Sep 20, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 20, 2026)

python -m pip install .

Source link

README

Tested Hardware and Environment Hardware : Orange Pi 5 Pro: (Rockchip RK3588S, NPU 6 TOPS), 16GB RAM. Hardware : Orange Pi 5 Plus: (Rockchip RK3588S, NPU 6 TOPS), 16GB RAM. Hardware : Orange Pi 5 Max: (Rockchip RK3588S, NPU 6 TOPS), 16GB RAM. Hardware : Radxa Rock 4d: (Rockchip R...

For agents

This page has a .md twin and JSON over the API.

/tools/notpunchnox-rkllama.md GET /api/graphcanon/tools/notpunchnox-rkllama GET /api/graphcanon/graph?tool=notpunchnox-rkllama