Home/Inference & Serving/ggrun

ggrun

raketenkater/ggrun

Auto-tuned launcher for GGUF models on llama.cpp with OpenAI-compatible server

GitHub

GraphCanon updated Aug 13, 2026 · GitHub synced Aug 13, 2026

38views this month

264 stars14 forksLast push Aug 11, 2026 Go MIT

Decision brief

ggrun, an auto-tuned launcher for GGUF models using llama.cpp, offers OpenAI-compatible server support with multi-GPU tensor-split and MoE expert placement capabilities.

Good fit when

When developing systems that require automatic hardware optimization and tuning for GGUF models on multiple GPUs
If you need a deployment method that supports OpenAI compatible APIs while offering crash recovery mechanisms

Avoid when

For environments where single-GPU setups are preferred, as ggrun specializes in multi-GPU configurations and may offer limited advantage or additional complexity
When you do not require auto-tuning capabilities for hardware performance optimization since this feature is specific to ggrun

Pricing:: freemium - Free to use under MIT license; no direct costs involved in usage.

Observed Jul 17, 2026 · Source: enrich:decision_facts

ggrun alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Very active (1d since push); As of Aug 13, 2026
Provenance: Not a fork · Personal account; As of Aug 13, 2026
Security (OSV): No lockfile; As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

go get github.com/raketenkater/ggrun

pkg.go.dev

How it fits your stack(1)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Integrates

llama.cpp

`ggrun` requires `llama.cpp` to function as the engine for launching GGUF models, indicating a dependency.

integrates

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

Similar tools

Same-category neighbours not already linked as typed edges.

omlx

LLM inference server with continuous batching and SSD caching for Apple Silicon

19k★

lm-evaluation-harness

A framework for few-shot evaluation of language models.

14k★

oumi

Easily fine-tune, evaluate and deploy open source LLMs/VLMs

9.4k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Offers auto-tuning capabilities and an OpenAI-compatible server for GGUF models leveraging llama.cpp, supports multi-GPU tensor-split and MoE expert placement.

Capability facts

Languages: go
Source: github.language · Aug 13, 2026

README

Quick start Linux / macOS: Windows (PowerShell): Then run a local GGUF, download one from Hugging Face, or open the TUI: ```bash ggrun model.gguf ggrun unsloth/Qwen3.6 27B GGUF download

For agents

This page has a .md twin and JSON over the API.

/tools/raketenkater-ggrun.md GET /api/graphcanon/tools/raketenkater-ggrun GET /api/graphcanon/graph?tool=raketenkater-ggrun