GraphCanon updated 3w · GitHub synced 3w
Decision brief
LLMKube is a Kubernetes operator designed for deploying and scaling Language Model (LM) inference across different GPU types, supporting multiple runtimes.
Good fit when
- Use LLMKube if you need to run self-hosted Language Model inference with support for various GPU types like NVIDIA CUDA, AMD Vulkan, or Apple Silicon Metal.
- Ideal for environments where Kubernetes orchestration is a requirement and where multi-GPU sharding and model caching capabilities are needed for efficient resource utilization.
Avoid when
- Avoid LLMKube if your deployment environment strictly limits the use of Kubernetes or does not support the specified GPU types - NVIDIA CUDA, AMD Vulkan, Apple Silicon Metal.
- Not recommended for users who require a solution that only supports specific models or runtimes which are not covered by the runtime options provided (llama.cpp, vLLM, TGI, mlx-server).
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of 3w
- Provenance
- Not a fork · Organization account
- As of 3w
- Security (OSV)
- No criticals
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
go get github.com/defilantech/LLMKubeSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Provides Kubernetes-based orchestration for deploying and scaling Language Model inference across various GPU types including NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Supports multiple runtimes such as llama.cpp, vLLM, TGI, and mlx-server.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Aug 2, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Aug 2, 2026
- Languages
- go
Source: github.language · Aug 2, 2026
Categories
Tags
README
Install the CLI
brew install defilantech/tap/llmkube
Install the operator on any K8s cluster
helm repo add llmkube https://defilantech.github.io/LLMKube helm install llmkube llmkube/llmkube --namespace llmkube-system --create-namespace
License
Apache 2.0 — see LICENSE.
For agents
This page has a .md twin and JSON over the API.