LLMKube
Enrichment pendingKubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, mo
GraphCanon updated today · GitHub synced today
Trust & integrity
Full report- Maintenance
- Very active (0d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Organization account
- As of today · Source: github_public_v1
- Security (OSV)
- No criticals
- As of today · Source: osv@v1
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Jul 11, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Jul 11, 2026
- Languages
- go
Source: github.language · Jul 11, 2026
Categories
Tags
README
Install the CLI
brew install defilantech/tap/llmkube
Install the operator on any K8s cluster
helm repo add llmkube https://defilantech.github.io/LLMKube helm install llmkube llmkube/llmkube --namespace llmkube-system --create-namespace
License
Apache 2.0 — see LICENSE.