“sglang”

sglang

High-performance serving framework for large language and multimodal models

gpustack

A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.

ome

Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton

openmodelz

Autoscale LLM (vLLM, SGLang, LMDeploy) inferences on Kubernetes (and others)

dynamo

A Datacenter Scale Distributed Inference Serving Framework

kvcached

Virtualized Elastic KV Cache for Dynamic GPU Sharing and Beyond