vLLM

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

semantic-router

System level intelligent runtime for Mixture-of-Models across edge, data center and cloud

vllm-ascend

Community maintained hardware plugin for vLLM on Ascend