Results for “inference”

PowerInfer

Tiiny-AI/PowerInfer

High-speed Large Language Model Serving for Local Deployment

vllm-ascend

vllm-project/vllm-ascend

Community maintained hardware plugin for vLLM on Ascend