Search results
“inference & serving”
13 tools · ranked by relevance and freshness, not ad spend.
Result neighbourhood
Top matches for your query, linked as a graph hub. Follow any node to drill in.
A high-throughput and memory-efficient inference and serving engine for LLMs
The easiest way to serve AI apps and models
Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs
Unified framework for scaling AI and Python applications
分享大模型技术原理与实战经验，涵盖工程化和应用落地
Self-Hosting LLMs Made Easy
End-to-end framework for building custom AI applications and agents.
Artificial Intelligence Controller Interface (AICI)
Community maintained hardware plugin for vLLM on Ascend
Awesome System for Machine Learning ⚡️ AI System Papers and Industry Practice
High-speed Large Language Model Serving for Local Deployment
RTP-LLM: Alibaba's high-performance LLM inference engine for diverse applications.
SGLang high-performance serving framework