3 tools
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
mlc-ai/mlc-llm
Universal LLM Deployment Engine with ML Compilation
openlit/openlit
Open source platform for AI Engineering