“moe”

MOE

A global, black box optimization engine for real world metric optimization.

vllm

A high-throughput and memory-efficient inference and serving engine for LLMs

LlamaFactory

Unified Efficient Fine-Tuning of 100+ LLMs & VLMs

sglang

High-performance serving framework for large language and multimodal models

TensorRT-LLM

Python API for defining and optimizing Large Language Models (LLMs) on NVIDIA GPUs

flashinfer

FlashInfer: Kernel Library for LLM Serving

openinfer

Pure Rust + CUDA LLM inference engine — no PyTorch, OpenAI-compatible, serves Qwen3 to Kimi-K2