sglang
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
SGLang is a high-performance serving framework for large language models and multimodal models.
Categories
Tags
Similar tools
ollama
ollama/ollama
Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.
prompts.chat
f/prompts.chat
The world's largest open-source prompt library for AI
transformers
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models
open-webui
open-webui/open-webui
User-friendly AI Interface (Supports Ollama, OpenAI API, ...)
awesome-llm-apps
Shubhamsaboo/awesome-llm-apps
100+ AI Agent & RAG apps you can actually run — clone, customize, ship.
LLMs-from-scratch
rasbt/LLMs-from-scratch
Implement a ChatGPT-like LLM in PyTorch from scratch
Install
pip install sglangREADME
Blog | Documentation | Roadmap | Join Slack | Weekly Dev Meeting | Slides
News
- [2026/06] 🔥 The next generation of speculative decoding: DFlash and Spec V2 (blog).
- [2026/04] 🔥 DeepSeek-V4 on Day 0: From Fast Inference to Verified RL with SGLang and Miles (blog).
- [2026/06] SGLang provides day-0 support for latest open models (Nemotron 3 Ultra, Nemotron 3 Super, Higgs Audio v3 TTS).
- [2026/02] 🔥 Unlocking 25x Inference Performance with SGLang on NVIDIA GB300 NVL72 (blog).
- [2026/01] SGLang Diffusion accelerates video and image generation (blog).
- [2025/12] SGLang provides day-0 support for latest open models (MiMo-V2-Flash, Nemotron 3 Nano, Mistral Large 3, LLaDA 2.0 Diffusion LLM, MiniMax M2).
- [2025/10] SGLang now runs natively on TPU with the SGLang-Jax backend (blog).
More
-
[2025/09] Deploying DeepSeek on GB200 NVL72 with PD and Large Scale EP (Part II): 3.8x Prefill, 4.8x Decode Throughput (blog).
-
[2025/09] SGLang Day 0 Support for DeepSeek-V3.2 with Sparse Attention (blog).
-
[2025/08] SGLang x AMD SF Meetup on 8/22: Hands-on GPU workshop, tech talks by AMD/xAI/SGLang, and networking (Roadmap, Large-scale EP, Highlights, AITER/MoRI, Wave).
-
[2025/11] SGLang Diffusion accelerates video and image generation (blog).
-
[2025/10] PyTorch Conference 2025 SGLang Talk (slide).
-
[2025/10] SGLang x Nvidia SF Meetup on 10/2 (recap).
-
[2025/08] SGLang provides day-0 support for OpenAI gpt-oss model (instructions)
-
[2025/06] SGLang, the high-performance serving infrastructure powering trillions of tokens daily, has been awarded the third batch of the Open Source AI Grant by a16z (a16z blog).
-
[2025/05] Depl