---
title: "mlc-llm"
type: "tool"
slug: "mlc-ai-mlc-llm"
canonical_url: "https://www.graphcanon.com/tools/mlc-ai-mlc-llm"
github_url: "https://github.com/mlc-ai/mlc-llm"
homepage_url: "https://llm.mlc.ai/"
stars: 22917
forks: 2080
primary_language: "Python"
license: "Apache-2.0"
categories: ["llm-frameworks"]
tags: ["llm", "tvm", "machine-learning-compilation", "language-model"]
updated_at: "2026-07-07T18:25:31.733933+00:00"
---
# mlc-llm
> Universal LLM Deployment Engine with ML Compilation
MLC-LLM is a machine learning compiler and high-performance deployment engine designed to optimize large language models for various platforms, including AMD GPU, NVIDIA GPU, Apple GPU, Intel GPU, Web Browser, iOS/ iPadOS, Android. It provides an OpenAI-compatible API through REST server and supports multiple programming languages.
## Facts
- Repository: https://github.com/mlc-ai/mlc-llm
- Homepage: https://llm.mlc.ai/
- Stars: 22,917 · Forks: 2,080 · Open issues: 317 · Watchers: 193
- Primary language: Python
- License: Apache-2.0
- Last pushed: 2026-07-07T16:34:10+00:00
## Categories
- [LLM Frameworks](/categories/llm-frameworks.md)
## Tags
llm, tvm, machine-learning-compilation, language-model
## Related tools
- [ollama](/tools/ollama-ollama.md) - Get up and running with Kimi-K2.6, GLM-5.1, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models. (★ 175,659)
- [prompts.chat](/tools/f-prompts-chat.md) - The world's largest open-source prompt library for AI (★ 165,019)
- [transformers](/tools/huggingface-transformers.md) - 🤗 Transformers: the model-definition framework for state-of-the-art machine learning models (★ 162,347)
- [open-webui](/tools/open-webui-open-webui.md) - User-friendly AI Interface (Supports Ollama, OpenAI API, ...) (★ 144,575)
- [awesome-llm-apps](/tools/shubhamsaboo-awesome-llm-apps.md) - 100+ AI Agent & RAG apps you can actually run — clone, customize, ship. (★ 116,702)
- [LLMs-from-scratch](/tools/rasbt-llms-from-scratch.md) - Implement a ChatGPT-like LLM in PyTorch from scratch (★ 98,711)
- [TradingAgents](/tools/tauricresearch-tradingagents.md) - TradingAgents: Multi-Agents LLM Financial Trading Framework (★ 91,610)
- [caveman](/tools/juliusbrussee-caveman.md) - Cuts 65% of tokens in AI coding agent responses. (★ 86,150)
## README (excerpt)
```text
# MLC LLM
**Universal LLM Deployment Engine with ML Compilation**
[Get Started](https://llm.mlc.ai/docs/get_started/quick_start) | [Documentation](https://llm.mlc.ai/docs) | [Blog](https://blog.mlc.ai/)
## About
MLC LLM is a machine learning compiler and high-performance deployment engine for large language models. The mission of this project is to enable everyone to develop, optimize, and deploy AI models natively on everyone's platforms.
| |
AMD GPU |
NVIDIA GPU |
Apple GPU |
Intel GPU |
| Linux / Win |
✅ Vulkan, ROCm |
✅ Vulkan, CUDA |
N/A |
✅ Vulkan |
| macOS |
✅ Metal (dGPU) |
N/A |
✅ Metal |
✅ Metal (iGPU) |
| Web Browser |
✅ WebGPU and WASM |
| iOS / iPadOS |
✅ Metal on Apple A-series GPU |
| Android |
✅ OpenCL on Adreno GPU |
✅ OpenCL on Mali GPU |
MLC LLM compiles and runs code on MLCEngine -- a unified high-performance LLM inference engine across the above platforms. MLCEngine provides OpenAI-compatible API available through REST server, python, javascript, iOS, Android, all backed by the same engine and compiler that we keep improving with the community.
## Get Started
Please visit our [documentation](https://llm.mlc.ai/docs/) to get started with MLC LLM.
- [Installation](https://llm.mlc.ai/docs/install/mlc_llm)
- [Quick start](https://llm.mlc.ai/docs/get_started/quick_start)
- [Introduction](https://llm.mlc.ai/docs/get_started/introduction)
## Citation
Please consider citing our project if you find it useful:
```bibtex
@software{mlc-llm,
author = {{MLC team}},
title = {{MLC-LLM}},
url = {https://github.com/mlc-ai/mlc-llm},
year = {2023-2025}
}
```
The underlying techniques of MLC LLM include:
References (Click to expand)
```bibtex
@inproceedings{tensorir,
author = {Feng, Siyuan and Hou, Bohan and Jin, Hongyi and Lin, Wuwei and Shao, Junru and Lai, Ruihang and Ye, Zihao and Zheng, Lianmin and Yu, Cody Hao and Yu, Yong and Chen, Tianqi},
title = {TensorIR: An Abstraction for Automatic Tensorized Program Optimization},
year = {2023},
isbn = {9781450399166},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3575693.3576933},
doi = {10.1145/3575693.3576933},
booktitle = {Proceedings of the 28th ACM International Conference on Architectural Support for Programming Languages and Operating Systems, Volume 2},
pages = {804–817},
numpages = {14},
keywords = {Tensor Computation, Machine Learning Compiler, Deep Neural Network},
location = {Vancouver, BC, Canada},
series = {ASPLOS 2023}
}
@inproceedings{metaschedule,
author = {Shao, Junru and Zhou, Xiyou and Feng, Siyuan and Hou, Bohan and Lai, Ruihang and Jin, Hongyi and Lin, Wuwei and Masuda, Masahiro and Yu, Cody Hao and Chen, Tianqi},
booktitle = {Advances in Neural Information Processing Systems},
editor = {S. Koyejo and S. Mohamed and A. Agarwal and D. Belgrave and K. Cho and A. Oh},
pages = {35783--35796},
publisher = {Curran Associates, Inc.},
title = {Tensor Program Optimization with Probabilistic Programs},
url = {https://proceedings.neurips.cc/paper_files/paper/2022/file/e894eafae43e68b4c8dfdacf742bcbf3-Paper-Conference.pdf},
volume = {35},
year = {2
```
---
**Machine-readable endpoints**
- JSON: [`/api/graphcanon/tools/mlc-ai-mlc-llm`](/api/graphcanon/tools/mlc-ai-mlc-llm)
- LLM index: [/llms.txt](/llms.txt)
- Full corpus: [/llms-full.txt](/llms-full.txt)
_GraphCanon - The knowledge graph for AI development. https://www.graphcanon.com/_