{"data":{"slug":"lm-sys-fastchat","name":"FastChat","tagline":"An open platform for training, serving, and evaluating large language models","github_url":"https://github.com/lm-sys/FastChat","owner":"lm-sys","repo":"FastChat","owner_avatar_url":"https://avatars.githubusercontent.com/u/126381704?v=4","primary_language":"Python","stars":39490,"forks":4788,"topics":[],"archived":false,"github_pushed_at":"2026-05-01T00:25:53+00:00","maintenance_label":"Steady","url":"https://www.graphcanon.com/tools/lm-sys-fastchat","markdown_url":"https://www.graphcanon.com/tools/lm-sys-fastchat.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/lm-sys-fastchat","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=lm-sys-fastchat","description":"An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.","homepage_url":null,"license":"Apache-2.0","open_issues":1027,"watchers":355,"ai_summary":"FastChat provides capabilities to train, serve, evaluate, and compare large language model-based chatbots. It includes components for multi-model serving systems with web and RESTful API interfaces.","readme_excerpt":"# FastChat\n| [**Demo**](https://lmarena.ai/) | [**Discord**](https://discord.gg/6GXcFg3TH8) | [**X**](https://x.com/lmsysorg) |\n\nFastChat is an open platform for training, serving, and evaluating large language model based chatbots.\n- FastChat powers Chatbot Arena ([lmarena.ai](https://lmarena.ai)), serving over 10 million chat requests for 70+ LLMs.\n- Chatbot Arena has collected over 1.5M human votes from side-by-side LLM battles to compile an online [LLM Elo leaderboard](https://lmarena.ai/?leaderboard).\n\nFastChat's core features include:\n- The training and evaluation code for state-of-the-art models (e.g., Vicuna, MT-Bench).\n- A distributed multi-model serving system with web UI and OpenAI-compatible RESTful APIs.\n\n## News\n- [2024/03] 🔥 We released Chatbot Arena technical [report](https://arxiv.org/abs/2403.04132).\n- [2023/09] We released **LMSYS-Chat-1M**, a large-scale real-world LLM conversation dataset. Read the [report](https://arxiv.org/abs/2309.11998).\n- [2023/08] We released **Vicuna v1.5** based on Llama 2 with 4K and 16K context lengths. Download [weights](#vicuna-weights).\n- [2023/07] We released **Chatbot Arena Conversations**, a dataset containing 33k conversations with human preferences. Download it [here](https://huggingface.co/datasets/lmsys/chatbot_arena_conversations).\n\n
\nMore\n\n- [2023/08] We released **LongChat v1.5** based on Llama 2 with 32K context lengths. Download [weights](#longchat).\n- [2023/06] We introduced **MT-bench**, a challenging multi-turn question set for evaluating chatbots. Check out the blog [post](https://lmsys.org/blog/2023-06-22-leaderboard/).\n- [2023/06] We introduced **LongChat**, our long-context chatbots and evaluation tools. Check out the blog [post](https://lmsys.org/blog/2023-06-29-longchat/).\n- [2023/05] We introduced **Chatbot Arena** for battles among LLMs. Check out the blog [post](https://lmsys.org/blog/2023-05-03-arena).\n- [2023/03] We released **Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality**. Check out the blog [post](https://vicuna.lmsys.org).\n\n
\n\n\n\n## Contents\n- [Install](#install)\n- [Model Weights](#model-weights)\n- [Inference with Command Line Interface](#inference-with-command-line-interface)\n- [Serving with Web GUI](#serving-with-web-gui)\n- [API](#api)\n- [Evaluation](#evaluation)\n- [Fine-tuning](#fine-tuning)\n- [Citation](#citation)\n\n## Install\n\n### Method 1: With pip\n\n```bash\npip3 install \"fschat[model_worker,webui]\"\n```\n\n### Method 2: From source\n\n1. Clone this repository and navigate to the FastChat folder\n```bash\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\n```\n\nIf you are running on Mac:\n```bash\nbrew install rust cmake\n```\n\n2. Install Package\n```bash\npip3 install --upgrade pip # enable PEP 660 support\npip3 install -e \".[model_worker,webui]\"\n```\n\n## Model Weights\n### Vicuna Weights\n[Vicuna](https://lmsys.org/blog/2023-03-30-vicuna/) is based on Llama 2 and should be used under Llama's [model license](https://github.com/facebookresearch/llama/blob/main/LICENSE).\n\nYou can use the commands below to start chatting. It will automatically download the weights from Hugging Face repos.\nDownloaded weights are stored in a `.cache` folder in the user's home folder (e.g., `~/.cache/huggingface/hub/`).\n\nSee more command options and how to handle out-of-memory in the \"Inference with Command Line Interface\" section below.\n\n**NOTE: `transformers>=4.31` is required for 16K versions.**\n\n| Size | Chat Command | Hugging Face Repo |\n| --- | --- | --- |\n| 7B | `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5` | [lmsys/vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-7b-v1.5) |\n| 7B-16k | `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5-16k` | [lmsys/vicuna-7b-v1.5-16k](https://huggingface.co/lmsys/vicuna-7b-v1.5-16k) |\n| 13B | `python3 -m fastchat.serve.cli --model-","github_created_at":"2023-03-19T00:18:02+00:00","created_at":"2026-07-11T10:37:14.412989+00:00","updated_at":"2026-07-11T12:39:47.060125+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"chatbots","name":"chatbots"},{"slug":"distributed-serving","name":"distributed serving"},{"slug":"evaluation-system","name":"evaluation system"},{"slug":"large-language-models","name":"large-language-models"}],"trust":{"provenance":{"is_fork":false,"github_id":615882673,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T10:37:15.081Z","maintenance":{"label":"Steady","score":60,"methodology":"github_public_v1","releases_90d":0,"days_since_push":71,"last_release_at":"2024-02-11T15:40:27Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:37:15.927Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T12:39:11.645Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-07-11T12:39:11.645Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T12:39:11.645Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["- You are looking to train and evaluate state-of-the-art models such as Vicuna or MT-Bench.","- You need a distributed multi-model serving system with OpenAI-compatible RESTful APIs for integration into existing workflows.","- Your team requires access to the LLM Elo leaderboard, which is compiled from over 1.5M human votes in side-by-side comparisons of over 70 different models."],"when_not_to_use":["- You require a proprietary or closed-source framework; FastChat is open-source under Apache-2.0 license and its use might be unsuitable for environments requiring proprietary solutions.","- Your chatbot evaluation needs do not align with the types of data used in FastChat's datasets (e.g., human votes, MT-Bench evaluations).","- You prefer a more user-friendly setup without the need to clone a repository and manually install dependencies; FastChat requires installation from source with additional steps for Rust and CMake on","+ Mac."],"source":"enrich:decision_facts","observed_at":"2026-07-11T12:39:46.741Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"FastChat is a comprehensive open platform for managing large language models (LLMs) that includes capabilities for training, serving, evaluating, and comparing chatbot models via web UIs and RESTful APIs. It powers ChatB"}]}}