{"data":{"slug":"lm-sys-fastchat","name":"FastChat","tagline":"An open platform for training, serving, and evaluating large language models","github_url":"https://github.com/lm-sys/FastChat","owner":"lm-sys","repo":"FastChat","owner_avatar_url":"https://avatars.githubusercontent.com/u/126381704?v=4","primary_language":"Python","stars":39536,"forks":4774,"topics":[],"archived":false,"github_pushed_at":"2026-05-01T00:25:53+00:00","maintenance_label":"Slowing","stars_delta_30d":19,"url":"https://www.graphcanon.com/tools/lm-sys-fastchat","markdown_url":"https://www.graphcanon.com/tools/lm-sys-fastchat.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/lm-sys-fastchat","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=lm-sys-fastchat","description":"An open platform for training, serving, and evaluating large language models. Release repo for Vicuna and Chatbot Arena.","homepage_url":null,"license":"Apache-2.0","open_issues":1043,"watchers":360,"ai_summary":"FastChat is an open-source platform for training, serving, and evaluating large language models, including Vicuna and supporting Chatbot Arena, a platform for LLM evaluations with human feedback.","readme_excerpt":"# FastChat\n| [**Demo**](https://lmarena.ai/) | [**Discord**](https://discord.gg/6GXcFg3TH8) | [**X**](https://x.com/lmsysorg) |\n\nFastChat is an open platform for training, serving, and evaluating large language model based chatbots.\n- FastChat powers Chatbot Arena ([lmarena.ai](https://lmarena.ai)), serving over 10 million chat requests for 70+ LLMs.\n- Chatbot Arena has collected over 1.5M human votes from side-by-side LLM battles to compile an online [LLM Elo leaderboard](https://lmarena.ai/?leaderboard).\n\nFastChat's core features include:\n- The training and evaluation code for state-of-the-art models (e.g., Vicuna, MT-Bench).\n- A distributed multi-model serving system with web UI and OpenAI-compatible RESTful APIs.\n\n## News\n- [2024/03] 🔥 We released Chatbot Arena technical [report](https://arxiv.org/abs/2403.04132).\n- [2023/09] We released **LMSYS-Chat-1M**, a large-scale real-world LLM conversation dataset. Read the [report](https://arxiv.org/abs/2309.11998).\n- [2023/08] We released **Vicuna v1.5** based on Llama 2 with 4K and 16K context lengths. Download [weights](#vicuna-weights).\n- [2023/07] We released **Chatbot Arena Conversations**, a dataset containing 33k conversations with human preferences. Download it [here](https://huggingface.co/datasets/lmsys/chatbot_arena_conversations).\n\n
\nMore\n\n- [2023/08] We released **LongChat v1.5** based on Llama 2 with 32K context lengths. Download [weights](#longchat).\n- [2023/06] We introduced **MT-bench**, a challenging multi-turn question set for evaluating chatbots. Check out the blog [post](https://lmsys.org/blog/2023-06-22-leaderboard/).\n- [2023/06] We introduced **LongChat**, our long-context chatbots and evaluation tools. Check out the blog [post](https://lmsys.org/blog/2023-06-29-longchat/).\n- [2023/05] We introduced **Chatbot Arena** for battles among LLMs. Check out the blog [post](https://lmsys.org/blog/2023-05-03-arena).\n- [2023/03] We released **Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality**. Check out the blog [post](https://vicuna.lmsys.org).\n\n
\n\n\n\n## Contents\n- [Install](#install)\n- [Model Weights](#model-weights)\n- [Inference with Command Line Interface](#inference-with-command-line-interface)\n- [Serving with Web GUI](#serving-with-web-gui)\n- [API](#api)\n- [Evaluation](#evaluation)\n- [Fine-tuning](#fine-tuning)\n- [Citation](#citation)\n\n## Install\n\n### Method 1: With pip\n\n```bash\npip3 install \"fschat[model_worker,webui]\"\n```\n\n### Method 2: From source\n\n1. Clone this repository and navigate to the FastChat folder\n```bash\ngit clone https://github.com/lm-sys/FastChat.git\ncd FastChat\n```\n\nIf you are running on Mac:\n```bash\nbrew install rust cmake\n```\n\n2. Install Package\n```bash\npip3 install --upgrade pip # enable PEP 660 support\npip3 install -e \".[model_worker,webui]\"\n```\n\n## Model Weights\n### Vicuna Weights\n[Vicuna](https://lmsys.org/blog/2023-03-30-vicuna/) is based on Llama 2 and should be used under Llama's [model license](https://github.com/facebookresearch/llama/blob/main/LICENSE).\n\nYou can use the commands below to start chatting. It will automatically download the weights from Hugging Face repos.\nDownloaded weights are stored in a `.cache` folder in the user's home folder (e.g., `~/.cache/huggingface/hub/`).\n\nSee more command options and how to handle out-of-memory in the \"Inference with Command Line Interface\" section below.\n\n**NOTE: `transformers>=4.31` is required for 16K versions.**\n\n| Size | Chat Command | Hugging Face Repo |\n| --- | --- | --- |\n| 7B | `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5` | [lmsys/vicuna-7b-v1.5](https://huggingface.co/lmsys/vicuna-7b-v1.5) |\n| 7B-16k | `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5-16k` | [lmsys/vicuna-7b-v1.5-16k](https://huggingface.co/lmsys/vicuna-7b-v1.5-16k) |\n| 13B | `python3 -m fastchat.serve.cli --model-","github_created_at":"2023-03-19T00:18:02+00:00","created_at":"2026-07-11T10:37:14.412989+00:00","updated_at":"2026-09-20T04:29:18.103176+00:00","categories":[{"slug":"evaluation-observability","name":"Evaluation & Observability","url":"https://www.graphcanon.com/categories/evaluation-observability","markdown_url":"https://www.graphcanon.com/categories/evaluation-observability.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/evaluation-observability"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"chatbot-arena","name":"chatbot arena"},{"slug":"llm-evaluation","name":"llm-evaluation"},{"slug":"multi-model-serving","name":"multi-model serving"},{"slug":"openai-compatible-apis","name":"openai-compatible apis"},{"slug":"vicuna","name":"vicuna"}],"trust":{"provenance":{"is_fork":false,"github_id":615882673,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:14:29.084Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":140,"last_release_at":"2024-02-11T15:40:27Z","stars_delta_30d":19,"open_issues_delta_30d":5},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:14:31.206Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:14:28.035Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:14:28.035Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:14:28.035Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need a platform that supports both training and evaluation of large language models, such as Vicuna, and you want to leverage human feedback for model evaluation.","If you require a distributed multi-model serving system with a web UI and OpenAI-compatible RESTful APIs for deploying and serving multiple LLMs.","When you are interested in using or contributing to a large-scale real-world LLM conversation dataset like LMSYS-Chat-1M.","If you are working on long-context chatbots and need evaluation tools, FastChat supports LongChat with 32K context lengths."],"when_not_to_use":["If you are looking for a platform that focuses solely on model training without serving or evaluation capabilities, FastChat offers a comprehensive suite that might be more than you need.","If your project does not require human feedback for model evaluation, FastChat's integration with Chatbot Arena might be an unnecessary feature.","If you are working with proprietary or closed-source models, FastChat's open-source nature and reliance on open-source models like Vicuna might not align with your project requirements."],"source":"enrich:decision_facts","observed_at":"2026-09-18T09:47:24.868Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"FastChat is an open-source platform for training, serving, and evaluating large language models, including Vicuna, and supports Chatbot Arena, a platform for LLM evaluations with human feedback."}]}}