GraphCanon updated today · GitHub synced today · 188 views this month
Trust & integrity
Full report- Maintenance
- Steady (71d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Organization account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
FastChat provides capabilities to train, serve, evaluate, and compare large language model-based chatbots. It includes components for multi-model serving systems with web and RESTful API interfaces.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Jul 11, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 11, 2026)
| 7B | `python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5` | [lmsys/vicuna-7b-v1Source link
Source: README excerpt (regex_v1, Jul 11, 2026)
2023/03] We released **Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality**. Check out the blog [post](https://vicuna.lmsys.org).Source link
Tags
README
FastChat
FastChat is an open platform for training, serving, and evaluating large language model based chatbots.
- FastChat powers Chatbot Arena (lmarena.ai), serving over 10 million chat requests for 70+ LLMs.
- Chatbot Arena has collected over 1.5M human votes from side-by-side LLM battles to compile an online LLM Elo leaderboard.
FastChat's core features include:
- The training and evaluation code for state-of-the-art models (e.g., Vicuna, MT-Bench).
- A distributed multi-model serving system with web UI and OpenAI-compatible RESTful APIs.
News
- [2024/03] 🔥 We released Chatbot Arena technical report.
- [2023/09] We released LMSYS-Chat-1M, a large-scale real-world LLM conversation dataset. Read the report.
- [2023/08] We released Vicuna v1.5 based on Llama 2 with 4K and 16K context lengths. Download weights.
- [2023/07] We released Chatbot Arena Conversations, a dataset containing 33k conversations with human preferences. Download it here.
More
- [2023/08] We released LongChat v1.5 based on Llama 2 with 32K context lengths. Download weights.
- [2023/06] We introduced MT-bench, a challenging multi-turn question set for evaluating chatbots. Check out the blog post.
- [2023/06] We introduced LongChat, our long-context chatbots and evaluation tools. Check out the blog post.
- [2023/05] We introduced Chatbot Arena for battles among LLMs. Check out the blog post.
- [2023/03] We released Vicuna: An Open-Source Chatbot Impressing GPT-4 with 90% ChatGPT Quality. Check out the blog post.
Contents
- Install
- Model Weights
- Inference with Command Line Interface
- Serving with Web GUI
- API
- Evaluation
- Fine-tuning
- Citation
Install
Method 1: With pip
pip3 install "fschat[model_worker,webui]"
Method 2: From source
- Clone this repository and navigate to the FastChat folder
git clone https://github.com/lm-sys/FastChat.git
cd FastChat
If you are running on Mac:
brew install rust cmake
- Install Package
pip3 install --upgrade pip # enable PEP 660 support
pip3 install -e ".[model_worker,webui]"
Model Weights
Vicuna Weights
Vicuna is based on Llama 2 and should be used under Llama's model license.
You can use the commands below to start chatting. It will automatically download the weights from Hugging Face repos.
Downloaded weights are stored in a .cache folder in the user's home folder (e.g., ~/.cache/huggingface/hub/<model_name>).
See more command options and how to handle out-of-memory in the "Inference with Command Line Interface" section below.
NOTE: transformers>=4.31 is required for 16K versions.
| Size | Chat Command | Hugging Face Repo |
|---|---|---|
| 7B | python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5 | lmsys/vicuna-7b-v1.5 |
| 7B-16k | python3 -m fastchat.serve.cli --model-path lmsys/vicuna-7b-v1.5-16k | lmsys/vicuna-7b-v1.5-16k |
| 13B | `python3 -m fastchat.serve.cli --model- |