GraphCanon updated 4w · GitHub synced 4w
Decision brief
vllm-cli serves large language models via vLLM with a straightforward CLI interface, ideal for users preferring a command-line environment over graphical tools.
Good fit when
- When you require an efficient and robust way to serve large language models through the command line using vLLM framework
- If your development pipeline already heavily relies on Python scripts and you prefer not to leave your terminal
Avoid when
- If your project demands a graphical user interface or web-based interaction for model serving
- When ease of use with non-Python environments is a priority, as vllm-cli is designed specifically for Python users
Observed Jul 17, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Slowing (180d since push)
- As of 4w
- Provenance
- Not a fork · Personal account
- As of 4w
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install vllm-cliSimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Chen-zexi/vllm-cli is a Python-based tool designed to serve large language models with a command-line interface through the vLLM framework.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Jul 25, 2026
- Languages
- python
Source: github.language+pyproject.toml · Jul 25, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 25, 2026)
uv venv --python 3.12 --seedSource link
Tags
README
🚀 Hardware-Optimized Profiles for GPT-OSS Models
New built-in profiles specifically optimized for serving GPT-OSS models on different GPU architectures:
gpt_oss_ampere- Optimized for NVIDIA A100 GPUsgpt_oss_hopper- Optimized for NVIDIA H100/H200 GPUsgpt_oss_blackwell- Optimized for NVIDIA Blackwell GPUs
Based on official vLLM GPT recipes for maximum performance.
Important: vLLM Installation Notes
⚠️ Binary Compatibility Warning: vLLM contains pre-compiled CUDA kernels that must match your PyTorch version exactly. Installing mismatched versions will cause errors.
vLLM-CLI will not install vLLM or Pytorch by default.
Installation
Option 1: Install vLLM seperately and then install vLLM CLI (Recommended)
---
# Install vLLM -- Skip this step if you have vllm installed in your environment
uv venv --python 3.12 --seed
source .venv/bin/activate
uv pip install vllm --torch-backend=auto
---
# Or specify a backend: uv pip install vllm --torch-backend=cu128
---
# Install vLLM CLI
uv pip install --upgrade vllm-cli
uv run vllm-cli
---
# Install vLLM CLI + vLLM
pip install vllm-cli[vllm]
vllm-cli
Option 3: Build from source (You still need to install vLLM seperately)
git clone https://github.com/Chen-zexi/vllm-cli.git
cd vllm-cli
pip install -e .
Option 4: For Isolated Installation (pipx/system packages)
⚠️ Compatibility Note: pipx creates isolated environments which may have compatibility issues with vLLM's CUDA dependencies. Consider using uv or conda (see above) for better PyTorch/CUDA compatibility.
---
# If you do not want to use virtual environment and want to install vLLM along with vLLM CLI
pipx install "vllm-cli[vllm]"
---
# If you want to install pre-release version
pipx install --pip-args="--pre" "vllm-cli[vllm]"
License
MIT License - see LICENSE file for details.
For agents
This page has a .md twin and JSON over the API.