{"data":{"slug":"chen-zexi-vllm-cli","name":"vllm-cli","tagline":"Command-line interface for serving LLM using vLLM","github_url":"https://github.com/Chen-zexi/vllm-cli","owner":"Chen-zexi","repo":"vllm-cli","owner_avatar_url":"https://avatars.githubusercontent.com/u/128259419?v=4","primary_language":"Python","stars":506,"forks":29,"topics":["llm","llm-inference","llm-tools","vllm"],"archived":false,"github_pushed_at":"2026-01-25T19:37:43+00:00","maintenance_label":"Slowing","stars_delta_30d":0,"url":"https://www.graphcanon.com/tools/chen-zexi-vllm-cli","markdown_url":"https://www.graphcanon.com/tools/chen-zexi-vllm-cli.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/chen-zexi-vllm-cli","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=chen-zexi-vllm-cli","description":"A command-line interface tool for serving LLM using vLLM.","homepage_url":null,"license":"MIT","open_issues":5,"watchers":2,"ai_summary":"Chen-zexi/vllm-cli is a Python-based tool designed to serve large language models with a command-line interface through the vLLM framework.","readme_excerpt":"### 🚀 Hardware-Optimized Profiles for GPT-OSS Models\nNew built-in profiles specifically optimized for serving GPT-OSS models on different GPU architectures:\n- **`gpt_oss_ampere`** - Optimized for NVIDIA A100 GPUs\n- **`gpt_oss_hopper`** - Optimized for NVIDIA H100/H200 GPUs\n- **`gpt_oss_blackwell`** - Optimized for NVIDIA Blackwell GPUs\n\nBased on official [vLLM GPT recipes](https://docs.vllm.ai/projects/recipes/en/latest/OpenAI/GPT-OSS.html) for maximum performance.\n\n---\n\n### Important: vLLM Installation Notes\n⚠️ **Binary Compatibility Warning**: vLLM contains pre-compiled CUDA kernels that must match your PyTorch version exactly. Installing mismatched versions will cause errors.\n\nvLLM-CLI will not install vLLM or Pytorch by default.\n\n---\n\n### Installation\n\n#### Option 1: Install vLLM seperately and then install vLLM CLI (Recommended)\n```bash\n\n---\n\n# Install vLLM -- Skip this step if you have vllm installed in your environment\nuv venv --python 3.12 --seed\nsource .venv/bin/activate\nuv pip install vllm --torch-backend=auto\n\n---\n\n# Or specify a backend: uv pip install vllm --torch-backend=cu128\n\n---\n\n# Install vLLM CLI\nuv pip install --upgrade vllm-cli\nuv run vllm-cli\n\n---\n\n# Install vLLM CLI + vLLM\npip install vllm-cli[vllm]\nvllm-cli\n```\n\n#### Option 3: Build from source (You still need to install vLLM seperately)\n```bash\ngit clone https://github.com/Chen-zexi/vllm-cli.git\ncd vllm-cli\npip install -e .\n```\n\n#### Option 4: For Isolated Installation (pipx/system packages)\n\n⚠️ **Compatibility Note:** pipx creates isolated environments which may have compatibility issues with vLLM's CUDA dependencies. Consider using uv or conda (see above) for better PyTorch/CUDA compatibility.\n\n```bash\n\n---\n\n# If you do not want to use virtual environment and want to install vLLM along with vLLM CLI\npipx install \"vllm-cli[vllm]\"\n\n---\n\n# If you want to install pre-release version\npipx install --pip-args=\"--pre\" \"vllm-cli[vllm]\"\n```\n\n---\n\n## License\n\nMIT License - see [LICENSE](LICENSE) file for details.","github_created_at":"2025-08-14T22:33:18+00:00","created_at":"2026-07-11T11:45:32.033669+00:00","updated_at":"2026-08-25T06:02:15.243899+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"llm","name":"llm"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"llm-tools","name":"llm-tools"},{"slug":"vllm","name":"vllm"}],"trust":{"provenance":{"is_fork":false,"github_id":1038267026,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-25T06:02:14.517Z","maintenance":{"label":"Slowing","score":36,"methodology":"github_public_v1","releases_90d":0,"days_since_push":211,"last_release_at":"2025-08-25T13:46:54Z","stars_delta_30d":0,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T11:45:33.339Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-25T06:02:14.970Z"},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-08-25T06:02:14.970Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-25T06:02:14.970Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-25T06:02:14.970Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you require an efficient and robust way to serve large language models through the command line using vLLM framework","If your development pipeline already heavily relies on Python scripts and you prefer not to leave your terminal"],"when_not_to_use":["If your project demands a graphical user interface or web-based interaction for model serving","When ease of use with non-Python environments is a priority, as vllm-cli is designed specifically for Python users"],"source":"enrich:decision_facts","observed_at":"2026-07-17T13:10:42.690Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"vllm-cli serves large language models via vLLM with a straightforward CLI interface, ideal for users preferring a command-line environment over graphical tools."}]}}