GraphCanon updated today · GitHub synced today
4.3k
Stars
883
Forks
443
Open issues
11
Watchers
11mo
Last push
Java Apache-2.0Created Oct 3, 2019
Trust & integrity
Full report- Maintenance
- Archived (339d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Organization account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
Serve, optimize and scale PyTorch models in production
Capability facts
- Languages
- java
Source: github.language · Jul 11, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Python runtimePython
Source: README excerpt (regex_v1, Jul 11, 2026)
python ./ts_scripts/install_dependencies.pySource link
Tags
README
🚀 Quick start with TorchServe
---
# Install dependencies
python ./ts_scripts/install_dependencies.py
---
### 🚀 Quick start with TorchServe (conda)
```bash
---
### 🤖 Quick Start LLM Deployment
#### VLLM Engine
```bash
---
# Make sure to install torchserve with pip or conda as described above and login with `huggingface-cli login`
python -m ts.llm_launcher --model_id meta-llama/Llama-3.2-3B-Instruct --disable_token_auth
---
# Make sure to install torchserve with python venv as described above and login with `huggingface-cli login`
---
# pip install -U --use-deprecated=legacy-resolver -r requirements/trt_llm.txt
python -m ts.llm_launcher --model_id meta-llama/Meta-Llama-3.1-8B-Instruct --engine trt_llm --disable_token_auth
---
### 🚢 Quick Start LLM Deployment with Docker
```bash
#export token=<HUGGINGFACE_HUB_TOKEN>
docker build --pull . -f docker/Dockerfile.vllm -t ts/vllm
docker run --rm -ti --shm-size 10g --gpus all -e HUGGING_FACE_HUB_TOKEN=$token -p 8080:8080 -v data:/data ts/vllm --model_id meta-llama/Meta-Llama-3-8B-Instruct --disable_token_auth