{"data":{"slug":"pytorch-serve","name":"serve","tagline":"Serve, optimize and scale PyTorch models in production","github_url":"https://github.com/pytorch/serve","owner":"pytorch","repo":"serve","owner_avatar_url":"https://avatars.githubusercontent.com/u/21003710?v=4","primary_language":"Java","stars":4350,"forks":883,"topics":["cpu","deep-learning","docker","gpu","kubernetes","machine-learning","metrics","mlops","optimization","pytorch","serving"],"archived":true,"github_pushed_at":"2025-08-06T19:17:08+00:00","maintenance_label":"Archived","url":"https://www.graphcanon.com/tools/pytorch-serve","markdown_url":"https://www.graphcanon.com/tools/pytorch-serve.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/pytorch-serve","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=pytorch-serve","description":"Serve, optimize and scale PyTorch models in production","homepage_url":"https://pytorch.org/serve/","license":"Apache-2.0","open_issues":443,"watchers":11,"ai_summary":null,"readme_excerpt":"### 🚀 Quick start with TorchServe\n\n```bash\n\n---\n\n# Install dependencies\npython ./ts_scripts/install_dependencies.py\n\n---\n\n### 🚀 Quick start with TorchServe (conda)\n\n```bash\n\n---\n\n### 🤖 Quick Start LLM Deployment\n\n#### VLLM Engine\n```bash\n\n---\n\n# Make sure to install torchserve with pip or conda as described above and login with `huggingface-cli login`\npython -m ts.llm_launcher --model_id meta-llama/Llama-3.2-3B-Instruct --disable_token_auth\n\n---\n\n# Make sure to install torchserve with python venv as described above and login with `huggingface-cli login`\n\n---\n\n# pip install -U --use-deprecated=legacy-resolver -r requirements/trt_llm.txt\npython -m ts.llm_launcher --model_id meta-llama/Meta-Llama-3.1-8B-Instruct --engine trt_llm --disable_token_auth\n\n---\n\n### 🚢 Quick Start LLM Deployment with Docker\n\n```bash\n#export token=<HUGGINGFACE_HUB_TOKEN>\ndocker build --pull . -f docker/Dockerfile.vllm -t ts/vllm\n\ndocker run --rm -ti --shm-size 10g --gpus all -e HUGGING_FACE_HUB_TOKEN=$token -p 8080:8080 -v data:/data ts/vllm --model_id meta-llama/Meta-Llama-3-8B-Instruct --disable_token_auth","github_created_at":"2019-10-03T03:17:43+00:00","created_at":"2026-07-11T23:12:45.393703+00:00","updated_at":"2026-07-11T23:13:12.003322+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"cpu","name":"cpu"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"docker","name":"docker"},{"slug":"gpu","name":"gpu"},{"slug":"kubernetes","name":"kubernetes"},{"slug":"machine-learning","name":"machine-learning"},{"slug":"metrics","name":"metrics"},{"slug":"mlops","name":"mlops"}],"trust":{"provenance":{"is_fork":false,"github_id":212488700,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:12:46.841Z","maintenance":{"label":"Archived","score":8,"methodology":"github_public_v1","releases_90d":0,"days_since_push":339,"last_release_at":"2024-09-30T22:46:39Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:12:47.373Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:12:46.514Z"},"languages":{"value":["java"],"source":"github.language","observed_at":"2026-07-11T23:12:46.514Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-07-11T23:12:46.514Z"}}}}