{"data":{"slug":"internlm-lmdeploy","name":"lmdeploy","tagline":"Toolkit for compressing, deploying, and serving LLMs","github_url":"https://github.com/InternLM/lmdeploy","owner":"InternLM","repo":"lmdeploy","owner_avatar_url":"https://avatars.githubusercontent.com/u/135356492?v=4","primary_language":"Python","stars":8044,"forks":737,"topics":["codellama","cuda-kernels","deepspeed","fastertransformer","internlm","llama","llama2","llama3","llm","llm-inference","turbomind"],"archived":false,"github_pushed_at":"2026-09-05T04:35:03+00:00","maintenance_label":"Very active","stars_delta_30d":49,"url":"https://www.graphcanon.com/tools/internlm-lmdeploy","markdown_url":"https://www.graphcanon.com/tools/internlm-lmdeploy.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/internlm-lmdeploy","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=internlm-lmdeploy","description":"LMDeploy is a toolkit for compressing, deploying, and serving LLMs.","homepage_url":"https://lmdeploy.readthedocs.io/en/latest","license":"Apache-2.0","open_issues":608,"watchers":57,"ai_summary":"LMDeploy is designed to facilitate the process of compressing deep learning models, particularly large language models (LLMs), as well as their seamless deployment and serving through optimized frameworks.","readme_excerpt":"## Installation\n\nIt is recommended to install lmdeploy using pip in a conda environment (python 3.10 - 3.13):\n\n```shell\nconda create -n lmdeploy python=3.12 -y\nconda activate lmdeploy\npip install lmdeploy\n```\n\nStarting from **v0.13.0**, the default prebuilt wheels published on **PyPI** are built against **CUDA 12.8**, so `pip install lmdeploy` is sufficient for typical setups including GeForce RTX 50 series.\n\n---\n\n# License\n\nThis project is released under the [Apache 2.0 license](LICENSE).","github_created_at":"2023-06-15T12:38:06+00:00","created_at":"2026-07-11T10:37:53.448707+00:00","updated_at":"2026-09-20T04:23:30.826325+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"codellama","name":"codellama"},{"slug":"cuda-kernels","name":"cuda-kernels"},{"slug":"deepspeed","name":"deepspeed"},{"slug":"fastertransformer","name":"fastertransformer"},{"slug":"internlm","name":"internlm"},{"slug":"llama","name":"llama"},{"slug":"llama2","name":"llama2"},{"slug":"llama3","name":"llama3"}],"trust":{"provenance":{"is_fork":false,"github_id":654122609,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-07T00:01:32.372Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":5,"days_since_push":1,"last_release_at":"2026-09-01T07:13:39Z","stars_delta_30d":49,"open_issues_delta_30d":1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:37:54.793Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-07T00:01:32.812Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-07T00:01:32.812Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-07T00:01:32.812Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Installation is optimized through pip in a Conda environment using Python versions between 3.10 and 3.13."],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"requires_docker":false},"when_to_use":["When your team operates within a CUDA environment, such as using an Nvidia GeForce RTX 50 series GPU, because the default prebuilt wheels are optimized for CUDA 12.8.","If you aim to streamline deployment and serving processes for large language models with out-of-the-box support for frameworks like FasterTransformer.","For specific model families including Codellama, LLaMA, or TurboMind where LMDeploy offers tailored optimizations."],"when_not_to_use":["When your infrastructure relies on software environments or GPUs not aligned with CUDA 12.8, as LMDeploy's default prebuilt wheels might require adjustments to operate optimally.","If you are working exclusively in non-Nvidia GPU ecosystems where LMDeploy's CUDA focus does not align with the hardware optimizations available."],"source":"enrich:decision_facts","observed_at":"2026-07-14T20:19:44.025Z"},"constraint_facets":{"min_ram_gb":null,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"Installation is optimized through pip in a Conda environment using Python versions between 3.10 and 3.13."},{"label":"Adopt for","value":"LMDeploy is focused on compressing and efficiently serving LLMs, making it suitable for teams already invested in CUDA environments like Nvidia's GeForce RTX 50 series."},{"label":"License detail","value":"Licensed under Apache-2.0, enabling flexible use and modification for both commercial and open-source projects, provided that users comply with its terms."}]}}