GraphCanon
Home/Model Training/lmdeploy
lmdeploy logo

lmdeploy

Enrichment pending
InternLM/lmdeploy

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

GitHub Website

GraphCanon updated today · GitHub synced today

8.0k
Stars
703
Forks
597
Open issues
55
Watchers
1d
Last push
Python Apache-2.0Created Jun 15, 2023

Trust & integrity

Full report
Maintenance
Very active (0d since push)
As of today · Source: github_public_v1
Provenance
Not a fork · Organization account
As of today · Source: github_public_v1
Security (OSV)
No lockfile
As of today · Source: none

Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.

Overview

LMDeploy is a toolkit for compressing, deploying, and serving LLMs.

Capability facts

Languages
python

Source: github.language+pyproject.toml · Jul 11, 2026

Categories

Model TrainingLLM FrameworksInference & Serving

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Jul 11, 2026)

It is recommended installing lmdeploy using pip in a conda environment (python 3.10 - 3.13):
Source link

Tags

codellamallamadeepspeedfastertransformercuda-kernelsinternlmllama3llama2

README

Installation

It is recommended installing lmdeploy using pip in a conda environment (python 3.10 - 3.13):

conda create -n lmdeploy python=3.12 -y
conda activate lmdeploy
pip install lmdeploy

Starting from v0.13.0, the default prebuilt wheels published on PyPI are built against CUDA 12.8, so pip install lmdeploy is sufficient for typical setups including GeForce RTX 50 series.

License

This project is released under the Apache 2.0 license.