DeepSpeed-MII
Enables low-latency and high-throughput inference for deep learning models
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
31views this month
Decision brief
DeepSpeed-MII is a tool for low-latency and high-throughput inference in deep learning models, supporting PyTorch and offering both non-persistent and persistent deployment options.
Good fit when
- Use DeepSpeed-MII when you need to deploy deep learning models with low latency and high throughput, especially for production applications that require persistent deployment.
- Choose DeepSpeed-MII if you are working with PyTorch models and want to avoid lengthy compile times, as it comes with pre-compiled Python wheels for easy setup.
Avoid when
- Avoid DeepSpeed-MII if your environment does not meet the requirements of NVIDIA GPUs with compute capabilities 8.0+ (Ampere+), CUDA 11.6+, and Ubuntu 20+.
- Do not use DeepSpeed-MII if you require support for frameworks other than PyTorch, as it is specifically designed for PyTorch models.
- Pricing:
- freemium - DeepSpeed-MII is open-source under the Apache-2.0 license, making it free to use, but users may incur costs based on the infrastructure they use to run the models.
- Requirements:
- Min 8 GB RAM; Requires NVIDIA GPUs with compute capabilities 8.0+ (Ampere+), CUDA 11.6+, and Ubuntu 20+.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Dormant (444d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Organization account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
pip install DeepSpeed-MIISimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
DeepSpeed-MII is a tool that allows for the creation of non-persistent and persistent deployments for supported deep learning models, focusing on inference with low latency and high throughput. It supports PyTorch and is distributed with pre-compiled Python wheels for ease of use.
Capability facts
- Languages
- python
Source: github.language+pyproject.toml · Sep 18, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Sep 18, 2026)
ile times that many projects require in this space we distribute a pre-compiled python wheel covering the majority of our custom kernels through a new library called [Source link
Tags
README
Getting Started with MII DeepSpeed MII allows users to create non persistent and persistent deployments for supported models in just a few lines of code. Installation Non Persistent Pipeline Persistent Deployment Installation The fasest way to get started is with our PyPI release...
For agents
This page has a .md twin and JSON over the API.