Home/Inference & Serving/dynamo

dynamo

ai-dynamo/dynamo

A Datacenter Scale Distributed Inference Serving Framework

GitHub Website

GraphCanon updated 1d · GitHub synced 1d

7.8k stars1.5k forksLast push 1d Rust Other

Decision brief

Dynamo is a Rust-built framework for large-scale distributed inference serving, aimed at efficient management and deployment of machine learning models in a datacenter environment.

Good fit when

When you are working with high-throughput, low-latency requirements using Kubernetes.
For projects that leverage diffusion models or require disaggregated-serving capabilities.

Avoid when

If your project is not compatible with Rust and you face limitations in leveraging the dynamo's full potential without a strong Rust support team on hand.
In scenarios where fine-grained model management is less important than ease of use or when a more universally-supported language (like Python) is required.

Observed Jul 12, 2026 · Source: enrich:decision_facts

dynamo alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Very active (0d since push); As of 1d
Provenance: Not a fork · Organization account; As of 1d
Security (OSV): No lockfile; As of 1mo

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

cargo add dynamo

crates.io

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

serving

A flexible, high-performance serving system for machine learning models

6.4k★

flashinfer

FlashInfer is a kernel library for serving large language models

6.2k★

kserve

Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes

5.8k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Dynamo is a framework designed for large-scale distributed inference serving, primarily implemented in Rust and focusing on efficient handling of machine learning models.

Capability facts

Languages: rust, python
Source: github.language+pyproject.toml · Aug 24, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Aug 24, 2026)

ld-essential libhwloc-dev libudev-dev pkg-config libclang-dev protobuf-compiler python3-dev cmake

Source link

Works with CursorCursor

Source: README excerpt (regex_v1, Aug 24, 2026)

> Cursor), clone the repo and ask it to deploy, troubleshoot, benchmark, or optimize a D

Source link

README

Quick Start

This repo ships agent skills: if you work with an AI coding agent (Claude Code, Codex, Cursor), clone the repo and ask it to deploy, troubleshoot, benchmark, or optimize a Dynamo deployment. The skills activate automatically; no setup required.

Option B: Install from PyPI

Install uv (curl -LsSf https://astral.sh/uv/install.sh | sh), then:

uv pip install --prerelease=allow "ai-dynamo[sglang]"   # or [vllm]

Note: TensorRT-LLM requires pip with --extra-index-url https://pypi.nvidia.com. See the install guide for TRT-LLM-specific instructions.

Then start the frontend and a worker as shown above. See the full installation guide for system dependencies and backend-specific notes.

Install system deps (Ubuntu 24.04)

sudo apt install -y build-essential libhwloc-dev libudev-dev pkg-config libclang-dev protobuf-compiler python3-dev cmake

Install Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh && source $HOME/.cargo/env

For agents

This page has a .md twin and JSON over the API.

/tools/ai-dynamo-dynamo.md GET /api/graphcanon/tools/ai-dynamo-dynamo GET /api/graphcanon/graph?tool=ai-dynamo-dynamo