Home/Inference & Serving/vllm-mlx

vllm-mlx

waybarrios/vllm-mlx

Server for LLMs and vision-language models compatible with Apple Silicon

GitHub Website

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

67views this month

1.6k stars222 forksLast push Sep 19, 2026 Python Apache-2.0

Decision brief

vllm-mlx is an open-source inference server that runs large language models and vision-language models on Apple Silicon devices with continuous batching and multimodal support using native MLX backend.

Good fit when

If you need to run LLMs or vision-language models like Llama, Qwen-VL, and LLaVA efficiently on Apple Silicon devices.
When developing applications that require multimodal AI functionalities including text-to-speech, speech-to-text, and image-understanding, specifically on macOS systems.

Avoid when

If your target environment is not an Apple device equipped with the required hardware to run models via MLX backend.
When seeking a solution that offers high-speed token throughput beyond 400 tok/s as vllm-mlx may not be adequate for such performance needs.

Observed Jul 16, 2026 · Source: enrich:decision_facts

vllm-mlx alternatives Trust report

Verify the decision

Maintenance and security

Full trust report

Maintenance: Very active (0d since push); As of Sep 20, 2026
Provenance: Not a fork · Personal account; As of Sep 20, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

pip install vllm-mlx

PyPI

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

181k★

omlx

LLM inference server with continuous batching and SSD caching for Apple Silicon

22k★

inference

Unified production-ready inference API for various LLMs and models

9.6k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

vllm-mlx is an open-source inference server that supports running large language models and vision-language models on Apple Silicon devices using native MLX backend.

Capability facts

CLI: CLI entrypoint
Source: pyproject.toml:[project.scripts] · Sep 20, 2026
Languages: python
Source: github.language+pyproject.toml · Sep 20, 2026

Compatibility

Sourced claims from the README excerpt - not unsourced marketing copy.

Python runtimePython

Source: README excerpt (regex_v1, Sep 20, 2026)

```python from openai import OpenAI

Source link

README

Quick start (30 seconds) OpenAI SDK: Anthropic SDK / Claude Code: Installation Using uv (recommended): ```bash uv tool install vllm mlx CLI, system wide License Apache 2.0. See LICENSE.

For agents

This page has a .md twin and JSON over the API.

/tools/waybarrios-vllm-mlx.md GET /api/graphcanon/tools/waybarrios-vllm-mlx GET /api/graphcanon/graph?tool=waybarrios-vllm-mlx