vllm-mlx
Enrichment pendingOpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX bac
GraphCanon updated today · GitHub synced today
Trust & integrity
Full report- Maintenance
- Active (12d since push)
- As of today · Source: github_public_v1
- Provenance
- Not a fork · Personal account
- As of today · Source: github_public_v1
- Security (OSV)
- No lockfile
- As of today · Source: none
Public GitHub metadata and optional OSV dependency scans. Signals, not a guarantee. Trust methodology.
Overview
OpenAI and Anthropic compatible server for Apple Silicon. Run LLMs and vision-language models (Llama, Qwen-VL, LLaVA) with continuous batching, MCP tool calling, and multimodal support. Native MLX backend, 400+ tok/s. Works with Claude Code.
Capability facts
- CLI
- CLI entrypoint
Source: pyproject.toml:[project.scripts] · Jul 11, 2026
- Languages
- python
Source: github.language+pyproject.toml · Jul 11, 2026
Categories
Compatibility
Sourced claims from the README excerpt - not unsourced marketing copy.
Source: README excerpt (regex_v1, Jul 11, 2026)
```python from openai import OpenAISource link
Tags
README
Quick start (30 seconds)
pip install vllm-mlx
vllm-mlx serve mlx-community/Llama-3.2-3B-Instruct-4bit --port 8000 --continuous-batching
OpenAI SDK:
from openai import OpenAI
client = OpenAI(base_url="http://localhost:8000/v1", api_key="not-needed")
r = client.chat.completions.create(model="default", messages=[{"role": "user", "content": "Hi!"}])
print(r.choices[0].message.content)
Anthropic SDK / Claude Code:
export ANTHROPIC_BASE_URL=http://localhost:8000
export ANTHROPIC_API_KEY=not-needed
claude
Installation
Using uv (recommended):
uv tool install vllm-mlx # CLI, system-wide
---
## License
Apache 2.0. See [LICENSE](LICENSE).