{"data":{"slug":"raullenchai-rapid-mlx","name":"Rapid-MLX","tagline":"Fast local AI engine for Apple Silicon","github_url":"https://github.com/raullenchai/Rapid-MLX","owner":"raullenchai","repo":"Rapid-MLX","owner_avatar_url":"https://avatars.githubusercontent.com/u/989846?v=4","primary_language":"Python","stars":3774,"forks":416,"topics":["apple-silicon","claude-code","cursor","deepseek","fastapi","hacktoberfest","inference","llm","local-llm","m1","m2","m3","macos","mlx","ollama-alternative","openai-api","python","qwen","tool-calling"],"archived":false,"github_pushed_at":"2026-09-18T08:10:29+00:00","maintenance_label":"Very active","stars_delta_30d":383,"url":"https://www.graphcanon.com/tools/raullenchai-rapid-mlx","markdown_url":"https://www.graphcanon.com/tools/raullenchai-rapid-mlx.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/raullenchai-rapid-mlx","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=raullenchai-rapid-mlx","description":"The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.","homepage_url":"https://pypi.org/project/rapid-mlx","license":"Other","open_issues":54,"watchers":62,"ai_summary":"Rapid-MLX is a high-performance local AI engine optimized for Apple Silicon, offering faster inference times and support for tool calling, prompt caching, and reasoning separation. It serves as a drop-in replacement for OpenAI and supports various tools and frameworks.","readme_excerpt":"## Quick Start (60 seconds)\n\n**1. Chat with a model right now:**\n\n```bash\nrapid-mlx chat\n```\n\nDefaults to `qwen3.5-4b-4bit`. First run downloads the weights (~3 GB) with a progress bar and drops you into a REPL. Type `/help` for slash commands, `/exit` to quit.\n\n**2. Or serve it for use from other apps:**\n\n```bash\nrapid-mlx serve qwen3.5-4b-4bit\n```\n\nStarts an OpenAI-compatible HTTP server bound to `http://localhost:8000`. Point any client that supports a local custom endpoint (Aider, LangChain, OpenCode, PydanticAI, your own scripts) at **`http://localhost:8000/v1`**; Claude Code / Anthropic SDK uses **`http://localhost:8000`** (the Anthropic messages route lives at `/v1/messages` under the same host).\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"default\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello\"}]}'\n```\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"not-needed\")\nprint(client.chat.completions.create(\n    model=\"default\",\n    messages=[{\"role\": \"user\", \"content\": \"Say hello\"}],\n).choices[0].message.content)\n```\n\n**3. Or wire up your coding agent — one command:**\n\n```bash\nrapid-mlx launch claude-code\n```\n\nWith a server running (step 2), this patches Claude Code's local config (`~/.claude/settings.json`) to route at `http://localhost:8000` — no manual env vars, no editing JSON by hand. You get a fully local Claude Code: `$0` per token, nothing leaves your Mac. Swap in `cline` or `continue-dev` for the other IDE clients, or run `rapid-mlx launch list` to see what's detected on this machine.\n\n> **Cursor:** Cursor currently routes BYOK requests through its own servers, so its servers cannot reach a Rapid-MLX endpoint on `localhost`. Rapid-MLX therefore does not generate a Cursor localhost config. If you intentionally expose the server through a public HTTPS tunnel, set `RAPID_MLX_API_KEY=your-secret` for both `rapid-mlx serve ...` and `rapid-mlx launch cursor --server-url https://your-public-host`. This is no longer a fully local connection; never expose an unauthenticated server. Rapid-MLX rejects explicit local/private addresses but cannot verify reachability from Cursor's network, whose DNS view may differ from your Mac.\n\n> **Vision / audio / video / diffusion models?** Base install is text-only (~460 MB). Vision, audio (TTS, STT, voice cloning), video generation, embeddings, and DFlash speculative decoding ship as opt-in extras. → [Optional extras](https://rapidmlx.com/docs/extras.html)\n\n> **Not into the terminal?** [**Rapid-MLX Desktop**](https://rapidmlx.com/desktop) bundles the same engine inside a one-click Mac app.\n\n---\n\n---\n\n## Alternative install methods\n\nThe two paths above cover most users — reach for these only if you already manage Python yourself.\n\n<details>\n<summary><strong>Homebrew</strong> — Mac-native, one command, prebuilt bottle from <code>homebrew/core</code></summary>\n\n```bash\nbrew install rapid-mlx\n```\n\nShips in homebrew-core since 0.10.12 — no tap, no trust prompt. Upgrade with `brew upgrade rapid-mlx`. If you previously installed from the legacy `raullenchai/rapid-mlx` tap, switch once: `brew uninstall rapid-mlx && brew untap raullenchai/rapid-mlx && brew install rapid-mlx`.\n\n</details>\n\n<details>\n<summary><strong>uv</strong> — isolated tool install, auto-manages Python</summary>\n\n```bash\nuv tool install rapid-mlx@latest\n```\n\nDon't have uv yet? `curl -LsSf https://astral.sh/uv/install.sh | sh`. Upgrade with `uv tool upgrade rapid-mlx`.\n\n</details>\n\n<details>\n<summary><strong>pip</strong> — requires Python 3.10+ (macOS ships 3.9)</summary>\n\n```bash\npython3.12 -m pip install rapid-mlx\n```\n\nIf `pip install rapid-mlx` says \"no matching distribution\", your Python is too old. `brew install python@3.12` first. Upgrade with `pip install -U rapid-mlx`.\n\nFor image-input / VLM models (Qwen-VL, true multimodal), install the vision extra: `pip install 'rapid-mlx[vision]'` — see [Optional extr","github_created_at":"2026-02-25T00:41:44+00:00","created_at":"2026-07-11T23:11:40.298387+00:00","updated_at":"2026-09-20T04:29:47.102575+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"llm-frameworks","name":"LLM Frameworks","url":"https://www.graphcanon.com/categories/llm-frameworks","markdown_url":"https://www.graphcanon.com/categories/llm-frameworks.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/llm-frameworks"}],"tags":[{"slug":"apple-silicon","name":"apple-silicon"},{"slug":"claude-code","name":"claude-code"},{"slug":"cursor","name":"cursor"},{"slug":"deepseek","name":"deepseek"},{"slug":"fastapi","name":"fastapi"},{"slug":"hacktoberfest","name":"hacktoberfest"},{"slug":"inference","name":"inference"},{"slug":"llm","name":"llm"}],"trust":{"provenance":{"is_fork":false,"github_id":1166182351,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:18:34.160Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":30,"days_since_push":0,"last_release_at":"2026-09-15T13:33:03Z","stars_delta_30d":383,"open_issues_delta_30d":6},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:18:36.328Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:18:33.072Z"},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-09-18T08:18:33.072Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-09-18T08:18:33.072Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-09-18T08:18:33.072Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["When you need a fast local AI engine specifically optimized for Apple Silicon devices, offering up to 4.2x faster performance compared to Ollama.","If you require a tool that supports prompt caching with a cached time-to-first-token (TTFT) of 0.08 seconds, ensuring quick responses.","When working with applications that support a local custom endpoint, such as Claude Code, Cursor, Aider, and others, and you want a fully local setup without cloud dependency."],"when_not_to_use":["If you are using Cursor, as it routes BYOK requests through its own servers, which cannot reach a Rapid-MLX endpoint on localhost, unless you intentionally expose the server through a public HTTPS隧道并为"],"source":"enrich:decision_facts","observed_at":"2026-09-18T10:18:39.513Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Rapid-MLX is a high-performance local AI engine optimized for Apple Silicon, providing faster inference times and support for tool calling, prompt caching, and reasoning separation. It serves as a drop-in replacement for"}]}}