{"data":{"slug":"raullenchai-rapid-mlx","name":"Rapid-MLX","tagline":"Fast local AI engine for Apple Silicon","github_url":"https://github.com/raullenchai/Rapid-MLX","owner":"raullenchai","repo":"Rapid-MLX","owner_avatar_url":"https://avatars.githubusercontent.com/u/989846?v=4","primary_language":"Python","stars":3391,"forks":388,"topics":["apple-silicon","claude-code","cursor","deepseek","fastapi","hacktoberfest","inference","llm","local-llm","m1","m2","m3","macos","mlx","ollama-alternative","openai-api","python","qwen","tool-calling"],"archived":false,"github_pushed_at":"2026-08-01T23:14:51+00:00","maintenance_label":"Very active","url":"https://www.graphcanon.com/tools/raullenchai-rapid-mlx","markdown_url":"https://www.graphcanon.com/tools/raullenchai-rapid-mlx.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/raullenchai-rapid-mlx","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=raullenchai-rapid-mlx","description":"The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.","homepage_url":"https://pypi.org/project/rapid-mlx","license":"Apache-2.0","open_issues":48,"watchers":64,"ai_summary":"A high-performance local AI engine tailored for Apple silicon devices. Offers OpenAI-compatible APIs and supports multiple models optimized for different RAM sizes.","readme_excerpt":"## Quick Start (60 seconds)\n\n**1. Install** — pick one path (run only one of these):\n\nOne-liner — detects your RAM, picks a starter model (recommended):\n\n```bash\ncurl -fsSL https://rapidmlx.com/install.sh | bash\n```\n\nor Homebrew — prebuilt bottle straight from homebrew-core:\n\n```bash\nbrew install rapid-mlx\n```\n\nBoth land the same `rapid-mlx` CLI. The curl installer additionally installs Python 3.10+ if missing, creates an isolated venv at `~/.rapid-mlx/`, symlinks the `rapid-mlx` CLI into `~/.local/bin/`, and prints a serve command sized to your Mac (8–23 GB → `qwen3.5-4b-4bit`; 24–47 GB → `gpt-oss-20b-mxfp4-q8`; 48–95 GB → `qwen3.6-35b-8bit`; 96 GB+ → `gpt-oss-120b-mxfp4-q8`).\n\n> **Install security.** `install.sh` is served over HTTPS (HSTS-preload) from `rapidmlx.com` and is a byte-identical mirror of [`install.sh`](install.sh) at the release commit — read it before running if you like. If you want a cryptographically verified installer rather than trusting the website pipe, don't `curl | bash` the URL above: instead download the release's `install.sh` asset, verify it against the cosign-signed `SHA256SUMS.txt` shipped alongside it, and run that verified copy — full recipe in [SECURITY.md](SECURITY.md). PyPI artifacts additionally carry Sigstore attestations (PEP 740). Two more low-trust paths:\n> - **Pin to a commit hash** — `curl -fsSL https://raw.githubusercontent.com/raullenchai/Rapid-MLX/<commit>/install.sh -o install.sh && shasum -a 256 install.sh && bash install.sh`\n> - **Skip the shell script entirely** — use Homebrew, `uv`, or `pip` below.\n\nSee [Alternative install methods](#alternative-install-methods) for the non-curl paths.\n\n**2. Chat with a model right now:**\n\n```bash\nrapid-mlx chat\n```\n\nDefaults to `qwen3.5-4b-4bit`. First run downloads the weights (~2.5 GB) with a progress bar and drops you into a REPL. Type `/help` for slash commands, `/exit` to quit.\n\n**3. Or serve it for use from other apps:**\n\n```bash\nrapid-mlx serve qwen3.5-4b-4bit\n```\n\nStarts an OpenAI-compatible HTTP server bound to `http://localhost:8000`. Point any OpenAI SDK / client (Cursor, Aider, LangChain, OpenCode, PydanticAI, your own scripts) at **`http://localhost:8000/v1`**; Claude Code / Anthropic SDK uses **`http://localhost:8000`** (the Anthropic messages route lives at `/v1/messages` under the same host).\n\n```bash\ncurl http://localhost:8000/v1/chat/completions \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\"model\":\"default\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hello\"}]}'\n```\n\n```python\nfrom openai import OpenAI\nclient = OpenAI(base_url=\"http://localhost:8000/v1\", api_key=\"not-needed\")\nprint(client.chat.completions.create(\n    model=\"default\",\n    messages=[{\"role\": \"user\", \"content\": \"Say hello\"}],\n).choices[0].message.content)\n```\n\n**4. Or wire up your coding agent — one command:**\n\n```bash\nrapid-mlx launch claude-code\n```\n\nWith a server running (step 3), this patches Claude Code's local config (`~/.config/claude/settings.json`) to route at `http://localhost:8000` — no manual env vars, no editing JSON by hand. You get a fully local Claude Code: `$0` per token, nothing leaves your Mac. Swap in `cursor`, `cline`, or `continue-dev` for the other IDE clients, or run `rapid-mlx launch list` to see what's detected on this machine.\n\n> **Vision / audio / video / diffusion models?** Base install is text-only (~460 MB). Vision, audio (TTS, STT, voice cloning), video generation, embeddings, and DFlash speculative decoding ship as opt-in extras. → [Optional extras](https://rapidmlx.com/docs/extras.html)\n\n> **Not into the terminal?** [**Rapid-MLX Desktop**](https://rapidmlx.com/desktop) bundles the same engine inside a one-click Mac app.\n\n---\n\n---\n\n## Alternative install methods\n\nThe two paths above cover most users — reach for these only if you already manage Python yourself.\n\n<details>\n<summary><strong>Homebrew</strong> — Mac-native, one command, prebuilt bottle from <code>homebrew/core</code></summary>\n\n```bash\nbrew install rapid-mlx\n```\n\nS","github_created_at":"2026-02-25T00:41:44+00:00","created_at":"2026-07-11T23:11:40.298387+00:00","updated_at":"2026-08-02T00:00:40.775346+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"apple-silicon","name":"apple-silicon"},{"slug":"local-llm","name":"local-llm"},{"slug":"openai-replacement","name":"openai-replacement"},{"slug":"tool-calling","name":"tool-calling"}],"trust":{"provenance":{"is_fork":false,"github_id":1166182351,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-02T00:00:40.004Z","maintenance":{"label":"Very active","score":96,"methodology":"github_public_v1","releases_90d":30,"days_since_push":0,"last_release_at":"2026-08-01T06:07:29Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:11:42.083Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-02T00:00:40.442Z"},"has_cli":{"value":true,"source":"pyproject.toml:[project.scripts]","observed_at":"2026-08-02T00:00:40.442Z"},"languages":{"value":["python"],"source":"github.language+pyproject.toml","observed_at":"2026-08-02T00:00:40.442Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-08-02T00:00:40.442Z"}},"decision_facts":{"hosting":null,"pricing":{"model":"freemium","summary":"Rapid-MLX is free to install and use, but some advanced features may require additional configuration or payment."},"requirements":{"min_ram_gb":8,"requires_docker":false},"constraints":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"when_to_use":["Use Rapid-MLX when you need an ultra-fast local inference solution specifically tailored for Apple's M1, M2, or M3 chips, as it is up to 4.2 times faster than Ollama.","Choose this tool if your Mac has between 8 GB and 95+ GB of RAM, as Rapid-MLX automatically selects the optimal model based on available memory."],"when_not_to_use":["Avoid Rapid-MLX if you do not have an Apple Silicon device, as its performance optimizations and support are exclusively for Apple's M1, M2, or M3 processors.","Do not use this tool if your project requires complex vision or audio models out of the box; these extras must be installed separately."],"source":"enrich:decision_facts","observed_at":"2026-07-17T04:11:33.717Z"},"constraint_facets":{"min_ram_gb":8,"pricing_model":"freemium","requires_docker":false},"decision_summary":[{"label":"Pricing","value":"freemium - Rapid-MLX is free to install and use, but some advanced features may require additional configuration or payment."},{"label":"Requirements","value":"Min 8 GB RAM"},{"label":"Adopt for","value":"Rapid-MLX is a high-speed local AI engine for Apple Silicon devices that supports OpenAI-compatible APIs and multiple models optimized based on system RAM size."}]}}