{"data":{"slug":"ericlbuehler-mistral-rs","name":"mistral.rs","tagline":"Fast flexible LLM inference","github_url":"https://github.com/EricLBuehler/mistral.rs","owner":"EricLBuehler","repo":"mistral.rs","owner_avatar_url":"https://avatars.githubusercontent.com/u/65165915?v=4","primary_language":"Rust","stars":7575,"forks":671,"topics":["llm","rust","uqff"],"archived":false,"github_pushed_at":"2026-07-29T20:21:17+00:00","maintenance_label":"Active","url":"https://www.graphcanon.com/tools/ericlbuehler-mistral-rs","markdown_url":"https://www.graphcanon.com/tools/ericlbuehler-mistral-rs.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/ericlbuehler-mistral-rs","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=ericlbuehler-mistral-rs","description":"Fast, flexible LLM inference","homepage_url":null,"license":"MIT","open_issues":380,"watchers":44,"ai_summary":"A Rust implementation for fast and flexible large language model (LLM) inference, supporting prebuilt binaries for multiple platforms including Linux, macOS, and Windows.","readme_excerpt":"### Install\n\n**Linux/macOS:**\n```bash\ncurl --proto '=https' --tlsv1.2 -sSf https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.sh | sh\n```\n\n**Windows (PowerShell):**\n```powershell\nirm https://raw.githubusercontent.com/EricLBuehler/mistral.rs/master/install.ps1 | iex\n```\n\nDownloads a self-contained prebuilt binary for your platform (Metal on Apple Silicon; per-GPU CUDA or CPU on Linux; CPU on Windows), falling back to a source build if none matches. Standard acceleration needs no Rust or CUDA toolkit. Optional cuTile acceleration requires NVIDIA's separately installed `tileiras` tool.\n\n[Manual installation, accelerator details & other platforms](https://ericlbuehler.github.io/mistral.rs/quickstart/)\n\n---\n\n# Recommend settings for your hardware and emit a config file\nmistralrs tune -m Qwen/Qwen3-4B --emit-config config.toml\n\n---\n\n## Docker\n\nPrebuilt CPU and CUDA images are published to GHCR. Pull commands, tags, and Kubernetes notes are in the [Docker guide](https://ericlbuehler.github.io/mistral.rs/guides/deploy/docker/).","github_created_at":"2024-02-26T22:30:06+00:00","created_at":"2026-07-11T10:37:19.176859+00:00","updated_at":"2026-08-07T06:01:21.288886+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"llm","name":"llm"},{"slug":"rust","name":"rust"},{"slug":"uqff","name":"uqff"}],"trust":{"provenance":{"is_fork":false,"github_id":763774926,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-08-07T06:01:20.504Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":30,"days_since_push":8,"last_release_at":"2026-07-07T12:34:24Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T10:37:22.278Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-08-07T06:01:20.959Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-08-07T06:01:20.959Z","managed_saas":false},"languages":{"value":["rust"],"source":"github.language","observed_at":"2026-08-07T06:01:20.959Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-08-07T06:01:20.959Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-08-07T06:01:20.959Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":null,"constraints":null,"when_to_use":["Mistral.rs should be used when seeking Rust-based implementation that supports quick and flexible deployment of large language models, particularly on Linux, macOS, or Windows systems","When you need to simplify setup without requiring the Rust compiler or CUDA toolkit for initial use"],"when_not_to_use":["Avoid Mistral.rs if your project is strictly dependent on another programming language framework as it is implemented in Rust","If needing tight control over model-specific optimizations not provided by default prebuild paths, then consider alternatives with extensive fine-tuning options out-of-the-box"],"source":"enrich:decision_facts","observed_at":"2026-07-14T19:22:24.758Z"},"constraint_facets":null,"decision_summary":[{"label":"Adopt for","value":"Mistral.rs is ideal for developers requiring fast and flexible LLM inference with support across multiple platforms. It provides prebuilt binaries and a simple installation process."}]}}