{"data":{"slug":"michael-a-kuykendall-shimmy","name":"shimmy","tagline":"Pure-Rust WebGPU inference engine, OpenAI-API compatible, GGUF native","github_url":"https://github.com/Michael-A-Kuykendall/shimmy","owner":"Michael-A-Kuykendall","repo":"shimmy","owner_avatar_url":"https://avatars.githubusercontent.com/u/196678413?v=4","primary_language":"Rust","stars":5890,"forks":576,"topics":["api-server","command-line-tool","developer-tools","gguf","huggingface","huggingface-models","huggingface-transformers","inference-server","llama","llamacpp","llm-inference","local-ai","machine-learning","ollama-api","openai-compatible","rust","rust-crate","transformers","webgpu","webgpu-shaders"],"archived":false,"github_pushed_at":"2026-08-30T22:45:24+00:00","maintenance_label":"Active","stars_delta_30d":193,"url":"https://www.graphcanon.com/tools/michael-a-kuykendall-shimmy","markdown_url":"https://www.graphcanon.com/tools/michael-a-kuykendall-shimmy.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/michael-a-kuykendall-shimmy","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=michael-a-kuykendall-shimmy","description":"⚡ Pure-Rust WebGPU inference engine — OpenAI-API compatible, GGUF native, runs on any GPU. No Python. No llama.cpp. Single binary.","homepage_url":null,"license":"Apache-2.0","open_issues":10,"watchers":41,"ai_summary":"Shimmy is a Rust-based inference engine that supports WebGPU for running machine learning models locally. It is compatible with the OpenAI API and supports GGUF models, offering a command-line interface for easy model serving and inference.","readme_excerpt":"## Quick Start\n\n```bash\ncargo install shimmy\nshimmy serve --model-path /absolute/path/to/model.gguf --bind 127.0.0.1:11435\n```\n\nThen in another terminal:\n\n```bash\nshimmy list --short\ncurl -s http://127.0.0.1:11435/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"tinyllama-1.1b\",\"messages\":[{\"role\":\"user\",\"content\":\"Say hi in 5 words.\"}],\"max_tokens\":32}'\n```\n\nFull install, model acquisition, GPU, VRAM sizing, platform-specific builds: **[docs/quickstart.md](docs/quickstart.md)**\n\n---\n\n---\n\n## License & Philosophy\n\nMIT License — see [LICENSE](LICENSE). **Shimmy will be free forever.**\n\n**Promise**: This will never become a paid product.\n\nShimmy is infrastructure: it should be invisible. Reliability through comprehensive validation and property-based testing.\n\n---\n\n**Maintainer**: Michael A. Kuykendall · **Mission**: Making local model inference simple and reliable\n\n---","github_created_at":"2025-08-28T22:55:46+00:00","created_at":"2026-07-11T11:42:39.325072+00:00","updated_at":"2026-09-20T04:29:42.1053+00:00","categories":[{"slug":"developer-tools","name":"Developer Tools","url":"https://www.graphcanon.com/categories/developer-tools","markdown_url":"https://www.graphcanon.com/categories/developer-tools.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/developer-tools"},{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"api-server","name":"api-server"},{"slug":"command-line-tool","name":"command-line-tool"},{"slug":"gguf","name":"gguf"},{"slug":"huggingface","name":"huggingface"},{"slug":"inference-server","name":"inference-server"},{"slug":"llm-inference","name":"llm-inference"},{"slug":"openai-compatible","name":"openai-compatible"},{"slug":"rust","name":"rust"}],"trust":{"provenance":{"is_fork":false,"github_id":1046584285,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-18T08:17:52.618Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":11,"days_since_push":18,"last_release_at":"2026-08-30T22:48:06Z","stars_delta_30d":193,"open_issues_delta_30d":-1},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-09-18T08:17:54.226Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-18T08:17:51.537Z"},"deploy":{"source":"dockerfile:Dockerfile","self_host":true,"observed_at":"2026-09-18T08:17:51.537Z","managed_saas":false},"languages":{"value":["rust"],"source":"github.language","observed_at":"2026-09-18T08:17:51.537Z"},"has_docker":{"value":true,"source":"dockerfile:Dockerfile","observed_at":"2026-09-18T08:17:51.537Z"},"license_spdx":{"value":"Apache-2.0","source":"github.license","observed_at":"2026-09-18T08:17:51.537Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Shimmy requires a GPU for optimal performance.","The tool is designed to be used with GGUF models and is compatible with the OpenAI API."],"min_ram_gb":null,"requires_docker":false},"constraints":{"min_ram_gb":null,"requires_docker":false},"when_to_use":["When you need a Rust-based inference engine that supports WebGPU for running machine learning models locally.","If you require an OpenAI API-compatible service for GGUF models.","For developers preferring a single binary solution without Python dependencies or llama.cpp.","When you need a tool that is free and will remain free, with a commitment to reliability through comprehensive validation and property-based testing."],"when_not_to_use":["If you require a solution that supports languages other than Rust or frameworks that are not WebGPU-based.","When you need a tool that integrates with Python or llama.cpp, as Shimmy explicitly avoids these dependencies.","If you are looking for a multi-component system; Shimmy is designed as a single binary for simplicity.","When you need a paid product with additional features or support, as Shimmy is committed to being free forever."],"source":"enrich:decision_facts","observed_at":"2026-09-18T10:09:22.652Z"},"constraint_facets":{"min_ram_gb":null,"requires_docker":false},"decision_summary":[{"label":"Requirements","value":"Shimmy requires a GPU for optimal performance.; The tool is designed to be used with GGUF models and is compatible with the OpenAI API."},{"label":"Adopt for","value":"Shimmy is a Rust-based inference engine that supports WebGPU for local machine learning model inference, compatible with the OpenAI API and GGUF models, offering a command-line interface."},{"label":"License detail","value":"Apache-2.0"}]}}