shimmy
Pure-Rust WebGPU inference engine, OpenAI-API compatible, GGUF native
GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026
40views this month
Decision brief
Shimmy is a Rust-based inference engine that supports WebGPU for local machine learning model inference, compatible with the OpenAI API and GGUF models, offering a command-line interface.
Good fit when
- When you need a Rust-based inference engine that supports WebGPU for running machine learning models locally.
- If you require an OpenAI API-compatible service for GGUF models.
Avoid when
- If you require a solution that supports languages other than Rust or frameworks that are not WebGPU-based.
- When you need a tool that integrates with Python or llama.cpp, as Shimmy explicitly avoids these dependencies.
- Requirements:
- Shimmy requires a GPU for optimal performance.; The tool is designed to be used with GGUF models and is compatible with the OpenAI API.
Observed Sep 18, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Active (18d since push)
- As of Sep 18, 2026
- Provenance
- Not a fork · Personal account
- As of Sep 18, 2026
- Security (OSV)
- No lockfile
- As of Sep 18, 2026
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
cargo add shimmySimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Shimmy is a Rust-based inference engine that supports WebGPU for running machine learning models locally. It is compatible with the OpenAI API and supports GGUF models, offering a command-line interface for easy model serving and inference.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Sep 18, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Sep 18, 2026
- Languages
- rust
Source: github.language · Sep 18, 2026
Categories
Tags
README
Quick Start Then in another terminal: Full install, model acquisition, GPU, VRAM sizing, platform specific builds: docs/quickstart.md License & Philosophy MIT License — see LICENSE. Shimmy will be free forever. Promise : This will never become a paid product. Shimmy is infrastruc...
For agents
This page has a .md twin and JSON over the API.