shimmy
⚡ A Pure-Rust WebGPU Inference Engine, OpenAI-API Compatible and Native to GGUF
GraphCanon updated 1d · GitHub synced 1d
Decision brief
Shimmy is a Rust-based inference engine that excels in running AI models on various GPUs without the need for Python or llama.cpp dependencies. It provides an OpenAI API-compatible interface and supports GGUF natively.
Good fit when
- - When you want to run AI models with WebGPU support directly through Rust, reducing dependency overhead associated with Python environments
- - For projects requiring a lightweight single binary solution that doesn't rely on external libraries like llama.cpp for GPU inference
Avoid when
- - If your project specifically requires Python-based dependencies or you prefer using the llama.cpp framework for model inference
- - In scenarios where compatibility with a wide range of existing Python machine learning ecosystems and their comprehensive tooling is necessary
Observed Jul 12, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (4d since push)
- As of 1d
- Provenance
- Not a fork · Personal account
- As of 1d
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
cargo add shimmySimilar tools
Same-category neighbours. No typed graph edges are catalogued for this tool yet.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
A Rust-based inference engine that supports WebGPU for AI model inference without dependency on Python or llama.cpp. It is designed to be compatible with the OpenAI API and can run on various GPUs.
Capability facts
- Deploy
- Self-host
Source: dockerfile:Dockerfile · Aug 24, 2026
- Docker
- Dockerfile present
Source: dockerfile:Dockerfile · Aug 24, 2026
- Languages
- rust
Source: github.language · Aug 24, 2026
Categories
Tags
README
Getting Started
| Document | Description |
|---|---|
| quickstart.md | Install, models, GPU, VRAM, extended context |
| MIGRATION_v2.md | Migrating from Shimmy v1.x |
| CONFIGURATION.md | All environment variables and config options |
| WINDOWS_GPU_BUILD_GUIDE.md | Windows-specific build instructions |
License & Philosophy
MIT License - forever and always.
Philosophy: Infrastructure should be invisible. Shimmy is infrastructure.
Testing Philosophy: Reliability through comprehensive validation and property-based testing.
Forever maintainer: Michael A. Kuykendall Promise: This will never become a paid product Mission: Making local model inference simple and reliable
For agents
This page has a .md twin and JSON over the API.