openinfer
Pure Rust CUDA LLM inference engine serving multiple models including Qwen3 and Kimi-K2
GraphCanon updated today · GitHub synced today · 30 views this month
Decision brief
high-performance GPU-based inference engine for Rust developers targeting Qwen3 and Kimi-K2 using pure CUDA kernels
Good fit when
- When you are working with large language models Qwen3 and/or Kimi-K2 specifically, and want to avoid PyTorch dependencies.
- If your project requires a high-performance inference engine that is compatible with the OpenAI API, built for Rust developers.
Avoid when
- Avoid if you are developing models other than Qwen3 or Kimi-K2 as support for other models might be limited.
- Not recommended for projects where PyTorch integration is crucial as this tool does not depend on it and may require changes in existing workflows.
Observed Jul 16, 2026 · Source: enrich:decision_facts
Verify the decision
Maintenance and security
Full trust report- Maintenance
- Very active (0d since push)
- As of today
- Provenance
- Not a fork · Organization account
- As of today
- Security (OSV)
- No lockfile
- As of 1mo
Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.
Install
cargo add openinferHow it fits your stack(2)
Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.
Alternative
Relationship graph
Optional deeper exploration of typed edges and category neighbours.
Similar tools
Same-category neighbours not already linked as typed edges.
Evidence and technical details
Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.
Overview
Provides a high-performance inference solution for large language models on GPU with no dependency on PyTorch, offering an OpenAI API-compatible service.
Capability facts
- Languages
- rust
Source: github.language · Aug 25, 2026
Categories
Tags
README
License
Apache-2.0 — see LICENSE and NOTICE. Components ported from
NVIDIA Dynamo (the kvbm/kvbm-logical crate) retain their original Apache-2.0 headers; see
NOTICE_DYNAMO.
For agents
This page has a .md twin and JSON over the API.