Home/Inference & Serving/openinfer

openinfer

openinfer-project/openinfer

Pure Rust CUDA LLM inference engine serving multiple models including Qwen3 and Kimi-K2

GitHub Website

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

64views this month

705 stars107 forksLast push Sep 18, 2026 Rust Apache-2.0

Decision brief

high-performance GPU-based inference engine for Rust developers targeting Qwen3 and Kimi-K2 using pure CUDA kernels

Good fit when

When you are working with large language models Qwen3 and/or Kimi-K2 specifically, and want to avoid PyTorch dependencies.
If your project requires a high-performance inference engine that is compatible with the OpenAI API, built for Rust developers.

Avoid when

Avoid if you are developing models other than Qwen3 or Kimi-K2 as support for other models might be limited.
Not recommended for projects where PyTorch integration is crucial as this tool does not depend on it and may require changes in existing workflows.

Observed Jul 16, 2026 · Source: enrich:decision_facts

openinfer alternatives Trust report

openinfer vs open-webui openinfer vs gpt4all

Verify the decision

Maintenance and security

Full trust report

Maintenance: Very active (1d since push); As of Sep 20, 2026
Provenance: Not a fork · Organization account; As of Sep 20, 2026
Security (OSV): No lockfile; As of Jul 11, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

cargo add openinfer

crates.io

How it fits your stack(2)

Typed graph edges - alternatives, integrations, successors, and dependencies. Ranked by relationship type, not raw GitHub stars.

Alternative

open-webui

Both projects aim to provide inference engines for locally running LLMs with a user-friendly interface or API.

alternative

gpt4all

Both focus on LLM inference but with different technical approaches (GPT4All is broader in scope while openinfer focuses more on Rust CUDA).

alternative

Relationship graph

Optional deeper exploration of typed edges and category neighbours.

Similar tools

Same-category neighbours not already linked as typed edges.

ollama

Get up and running with Kimi, GLM, MiniMax, DeepSeek, gpt-oss, Qwen, Gemma and other models.

181k★

llm

Access large language models from the command-line

12k★

inference

Unified production-ready inference API for various LLMs and models

9.6k★

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Provides a high-performance inference solution for large language models on GPU with no dependency on PyTorch, offering an OpenAI API-compatible service.

Capability facts

Languages: rust
Source: github.language · Sep 20, 2026

README

License Apache 2.0. See NOTICE for third party attributions. The Dynamo derived crate retains its original Apache 2.0 headers and NVIDIA Dynamo notice.

For agents

This page has a .md twin and JSON over the API.

/tools/openinfer-project-openinfer.md GET /api/graphcanon/tools/openinfer-project-openinfer GET /api/graphcanon/graph?tool=openinfer-project-openinfer