Home/Inference & Serving/onnxruntime-server
onnxruntime-server logo

onnxruntime-server

kibae/onnxruntime-server

ONNX Runtime Server for TCP and HTTP/HTTPS REST APIs

GraphCanon updated Sep 20, 2026 · GitHub synced Sep 20, 2026

199 stars19 forksLast push Sep 11, 2026 C++ MIT

Decision brief

Unverified catalog guidance: The ONNX Runtime Server delivers ONNX model inference services over TCP and HTTP/HTTPS through its versatile server framework.

Good fit when

  • Unverified catalog guidance: You need a flexible serving solution where models can be accessed via both TCP and HTTP/HTTPS REST APIs simultaneously, allowing for diverse application integration scenarios.
  • Unverified catalog guidance: Your setup requires a high-performance inferencing engine with optional support for Nvidia GPU acceleration through CUDA, enhancing the speed of deep learning inference tasks.

Avoid when

  • Unverified catalog guidance: If your deployment environment only supports package managers without official onnxruntime-server packages like Debian or MacOSbrew (requires manual installation).
  • Unverified catalog guidance: When your architecture lacks support for CUDA and you cannot utilize its performance benefits, as other frameworks might offer more straightforward CPU-only setups.
Requirements:
Requires Docker; Install ONNX Runtime, Boost, CMake, and pkg-config.; CUDA is optional for Nvidia GPU support; OpenSSL is required only if HTTPS is needed.

Observed Jul 17, 2026 · Source: enrich:decision_facts

Verify the decision

Maintenance and security

Full trust report
Maintenance
Active (8d since push)
As of Sep 20, 2026
Provenance
Not a fork · Personal account
As of Sep 20, 2026
Security (OSV)
No lockfile
As of Jul 15, 2026

Public GitHub metadata and optional OSV scans. Signals, not a guarantee. Trust methodology.

Install

Package installation is not established from the available documentation. Check the source repository for setup instructions.

git clone https://github.com/kibae/onnxruntime-server
Source repository

Similar tools

Same-category neighbours. No typed graph edges are catalogued for this tool yet.

Evidence and technical details

Sourced facts, taxonomy, compatibility claims, README excerpt, and machine-readable endpoints.

Overview

Provides inference services via ONNX Runtime over TCP and HTTP/HTTPS REST APIs.

Capability facts

Languages
c++

Source: github.language · Sep 20, 2026

Categories

Tags

README

Requirements ONNX Runtime Boost CMake, pkg config CUDA( optional, for Nvidia GPU support ) OpenSSL( optional, for HTTPS ) Install ONNX Runtime Linux Use script This script downloads the latest version of the binary and install to . Also, add to and run . Or manually download bina...

For agents

This page has a .md twin and JSON over the API.

Was this helpful?

Anonymous feedback helps us improve pages and translations.