{"data":{"slug":"kibae-onnxruntime-server","name":"onnxruntime-server","tagline":"ONNX Runtime Server for TCP and HTTP/HTTPS REST APIs","github_url":"https://github.com/kibae/onnxruntime-server","owner":"kibae","repo":"onnxruntime-server","owner_avatar_url":"https://avatars.githubusercontent.com/u/3859542?v=4","primary_language":"C++","stars":199,"forks":19,"topics":["ai","contributions-welcome","cuda","deep-learning","inference-server","machine-learning","nueral-networks","onnx","onnxruntime"],"archived":false,"github_pushed_at":"2026-09-11T10:21:53+00:00","maintenance_label":"Active","stars_delta_30d":4,"url":"https://www.graphcanon.com/tools/kibae-onnxruntime-server","markdown_url":"https://www.graphcanon.com/tools/kibae-onnxruntime-server.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/kibae-onnxruntime-server","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=kibae-onnxruntime-server","description":"ONNX Runtime Server: The ONNX Runtime Server is a server that provides TCP and HTTP/HTTPS REST APIs for ONNX inference.","homepage_url":null,"license":"MIT","open_issues":8,"watchers":2,"ai_summary":"Provides inference services via ONNX Runtime over TCP and HTTP/HTTPS REST APIs.","readme_excerpt":"## Requirements\n\n- [ONNX Runtime](https://onnxruntime.ai/)\n- [Boost](https://www.boost.org/)\n- [CMake](https://cmake.org/), pkg-config\n- CUDA(*optional, for Nvidia GPU support*)\n- OpenSSL(*optional, for HTTPS*)\n\n----\n\n---\n\n## Install ONNX Runtime\n\n#### Linux\n\n- Use `download-onnxruntime-linux.sh` script\n    - This script downloads the latest version of the binary and install to `/usr/local/onnxruntime`.\n    - Also, add `/usr/local/onnxruntime/lib` to `/etc/ld.so.conf.d/onnxruntime.conf` and run `ldconfig`.\n- Or manually download binary from [ONNX Runtime Releases](https://github.com/microsoft/onnxruntime/releases).\n\n#### Mac OS\n\n```shell\nbrew install onnxruntime\n```\n\n----\n\n---\n\n## Install dependencies\n\n#### Ubuntu/Debian\n\n```shell\nsudo apt install cmake pkg-config libboost-all-dev libssl-dev\n\n---\n\n# optional, for Nvidia GPU support with Docker\nsudo apt install nvidia-container-toolkit\n```\n\n#### Mac OS\n\n```shell\nbrew install cmake boost openssl\n```\n\n----\n\n---\n\n## Compile and Install\n\n```shell\ncmake -B build -S . -DCMAKE_BUILD_TYPE=Release\ncmake --build build --parallel\nsudo cmake --install build --prefix /usr/local/onnxruntime-server\n```\n\n----\n\n---\n\n# Install via a package manager\n\n| OS         | Method | Command                     |\n|------------|--------|-----------------------------|\n| Arch Linux | AUR    | `yay -S onnxruntime-server` |\n\n----\n\n---\n\n# Docker\n\n- Docker hub: [kibaes/onnxruntime-server](https://hub.docker.com/r/kibaes/onnxruntime-server)\n    - [\n      `1.30.0-linux-cuda13`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda13.dockerfile)\n      amd64(CUDA 13.x, cuDNN 9.x)\n    - [\n      `1.30.0-linux-cuda12`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cuda12.dockerfile)\n      amd64(CUDA 12.x, cuDNN 9.x)\n    - [\n      `1.30.0-linux-cpu`](https://github.com/kibae/onnxruntime-server/blob/main/deploy/build-docker/linux-cpu.dockerfile)\n      amd64, arm64\n\n```shell\nDOCKER_IMAGE=kibaes/onnxruntime-server:1.30.0-linux-cuda13 # or 1.30.0-linux-cuda12 or 1.30.0-linux-cpu\n\ndocker pull ${DOCKER_IMAGE}","github_created_at":"2023-09-04T17:42:13+00:00","created_at":"2026-07-15T11:19:56.979589+00:00","updated_at":"2026-09-20T05:20:32.575812+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"}],"tags":[{"slug":"ai","name":"ai"},{"slug":"cuda","name":"cuda"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"inference-server","name":"inference-server"},{"slug":"machine-learning","name":"machine-learning"},{"slug":"onnx","name":"onnx"},{"slug":"onnxruntime","name":"onnxruntime"}],"trust":{"provenance":{"is_fork":false,"github_id":687136184,"owner_type":"User","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-09-20T05:20:30.526Z","maintenance":{"label":"Active","score":82,"methodology":"github_public_v1","releases_90d":5,"days_since_push":8,"last_release_at":"2026-09-11T10:22:58Z","stars_delta_30d":4,"open_issues_delta_30d":0},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-15T11:19:59.862Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-09-20T05:20:31.832Z"},"languages":{"value":["c++"],"source":"github.language","observed_at":"2026-09-20T05:20:31.832Z"},"license_spdx":{"value":"MIT","source":"github.license","observed_at":"2026-09-20T05:20:31.832Z"}},"decision_facts":{"hosting":null,"pricing":null,"requirements":{"notes":["Install ONNX Runtime, Boost, CMake, and pkg-config.","CUDA is optional for Nvidia GPU support; OpenSSL is required only if HTTPS is needed."],"min_ram_gb":null,"requires_docker":true},"constraints":{"min_ram_gb":null,"requires_docker":true},"when_to_use":["You need a flexible serving solution where models can be accessed via both TCP and HTTP/HTTPS REST APIs simultaneously, allowing for diverse application integration scenarios.","Your setup requires a high-performance inferencing engine with optional support for Nvidia GPU acceleration through CUDA, enhancing the speed of deep learning inference tasks."],"when_not_to_use":["If your deployment environment only supports package managers without official onnxruntime-server packages like Debian or MacOSbrew (requires manual installation).","When your architecture lacks support for CUDA and you cannot utilize its performance benefits, as other frameworks might offer more straightforward CPU-only setups."],"source":"enrich:decision_facts","observed_at":"2026-07-17T11:54:40.400Z"},"constraint_facets":{"min_ram_gb":null,"requires_docker":true},"decision_summary":[{"label":"Requirements","value":"Requires Docker; Install ONNX Runtime, Boost, CMake, and pkg-config.; CUDA is optional for Nvidia GPU support; OpenSSL is required only if HTTPS is needed."},{"label":"Adopt for","value":"The ONNX Runtime Server delivers ONNX model inference services over TCP and HTTP/HTTPS through its versatile server framework."}]}}