ONNX-MLIR

This project (https://onnx.ai/onnx-mlir/) provides compiler technology to transform a valid Open Neural Network Exchange (ONNX) graph into code that implements the graph with minimum runtime support. It implements the ONNX standard and is based on the underlying LLVM/MLIR compiler technology.

System	Build Status	Model Zoo Status
s390x-Linux
amd64-Linux
amd64-Windows
amd64-macOS

This project contributes:

• an ONNX Dialect that can be integrated in other projects,
• a compiler interfaces that lower ONNX graphs into MLIR files/LLVM bytecodes/C & Java libraries,
• an onnx-mlir driver to perform these lowering,
• and a python/C/C++/Java runtime environment.

Current levels of support for the code generation of ONNX operations are listed here for a generic CPU and IBM's Telum integrated AI accelerator.

Interacting with the community.

For ongoing discussions, we use an #onnx-mlir-discussion slack channel established under the Linux Foundation AI and Data Workspace. Join this workspace using this link.

We use GitHub Issues for request for comments, questions, or bug reports. Security-related issues are reported using the channels listed in the SECURITY page.

We hold informal weekly meetings on Tuesdays where we discuss current issues and progress. Meeting agenda, notes, and links (to participate) are found here. Please email alexe@us.ibm.com to request a 15-30 min time slot to discuss a specific topic of interest.

Setting up ONNX-MLIR using Prebuilt Containers

The preferred approach to using and developing ONNX-MLIR is to use Docker Images and Containers, as getting the proper code dependences may be tricky on some systems. Our instructions on using ONNX-MLIR with Dockers are here.

If you intend to develop code, you should look at our workflow document which help you setup your Docker environment in a way that let you contribute code easily.

Setting up ONNX-MLIR directly

ONNX-MLIR runs natively on Linux, OSX, and Windows. Detailed instructions are provided below.

Prerequisites

python >= 3.11
clang >= 18.1.3
protobuf >= 33.5
cmake >= 3.26.0
make >= 4.3 or ninja >= 1.10.2
java >= 21 (optional)

All the PyPi package dependencies and their appropriate versions are captured in requirements.txt.

Look here for help to set up the prerequisite software.

At any point in time, ONNX-MLIR depends on a specific commit of the LLVM project that has been shown to work with the project. Periodically the maintainers need to move to a more recent LLVM level. Among other things, this requires to update the LLVM commit string in clone-mlir.sh. When updating ONNX-MLIR, it is good practice to check that the commit string of the MLIR/LLVM is the same as the one listed in that file. See instructions here when third-party ONNX also need to be updated.

Build

Directions to install MLIR and ONNX-MLIR are dependent on your OS.

• Linux or OSX.
• Windows.

After installation, an onnx-mlir executable should appear in the build/Debug/bin or build/Release/bin directory.

If you have difficulties building, rebuilding, or testing onnx-mlir, check this page for helpful hints.

Using ONNX-MLIR

The usage of onnx-mlir is as such:

OVERV