{"data":{"slug":"tencent-forward","name":"Forward","tagline":"A library for high performance deep learning inference on NVIDIA GPUs.","github_url":"https://github.com/Tencent/Forward","owner":"Tencent","repo":"Forward","owner_avatar_url":"https://avatars.githubusercontent.com/u/18461506?v=4","primary_language":"C++","stars":556,"forks":62,"topics":["cuda","deep-learning","forward","gpu","inference","inference-engine","keras","neural-network","onnx","pytorch","tensorflow","tensorrt"],"archived":false,"github_pushed_at":"2022-01-29T16:34:05+00:00","maintenance_label":"Dormant","url":"https://www.graphcanon.com/tools/tencent-forward","markdown_url":"https://www.graphcanon.com/tools/tencent-forward.md","api_url":"https://www.graphcanon.com/api/graphcanon/tools/tencent-forward","graph_url":"https://www.graphcanon.com/api/graphcanon/graph?tool=tencent-forward","description":"A library for high performance deep learning inference on NVIDIA GPUs. ","homepage_url":null,"license":"Other","open_issues":0,"watchers":20,"ai_summary":null,"readme_excerpt":"# Forward 深度学习推理加速框架\n\n \n\n----\n\n- [Forward 深度学习推理加速框架](#forward-深度学习推理加速框架)\n  - [什么是 Forward](#什么是-forward)\n  - [为什么选择 Forward](#为什么选择-forward)\n  - [快速上手 Forward](#快速上手-forward)\n    - [环境依赖](#环境依赖)\n    - [项目构建](#项目构建)\n    - [Forward-Cpp 使用](#forward-cpp-使用)\n    - [Forward-Python 使用](#forward-python-使用)\n    - [Forward-Bert 使用](#forward-bert-使用)\n    - [更多使用方法](#更多使用方法)\n    - [Logging 日志](#logging-日志)\n  - [模型和算子支持](#模型和算子支持)\n    - [模型](#模型)\n    - [算子](#算子)\n  - [参考资料](#参考资料)\n  - [贡献](#贡献)\n  - [许可证](#许可证)\n\n----\n\n[[English Version](README_EN.md)]\n\n## 什么是 Forward\n\nForward 是一款腾讯研发的 GPU 高性能推理加速框架。它提出了一种解析方案，可直接加载主流框架模型（Tensorflow / PyTorch / Keras / ONNX）转换成 TensorRT 推理加速引擎，帮助用户节省中间繁杂的模型转换或网络构建步骤。相对于直接使用 TensorRT，Forward 更易用以及更容易扩展支持更多模型和算子。目前，Forward 除了覆盖支持主流的 CV，NLP 及推荐领域的深度学习模型外，还支持一些诸如 BERT，FaceSwap，StyleTransfer 这类高级模型。\n\n## 为什么选择 Forward\n\n- **模型性能优化高**：基于 TensorRT API 开发网络层级的支持，保证对于通用网络层级的推理性能优化处于最优级别；\n- **模型支持范围广**：除了通用的 CV，NLP，及推荐类模型，还支持一些诸如 BERT，FaceSwap，StyleTransfer 这类高级模型；\n- **多种推理模式**：支持 FLOAT / HALF / INT8 推理模式；\n- **接口简单易用**：直接导入已训练好的 Tensorflow(.pb) / PyTorch(.pth) / Keras(.h5) / ONNX(.onnx) 模型文件，隐式转换为高性能的推理 Engine 进行推理加速；\n- **支持自研扩展**：可根据业务模型[扩展支持自定义网络层级](doc/cn/usages/add_support_op_CN.md)；\n- **支持 C++ 和 Python 接口调用**。\n\n## 快速上手 Forward\n\n### 环境依赖\n\n- NVIDIA CUDA >= 10.0, CuDNN >= 7 (推荐 CUDA 10.2 以上)\n- TensorRT >= 7.0.0.11 (推荐 TensorRT-7.2.1.6)\n- CMake >= 3.12.2\n- GCC >= 5.4.0, ld >= 2.26.1\n- PyTorch >= 1.7.0\n- TensorFlow >= 1.15.0 (若使用 Linux 操作系统，需额外下载 [Tensorflow 1.15.0](https://github.com/neargye-forks/tensorflow/releases)，并将解压出来的 `.so` 文件拷贝至 `Forward/source/third_party/tensorflow/lib` 目录下)\n- Keras HDF5 (从 `Forward/source/third_party/hdf5` 源码构建)\n\n### 项目构建\n\n使用 CMake 进行构建生成 Makefiles 或者 Visual Studio 项目。根据使用目的，Forward 可构建成适用于不同框架的库，如 Fwd-Torch、Fwd-Python-Torch、Fwd-Tf、Fwd-Python-Tf、Fwd-Keras、Fwd-Python-Keras、Fwd-Onnx 和 Fwd-Python-Onnx。\n\n以 Linux 平台构建 Fwd-Tf 为例，\n\n步骤一：克隆项目\n```bash\n1 git clone https://github.com/Tencent/Forward.git\n```\n步骤二：下载 `Tensorflow 1.15.0`（仅在 Linux 平台使用 Tensorflow 框架推理时需要）\n```bash\n1 cd Forward/source/third_party/tensorflow/\n2 wget https://github.com/neargye-forks/tensorflow/releases/download/v1.15.0/libtensorflow-cpu-linux-x86_64-1.15.0.tar.gz\n3 tar -xvf libtensorflow-gpu-linux-x86_64-1.15.0.tar.gz\n```\n步骤三：创建 `build` 文件夹\n```bash\n1 cd ~/Forward/\n2 rm -rf build\n3 mkdir -p build\n4 cd build/\n```\n步骤四：使用 `cmake` 生成构建关系，需指定 `TensorRT_ROOT` 安装路径\n```bash\n1 cmake ..  -DTensorRT_ROOT=<path_to_TensorRT> -DENABLE_TENSORFLOW=ON -DENABLE_UNIT_TESTS=ON\n```\n步骤五：使用 `make` 构建项目\n```bash\n1 make -j\n```\n步骤六：运行 `unit_test` 验证项目是否构建成功\n```bash\ncd bin/\n./unit_test --gtest_filter=TestTfNodes.*\n\n# 出现已下提示表示项目构建成\n# [       OK ] TestTfNodes.ZeroPadding (347 ms)\n# [----------] 22 tests from TestTfNodes (17555 ms total)\n\n# [----------] Global test environment tear-down\n# [==========] 22 tests from 1 test case ran. (17555 ms total)\n# [  PASSED  ] 22 tests.\n```\n\n更多构建流程可参考 [CMake 构建流程](doc/cn/usages/cmake_build_CN.md) 。\n\n### Forward-Cpp 使用\n\n参考 [Demo for using Forward-Cpp in Linux](demo/fwd_cpp/ReadMe_CN.md)\n\n### Forward-Python 使用\n\n参考 [Demo for using Forward-Python](demo/fwd_py/ReadMe_CN.md)\n\n### Forward-Bert 使用\n\nRefer to [Demo for using Forward-Bert](demo/bert/README_CN.md)\n\n### 更多使用方法\n\n**注意**: 模型输入名可通过模型查看器来查看, 例如用 [Netron](https://github.com/lutzroeder/Netron) 查看。\n\n- [PyTorch 使用说明](doc/cn/usages/torch_usage_CN.md)\n- [TensorFlow 使用说明](doc/cn/usages/tf_usage_CN.md)\n- [Keras 使用说明](doc/cn/usages/keras_usage_CN.md)\n- [ONNX 使用说明](doc/cn/usages/onnx_usage_CN.md)\n\n### Logging 日志\n\nForward 使用 [easylogging++](https://github.com/amrayn/easyloggingpp) 作为日志功能，并使用 `forward_log.conf` 作为日志配置文件。 \n\n- 若工作目录中存在 `forward_log.conf` 文件，Forward 将使用该配置文件，更多内容可参考 [Using-configuration-file](https://github.com/amrayn/easyloggingpp#using-configuration-file)；\n- 若工作目录中不存在 `forward_log.conf` 文件，Forward 将使用默认配置，并将日志记录到 `logs/myeasylog.log` 。\n\n`forward_log.conf` 文件配置样例\n```bash\n* GLOBAL:\n  FORMAT               =  \"[%level] %datetime %fbase(%line): %msg\"\n  FILENAME","github_created_at":"2021-03-11T02:40:04+00:00","created_at":"2026-07-11T23:37:11.870688+00:00","updated_at":"2026-07-11T23:37:30.789529+00:00","categories":[{"slug":"inference-serving","name":"Inference & Serving","url":"https://www.graphcanon.com/categories/inference-serving","markdown_url":"https://www.graphcanon.com/categories/inference-serving.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/inference-serving"},{"slug":"model-training","name":"Model Training","url":"https://www.graphcanon.com/categories/model-training","markdown_url":"https://www.graphcanon.com/categories/model-training.md","api_url":"https://www.graphcanon.com/api/graphcanon/categories/model-training"}],"tags":[{"slug":"cuda","name":"cuda"},{"slug":"deep-learning","name":"deep-learning"},{"slug":"forward","name":"forward"},{"slug":"gpu","name":"gpu"},{"slug":"inference","name":"inference"},{"slug":"inference-engine","name":"inference-engine"},{"slug":"keras","name":"keras"},{"slug":"neural-network","name":"neural-network"}],"trust":{"provenance":{"is_fork":false,"github_id":346556304,"owner_type":"Organization","methodology":"github_public_v1","parent_repo":null,"near_duplicate_slugs":[]},"computed_at":"2026-07-11T23:37:15.292Z","maintenance":{"label":"Dormant","score":18,"methodology":"github_public_v1","releases_90d":0,"days_since_push":1624,"last_release_at":"2021-11-30T10:30:48Z"},"security_summary":{"status":"no_lockfile","scanner":null,"low_count":0,"high_count":0,"last_scan_at":"2026-07-11T23:37:15.786Z","medium_count":0,"scan_profile":"none","critical_count":0}},"capability_facts":{"scan":{"source":"repo_scan","observed_at":"2026-07-11T23:37:15.047Z"},"languages":{"value":["c++"],"source":"github.language","observed_at":"2026-07-11T23:37:15.047Z"},"license_spdx":{"value":"Other","source":"github.license","observed_at":"2026-07-11T23:37:15.047Z"}}}}