NVIDIA Redtail project

Autonomous visual navigation components for drones and ground vehicles using deep learning. Refer to wiki for more information on how to get started.

This project contains deep neural networks, computer vision and control code, hardware instructions and other artifacts that allow users to build a drone or a ground vehicle which can autonomously navigate through highly unstructured environments like forest trails, sidewalks, etc. Our TrailNet DNN for visual navigation is running on NVIDIA's Jetson embedded platform. Our arXiv paper describes TrailNet and other runtime modules in detail.

The project's deep neural networks (DNNs) can be trained from scratch using publicly available data. A few pre-trained DNNs are also available as a part of this project. In case you want to train TrailNet DNN from scratch, follow the steps on this page.

The project also contains Stereo DNN models and runtime which allow to estimate depth from stereo camera on NVIDIA platforms.

IROS 2018: we presented our work at IROS 2018 conference as a part of Vision-based Drones: What's Next? workshop.

CVPR 2018: we presented our work at CVPR 2018 conference as a part of Workshop on Autonomous Driving.

References and Demos

• Stereo DNN, CVPR18 paper, Stereo DNN video demo
• TrailNet Forest Drone Navigation, IROS17 paper, TrailNet DNN video demo
• GTC 2017 talk: slides, video
• Demo video showing 250m autonomous flight with TrailNet DNN flying the drone
• Demo video showing our 1 kilometer autonomous drone flight with TrailNet DNN
• Demo video showing TrailNet DNN driving a robotic rover around the office
• Demo video showing TrailNet generalization to ground vehicles and other environments

News

• 2020-02-03: Alternative implementations. redtail is no longer being developed, but fortunately our community stepped in and continued developing the project. We thank our users for the interest in redtail, questions and feedback!

  Some alternative implementations are listed below.

  • @mtbsteve: https://github.com/mtbsteve/redtail

• 2018-10-10: Stereo DNN ROS node and fixes.

  • Added Stereo DNN ROS node and visualizer node.
  • Fixed issue with nvidia-docker v2.

• 2018-09-19: Updates to Stereo DNN.

  • Moved to TensorRT 4.0
  • Enabled FP16 support in ResNet18 2D model, resulting in 2x performance increase (20fps on Jetson TX2).
  • Enabled TensorRT serialization in ResNet18 2D model to reduce model loading time from minutes to less than a second.
  • Better logging and profiler support.

• 2018-06-04: CVPR 2018 workshop. Fast version of Stereo DNN.

  • Presenting our work at CVPR 2018 conference as a part of Workshop on Autonomous Driving.
  • Added fast version of Stereo DNN model based on ResNet18 2D model. The model runs at 10fps on Jetson TX2. See README for details and check out updated sample_app.

• GTC 2018: Here is our [Stereo DNN session page at GTC18](https://2018gputechconf.smarteventscloud.com/connect/sessionDetai