PredRecon

📢 News

• 20/06/2023: The code of Hierarchical Planner is available.
• 06/06/2023: The simulator (AirSim) is available.
• 10/02/2023: The code of Surface Prediction Module (SPM) is available.

📜 Introduction

[ICRA'23] This repository maintains the implementation of "PredRecon: A Prediction-boosted Planning Framework for Fast and High-quality Autonomous Aerial Reconstruction".

Paper: arXiv, IEEE

Complete video: Video

Authors: Chen Feng, Haojia Li, Fei Gao, Boyu Zhou, and Shaojie Shen.

Institutions: HKUST Aerial Robotics Group, SYSU STAR Group, and ZJU FASTLab.

Please cite our paper if you use this project in your research:

• PredRecon: A Prediction-boosted Planning Framework for Fast and High-quality Autonomous Aerial Reconstruction, Chen Feng, Haojia Li, Fei Gao, Boyu Zhou, and Shaojie Shen, 2023 IEEE International Conference on Robotics and Automation (ICRA).

@inproceedings{feng2023predrecon,
  title={PredRecon: A Prediction-boosted Planning Framework for Fast and High-quality Autonomous Aerial Reconstruction},
  author={Feng, Chen and Li, Haojia and Gao, Fei and Zhou, Boyu and Shen, Shaojie},
  booktitle={2023 IEEE International Conference on Robotics and Automation (ICRA)},
  pages={1207-1213},
  year={2023},
  organization={IEEE}
}

PredRecon is a prediction-boosted planning framework that can efficiently reconstruct high-quality 3D models for the target areas in unknown environments with a single flight. We obtain inspiration from humans can roughly infer the complete construction structure from partial observation. Hence, we devise a surface prediction module (SPM) to predict the coarse complete surfaces of the target from current partial reconstruction. Then, the uncovered surfaces are produced by online volumetric mapping waiting for the observation by UAV. Lastly, a hierarchical planner plans motions for 3D reconstruction, which sequentially find efficient global coverage paths, plans local paths for maximizing the performance of Multi-View Stereo (MVS) and generate smooth trajectories for image-pose pairs acquisition. We conduct benchmark in the realistic simulator, which validates the performance of PredRecon compared with classical and state-of-the-art methods.

Please kindly star ⭐️ this project if it helps you. We take great efforts to develop and maintain it 😁.

🛠️ Installation

The project has been tested on Ubuntu 20.04 LTS (ROS Noetic). Directly clone our package (using ssh here):

  cd ${YOUR_WORKSPACE_PATH}/src
  git clone https://github.com/HKUST-Aerial-Robotics/PredRecon.git

Then install individual components of PredRecon:

• To install Surface Prediction Module, please follow the steps in SPM.
• To install Hierarchical Planner, please follow the steps in Planner.

🚀 Quick Start

You can decide whether saving captured images by changing the parameter reconfsm/img_flag in algorithm.xml, where true for saving and false for not saving. Note: If you set such parameter as true, all captured images will be saved in reconfsm/img_dir_, which can be changed to your own path in algorithm.xml.

Open your Unreal Engine platform, then run AirSim simulator in the terminal:

  source devel/setup.zsh && roslaunch airsim_ros_pkgs airsim_node.launch

Firstly, run Rviz for trajectory visualization:

  source devel/setup.zsh && roslaunch exploration_manager rviz.launch

Then, run the simulation:

  source devel/setup.zsh && roslaunch exploration_manager recon.launch

Trigger the quadrotor to start flight for collecting information by the 2D Nav Goal tool in Rviz.

After the flight, all images-pose pairs captured is stored in your given folder. We recommend RealityCapture or COLMAP as the 3D reconstruction platform. If you want to uae COLMAP in your terminal, you can follow the usage example in COLMAP Usage.

💥 Known Issues

You can simply verify whether your AirSim is working properly

  source devel/setup.zsh && roslaunch airsim_ros_pkgs airsim_node.launch
  source devel/setup.zsh && roslaunch exploration_manager rviz.launch
  source devel/setup.zsh && roslaunch exploration_manager init_test.launch

If AirSim is working correctly, you will see the initial flight of the drone and the mapping results in both AirSim and Rviz.

❣️ Acknowledgements

We use NLopt for non-linear optimization, use LKH for travelling salesman problem, and thank the source code of mmdetection3d and FUEL.

🤗 FC-family Works

1. What is FC-family?

We aim to develop intelligent perception-centric flight to realize Fast Coverage / reConstruction / inspeCtion etc.

2. Projects list

• PredRecon (ICRA2023): Prediction-boosted Planner for Aerial Reconstruction.
• FC-Planner (ICRA2024): Highly Efficient Global Planner for Aerial Coverage.
• SOAR (IROS2024): Heterogenous Multi-UAV Planner for Aerial Reconstruction.