MAV Software
Figure 1 Overview
Our typical software communication setup for autonomous flight is shown in Figure 1.
The Base Station and NUC are connected to a common wifi network.
We use the Base Station to remotely control the NUC via ssh to start and stop the necessary software running on the drone's PC.
We recommend using screen when connecting remotely (install with sudo apt install screen) in order to reconnect to any terminal if your ssh connection drops unexpectedly.
In addition, we run QGroundControl (ground control software for the flight controller) on the Base Station and use RViz to visualize ROS nodes running on the NUC. To get telemetry data, the telemetry radio is connected to the Base Station via USB. Similarly, if you have an RTK GPS base station it is also connected to the base station PC via USB (QGroundControl will then send the RTK correction terms to the flight controller over the telemetry radio connection).
The NUC is running the ROS core, a Mavros node for communication with the flight controller, the autonomous planning software, and visual-inertial state estimation if we are not using GPS.
Mavros is a translation layer running on the drone's PC that converts standard ROS messages into mavlink messages which the flight controller can understand. Follow the installation instructions on the github repo.
The ground control software is used to setup the flight controller, read log files, display live telemetry (e.g. flight mode and battery status), etc. You will need the software to change the flight controllers settings and for the first setup. More infos are available here.
Planning with ROS is relatively straightforward: your planning node can get the current state estimate from mavros (subscribing to /mavros/local_position/pose), and you can send command poses, command velocities, or command trajectories to mavros (publishing e.g. to /mavros/setpoint_position/local).
A short guide on RTK GPS is available here.
When using RTK GPS we can use the flight controller's world frame and state estimate for planning which is published as a /geometry_msgs/PoseStamped on the topic /mavros/local_position/pose.
The flight controller's local position is either relative to the MAV's starting location (in case of using GPS) or relative to the RTK base station (in case of using RTK GPS).
You need to set the following settings via QGroundControl in Vehicle Setup/Parameters/EKF2 (this determines, which data is used for the flight controller's state estimation):
EKF2_AID_MASK: 1 (only tick use GPS)
EKF2_HGT_MODE: GPS
EKF2_MAG_TYPE: Automatic
and set these parameters according to your drone's geometry (offset between flight controller and GPS antenna):
EKF2_GPS_POS_X, EKF2_GPS_POS_Y, EKF2_GPS_POS_Z
A guide on external position estimates is available here. For Visual-Inertial Odometry (VIO) we need an appropriate sensor connected to the NUC and VIO/SLAM running on the drone which can output position estimates at a rate above 30 Hz (if the rate falls below this threshold the flight controller will ignore the messages). Forwarding the VIO estimate to the flight controller makes planning a lot easier, since we know that the flight controller and planner are using the same coordinate frame. In this setup, the world origin will be dictated by the VIO/SLAM system.
You need to set the following settings via QGroundControl in Vehicle Setup/Parameters/EKF2 (this determines, which data is used for the flight controller's state estimation):
EKF2_AID_MASK: 280 (tick vision position fusion, vision velocity fusion, and vision yaw fusion)
EKF2_HGT_MODE: Vision
EKF2_MAG_TYPE: None
and set these parameters according to your drone's geometry (offset between flight controller and the VO frame):
EKF2_EV_POS_X, EKF2_EV_POS_Y, EKF2_EV_POS_Z
Example code for a ROS node that publishes commands can be found here. A couple of important points to note:
- The flight controller needs to be in Offboard mode to accept any position, velocity, or trajectory commands.
- The command rate must be above 2 Hz. If the flight controller detects a timeout of more than 500 ms (default value) between consecutive commands it will fall out of Offboard mode.
- You can either switch manually into Offboard mode using the remote, or switch by sending the appropriate command via mavros (as in the linked example above).
- The flight controller will only switch to Offboard mode, if you are already streaming commands at a minimum rate of 2 Hz.
Whenever any of these pre-conditions for offboard control fail, the drone will fall back into the last flight mode used before entering offboard mode.
Here we list a few useful software tools with a short description and pitfalls if applicable:
Kalibr is a powerful calibration suite. We use it to calibrate camera intrinsics as well as camera-to-camera calibration, and camera-to-imu calibration.
Image Undistort can rectify images and create 3D pointclouds from stereo images and can be used as an alternative to the image_proc ROS package. A modified version with minor improvements is forked here.
VINS-Fusion is a stereo visual-inertial state estimator suitable for autonomous flight. A modified version with a few ease of use improvements is forked here.
Since a typical camera frame rate of 20 Hz is below the minimum rate input for the flight controller's vision input we cannot forward the usual output pose from VINS. We need to remap the imu_propagate topic, which is published at the same rate as the sensors IMU, as the input for mavros.
Voxblox is great for online 3D mapping and planning. Voxblox builds a global euclidean singed distance field from input pointclouds and a state estimate.
PX4 Flight Review is an online tool to inspect flight logs from the flight controller. You can download flight logs from the flight controller over USB or the telemetry connection and visualize the data to analyize a flight.