Table of Contents

• 1. Overview
  • System Structure
• 2. Software
  • MrExplorer
  • AutoMerge
  • BioSLAM
• 3. Multi-Agent Exploration
  • Run MrExplorer
  • FakeMerge
• 4. Multi-Agent Localization
  • Visualization
• 5. Lifelong Localization
  • Configure Datasets
• 6. TODO List

1. Overview

In traditional Multi-Agent Exploration (MAE) methods, all agents start in the same place and start with a shared coordinate system; and once the relative transformation missed due to localization failures, the MAE system will be very fregial and easily crash. This framework provide a novel Autonomous Exploration paradigm, all the agents can start in the darkness, without knowing anything about where they are, where each other are, and what the environment looks like. As the MAE system explorer individually, MetaUnion helps to find associated overlaps among different agents and solve the above three challenges gradually; and most importantly, MetaUnion can incrementally update the feature learning ability for new observed areas. In a summary, our key contributions includes:

• Multi-Agent Localiztion without Initial Transformation;

• Multi-Agent Hierarchical Exploration;

• Incremental Lifelong Localization.

System Structure

MetaUnion mainly contains three key modules within the framework: AutoMerge, MrExplorer, and BioSLAM. In the begining, each agent is randomly assigned at a new envionrment, and has neither information about other agents' localization, nor place recognition ability for the new areas, and conduct single-agent exploration individually. When the trajectries of different agents have overlaps, AutoMerge module can automatically detect the data association between agents, and estimate the relative positions for them. Based on the relative transformation, MrExplorer can divide the agent into different groups based on their connections, and apply hierarichal exploration for individual agent. Parallel with the above procedure, BioSLAM constructs a dual-memory system to memorize the large-scale and long-duration place features. As the MAS system explorer to bigger area, their cooperation ability and place recognition ability are improved gradually.

2. Software

To quickly install all the necessary softwares for MetaUnion, please following the instructions,

git clone --recurse-submodules -j8 https://github.com/MetaSLAM/MetaUnion.git
# On Server
sh scripts/install_server.sh
## On Client
sh scripts/install_client.sh
source ~/.bashrc

Please keep in mind, the scripts will automatically generate PATH into the bashrc, and you can use

METASLAM=/home/maxtom/codespace/MetaUnion
source /home/maxtom/codespace/MetaUnion/stack/devel/setup.bash
# on client
alias enclient='sh /home/maxtom/codespace/MetaUnion/scripts/start_client.sh'
# on server
alias enserver='sh /home/maxtom/codespace/MetaUnion/scripts/start_server.sh'
# general command
alias cdw='cd /home/maxtom/codespace/MetaUnion'
alias killmeta='sh /home/maxtom/codespace/MetaUnion/scripts/killall.sh'

In the following sections, we will investigate the details of each module.

MrExplorer

Since all agents begin from unknown darkness area within an given map, each agent are starting from a local exploration policy based on its observations. And based on the AutoMerge module, small group agents will find their relative transformations, then such agents are grouped into one set, and the single agent exploration policy are transfermed into multi-agent exploration policy. Gradually, all the agents are converted into one group. This hierarchical strategy help crowd agents quickly understand their coopeartion status with each others, and can balance the exploration efficiency and robustness without any initial position and relative transformation information.

Note	API: Input and Output Odometry for each agent Grouping status from AutoMerge Waypoints for each agent

AutoMerge

AutoMerge can provide an automatic data-association for agents without any knowledge about initial position and each others. When agents upload the observations to the cloud, the agents can detect their associated overlaps (if exists), and send back the grouping results among different agent groups.

Note	API: Input and Output Odometry from individual agents SubMaps from individual agents Grouping Indexes Relative Transformations

Note	TODO List Onine SubMap Generation Onine Feature Evaluation Onine Cache Checking Onine Position Estimation

BioSLAM

We present BioSLAM, a lifelong SLAM framework for learning various new appearances incrementally and maintaining accurate place recognition for previously visited areas.Unlike humans, artificial neural networks suffer from catastrophic forgetting and may forget the previously visited areas when trained with new arrivals. For humans, researchers discover that there exists a memory replay mechanism in the brain to keep the neuron active for previous events. Inspired by this discovery, BioSLAM designs a gated generative replay to control the robot’s learning behavior based on the feedback rewards.Specifically, BioSLAM provides a novel dual-memory mechanism for maintenance: 1) a dynamic memory to efficiently learn new observations and 2) a static memory to balance new-old knowledge. When combined with a visual-/LiDAR- based SLAM system, the complete processing pipeline can help the agent incrementally update the place recognition ability, robust to the increasing complexity of long-term place recognition.

Note	API: Input and Output Dual Memory Assistance Long-short Memorization Regular Updated Models

Note	TODO List Online Feature Updating Strategy Selection Group Waypoints Estimation

3. Multi-Agent Exploration

Run MrExplorer

cdw
enserver #on server
enclient #on client

Note	TODO List Change the name of the project Integrate the configuration of the client into the config file Debug the global planning Integrate the multi-master into this

FakeMerge

git clone https://github.com/MetaSLAM/fakemerge.git fakemerge
cd fakemerge
catkin_make
source devel/setup.bash
roslaunch fakemerge transformPose.launch

4. Multi-Agent Localization

DATA:
    OFFLINE_LENGTH: 100 # Set for desire testing length

Use OFFLINE_LENGTH to set the length for each agent.

python src/offline_merging.py

In the current global_rough_align step, we will use spectral clustering method to divide agents into different groups based on their connections.

Visualization

Outside the docker, subscibe /global_map with world frame with rviz. Different un-merged maps will be visualized along the z-axis (index*30).

5. Lifelong Localization

See the demo tutorial in the jupyter and python version.

Configure Datasets

Download Pittsburgh datasets for online map merging. https://drive.google.com/drive/folders/19AK8jc6yZpKN6Ub_ILGJaceZixpbwKcV?usp=sharing

Download pre-trained models and set path

cd data && sh download.sh
echo "export ROS_IP='172.17.0.1' " >> ~/.bashrc
echo "export MASTER_IP='172.17.0.2' " >> ~/.bashrc
echo "export ROS_MASTER_URI=http://$MASTER_IP:11311/ " >> ~/.bashrc
echo "export BAG_PATH='$PATH_TO_PITT_ROS_BAGS' " >> ~/.bashrc
source ~/.bashrc

And the following difference matrix in the data/results/. Then set ROS_IP to enable communication between docker and host computer. Finally, export dataset path, and replace PATH_TO_PITT_ROS_BAGS to the Pittusbrugh rosbags.

6. TODO List