Multi-robot semantic mapping based on deep learning framework

Abstract

In this work, we mathematically model the collaborative semantic perception problem with derive its probability decomposition, based on which we propose a novel incremental hierarchical collaborative probabilistic semantic mapping framework where a multi-modal sensor data fusion method is utilized to incorporate information of sensors, a Bayesian based update scheme is employed to update the current state with prior knowledge and current observations, an Expectation-Maximization approach is exploited to infer the hidden data association between local maps.

Experiments conducted both on open source dataset and real robot platforms show the respective 3D semantic reconstruction generated by our proposed system both in single robot local mapping level and multi-robot global mapping level. These results demonstrate the accuracy and utility of our system and indicates the potential of this system to be used in the real robot application.