OLIP-MIF: an improved method for object localization and intention prediction based on multimodal information fusion

Abstract

3D object localization and intention prediction have become crucial components in autonomous system applications, such as self-driving car. However, there still faces a lot of challenges, especially for complex and dynamic scenarios where a single modality information is insufficient to effectively and precisely localize the position and analyze the intention of objects. An improved method based on multimodal information fusion has been proposed via leveraging the advantages of 2D image segmentation and 3D geometrical characteristics of LiDAR point cloud. Extensive comparative experiments have been conducted and the results demonstrate that the proposed method significantly enhances both localization and prediction accuracy, comparing with the method where 2D bounding box of object instead of segmentation information is used to be fused with point cloud.