Robust day and night object detection based on heterogeneous sensors and information fusion

Abstract

Object detection and localization is now an important component in autonomous driving-related applications, in which the technology based on traditional RGB cameras has become increasingly mature. However, the detection ability of RGB cameras is greatly affected by lighting conditions, such as in a dim environment at night, the information available in RGB images may not be rich enough. We find that thermal infrared images and 3D point clouds from LiDAR can make up for the lack of light and capture more information missing from visible light images. Therefore, we propose a method to fuse RGB images, thermal images and 3D point clouds to facilitate accurate detection in both day and night. Experimental results show that this fusion method improves the detection performance.