Enhancing road object detection in rainy conditions with generative adversarial networks

Abstract

With the development of autonomous vehicle driving, object detection plays a great role in the application. Thus, the effectiveness and stability of object detector are significant. However, under some bad weather like rain, object detection system relied on camera images does not work well because raindrops would decrease the contrast between objects and background, which makes objects hard to recognize. Hence, improving the performance of object detector under rainy condition is urgently needed to conduct.

To solve this issue, much research is present. Some experiences showed that data augmentation is a good method. Combination image data and some sensors data is used for image enhancement under rainy conditions. Sensor’s data could decrease the influence of raindrops to a certain extent. In addition, some generative models are used for removing the raindrops by transferring rainy images to clear weather images. Some existing methods mentioned above could show their potential under rainy conditions.

This study investigates the use of this model for enhancing object detection performance in rainy conditions. A generative adversarial network is introduced in this project to generate object enhancement so that increase the object awareness of detector. The model will improve the performance of detector under rainy condition, which make sense in the development of autonomous vehicle driving. After the comparison between unenhanced and enhanced detectors F1 score, there is 17% improvement which is satisfied. Besides, mAP of enhanced detector is 13% better than that of unenhanced detector. In the future, the robustness of generative models could be improved to adopt to more situations. And the data used for generative models training could be also adjusted such as highlight objects outline and locate objects by any other features.