A deep learning framework for object detection under rainy conditions

Abstract

Adhesive raindrops on glass have been known to diffract light and distort parts of the scene behind them. In the modern days object detection applications, these raindrops pose as a nuisance since they hamper the detectability of the objects in a scene. As a result, much more effort has been placed on image enhancement and de-raining algorithms in the recent years. Unsupervised image – to – image translational networks are a form of deep learning models based on Generative Adversarial Networks which is also known as GANs. The advantage of the Generative Adversarial Networks is the ability to learn mathematical functions that can map one domain of data to another, this ability has been adopted in image enhancement and domain adaptations applications with great success. However, when an image has several target instances, the translation process involves considerable shape changes. The objective of this project is to improve the performance of current object detections under rainy weather conditions by using unsupervised image to image translation networks for deraining purposes. Among the various types of generative adversarial network that are available, this project will mainly focus the used of CycleGAN for de-raining. Object detection classifiers such as the single shot detector classifier and the Faster R-CNN classifier will be used for the classification of the vehicles. The mention method was used to conduct de-raining for rainy images and the output of the image was fed to the two different classifiers. The detection accuracy for vehicles after conducting de-raining is better as compared to the detection before de-raining. By comparing the two different classifier models, the Faster R-CNN model has a better detection accuracy as compared to the SSD model, as the SSD model is unable to detect vehicles that are further away. This project shows that the CycleGAN is able to conduct de-raining on the rainy image however more improvements can be made as the contextual loss in the background is not a desirable effect from the CycleGAN model. To further experiment a method to conduct deraining, would be using another generative adversarial network known as InstaGAN. InstaGAN is able to improve the de-raining process as it is able to preserve the loss that encourages the network to learn the identity function from the target instances.