Adversarial cross-modal unsupervised domain adaptation in semantic segmentation

Abstract

3D semantic segmentation is a vital problem in automatic driving, and thus a hot field in deep learning. These days, the research for unsupervised domain adaptation rises for solving the problem of lacking annotated datasets. However, the research on 3D UDA in semantic segmentation is still a blue sea. Our research aims to combine adversarial learning and cross-modal networks to boost the performance of 3D UDA across datasets in semantic segmentation. With this goal, we propose a new solution based on xMUDA and ADVENT, research several detailed change in this novel network and obtain better 3D and overall performances. In this dissertation, we use independent discriminators on cross-modal UDA networks. Firstly, we add uni-modal ones and get our best solution, which has a 3D mIoU 7.53% higher than the baseline and an improvement of overall performance by 3.68%. Then, we add two more cross-modal discriminators but the performance suffers a decrease. However, the performance is still better than the baseline. To research on the priority between MaxSqaureLoss and cross-modal loss in our aiming task, we design a pair of experiments and find cross-modal method act better in such tasks. Finally, in terms of the over-fitting issue occurring in both baseline and our method, we give our thoughts about the cause.