Improving unpaired image translation with CUT and lighting-adapted data augmentation

Abstract

Unpaired image-to-image translation has received extensive attention in computer vision due to its significant applications in various practical scenarios, such as human-computer interaction and robotic automation. This research addresses the task that translating human hand-grasping images into corresponding robotic arm-grasping images. Among existing unpaired image translation frameworks, CycleGAN and CUT are two prominent methodologies with unique strengths and weaknesses. CycleGAN is capable of bidirectional translation but suffers from instability, high computational costs, and detailed feature loss. In contrast, CUT achieves higher efficiency and better preservation of local features due to its single-direction translation mechanism and contrastive learning approach. However, CUT still faces performance degradation under variable lighting conditions. To overcome the lighting sensitivity issue of CUT, this dissertation introduces a Curve-based data augmentation strategy. By pre-processing various lighting scenarios in the training dataset, the proposed Curve enhancement significantly improves the robustness and generalization ability of the CUT model. Experimental results clearly demonstrate that the CUT model combined with Curve enhancement provides substantial improvements in terms of image quality, feature preservation, and stability under diverse lighting conditions.