Please use this identifier to cite or link to this item:
Title: GP-UNIT: generative prior for versatile unsupervised image-to-image translation
Authors: Yang, Shuai
Jiang, Liming
Liu, Ziwei
Loy, Chen Change
Keywords: Engineering::Computer science and engineering
Issue Date: 2023
Source: Yang, S., Jiang, L., Liu, Z. & Loy, C. C. (2023). GP-UNIT: generative prior for versatile unsupervised image-to-image translation. IEEE Transactions On Pattern Analysis and Machine Intelligence, 45(10), 11869-11883.
Project: T2EP20221-0011
Journal: IEEE Transactions on Pattern Analysis and Machine Intelligence
Abstract: Recent advances in deep learning have witnessed many successful unsupervised image-to-image translation models that learn correspondences between two visual domains without paired data. However, it is still a great challenge to build robust mappings between various domains especially for those with drastic visual discrepancies. In this paper, we introduce a novel versatile framework, Generative Prior-guided UNsupervised Image-to-image Translation (GP-UNIT), that improves the quality, applicability and controllability of the existing translation models. The key idea of GP-UNIT is to distill the generative prior from pre-trained class-conditional GANs to build coarse-level cross-domain correspondences, and to apply the learned prior to adversarial translations to excavate fine-level correspondences. With the learned multi-level content correspondences, GP-UNIT is able to perform valid translations between both close domains and distant domains. For close domains, GP-UNIT can be conditioned on a parameter to determine the intensity of the content correspondences during translation, allowing users to balance between content and style consistency. For distant domains, semi-supervised learning is explored to guide GP-UNIT to discover accurate semantic correspondences that are hard to learn solely from the appearance. We validate the superiority of GP-UNIT over state-of-the-art translation models in robust, high-quality and diversified translations between various domains through extensive experiments.
ISSN: 0162-8828
DOI: 10.1109/TPAMI.2023.3284003
Schools: School of Computer Science and Engineering 
Research Centres: S-Lab
Rights: © 2023 IEEE. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:SCSE Journal Articles

Citations 50

Updated on Jul 13, 2024

Page view(s)

Updated on Jul 13, 2024

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.