Please use this identifier to cite or link to this item:
Title: Precise region semantics-assisted GAN for pose-guided person image generation
Authors: Liu, Ji
Weng, Zhenyu
Zhu, Yuesheng
Keywords: Engineering::Electrical and electronic engineering
Issue Date: 2023
Source: Liu, J., Weng, Z. & Zhu, Y. (2023). Precise region semantics-assisted GAN for pose-guided person image generation. CAAI Transactions On Intelligence Technology, 1-14.
Journal: CAAI Transactions on Intelligence Technology 
Abstract: Generating a realistic person's image from one source pose conditioned on another different target pose is a promising computer vision task. The previous mainstream methods mainly focus on exploring the transformation relationship between the keypoint-based source pose and the target pose, but rarely investigate the region-based human semantic information. Some current methods that adopt the parsing map neither consider the precise local pose-semantic matching issues nor the correspondence between two different poses. In this study, a Region Semantics-Assisted Generative Adversarial Network (RSA-GAN) is proposed for the pose-guided person image generation task. In particular, a regional pose-guided semantic fusion module is first developed to solve the imprecise match issue between the semantic parsing map from a certain source image and the corresponding keypoints in the source pose. To well align the style of the human in the source image with the target pose, a pose correspondence guided style injection module is designed to learn the correspondence between the source pose and the target pose. In addition, one gated depth-wise convolutional cross-attention based style integration module is proposed to distribute the well-aligned coarse style information together with the precisely matched pose-guided semantic information towards the target pose. The experimental results indicate that the proposed RSA-GAN achieves a 23% reduction in LPIPS compared to the method without using the semantic maps and a 6.9% reduction in FID for the method with semantic maps, respectively, and also shows higher realistic qualitative results.
ISSN: 2468-2322
DOI: 10.1049/cit2.12255
Schools: School of Electrical and Electronic Engineering 
Rights: © 2023 The Authors. CAAI Transactions on Intelligence Technology published by John Wiley & Sons Ltd on behalf of The Institution of Engineering and Technology and Chongqing University of Technology. This is an open access article under the terms of the Creative Commons Attribution‐NonCommercial‐NoDerivs License, which permits use and distribution in any medium, provided the original work is properly cited, the use is non‐commercial and no modifications or adaptations are made.
Fulltext Permission: open
Fulltext Availability: With Fulltext
Appears in Collections:EEE Journal Articles

Citations 50

Updated on Jun 22, 2024

Page view(s)

Updated on Jun 21, 2024


Updated on Jun 21, 2024

Google ScholarTM




Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.