Please use this identifier to cite or link to this item: https://hdl.handle.net/10356/182305
Title: MoDA: modeling deformable 3D objects from casual videos
Authors: Song, Chaoyue
Wei, Jiacheng
Chen, Tianyi
Chen, Yiwen
Foo, Chuan-Sheng
Liu, Fayao
Lin, Guosheng
Keywords: Computer and Information Science
Issue Date: 2024
Source: Song, C., Wei, J., Chen, T., Chen, Y., Foo, C., Liu, F. & Lin, G. (2024). MoDA: modeling deformable 3D objects from casual videos. International Journal of Computer Vision. https://dx.doi.org/10.1007/s11263-024-02310-5
Project: M23L7b0021 
Journal: International Journal of Computer Vision
Abstract: In this paper, we focus on the challenges of modeling deformable 3D objects from casual videos. With the popularity of NeRF, many works extend it to dynamic scenes with a canonical NeRF and a deformation model that achieves 3D point transformation between the observation space and the canonical space. Recent works rely on linear blend skinning (LBS) to achieve the canonical-observation transformation. However, the linearly weighted combination of rigid transformation matrices is not guaranteed to be rigid. As a matter of fact, unexpected scale and shear factors often appear. In practice, using LBS as the deformation model can always lead to skin-collapsing artifacts for bending or twisting motions. To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts. To register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space, and 2D image features by solving an optimal transport problem. Besides, we introduce a texture filtering approach for texture rendering that effectively minimizes the impact of noisy colors outside target deformable objects.
URI: https://hdl.handle.net/10356/182305
ISSN: 0920-5691
DOI: 10.1007/s11263-024-02310-5
Schools: College of Computing and Data Science 
Organisations: Institute for Infocomm Research, A*STAR
Rights: © 2024 The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature. All rights reserved.
Fulltext Permission: none
Fulltext Availability: No Fulltext
Appears in Collections:CCDS Journal Articles

SCOPUSTM   
Citations 50

1
Updated on Feb 13, 2025

Page view(s)

28
Updated on Feb 16, 2025

Google ScholarTM

Check

Altmetric


Plumx

Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.