Please use this identifier to cite or link to this item:
https://hdl.handle.net/10356/182305
Title: | MoDA: modeling deformable 3D objects from casual videos | Authors: | Song, Chaoyue Wei, Jiacheng Chen, Tianyi Chen, Yiwen Foo, Chuan-Sheng Liu, Fayao Lin, Guosheng |
Keywords: | Computer and Information Science | Issue Date: | 2024 | Source: | Song, C., Wei, J., Chen, T., Chen, Y., Foo, C., Liu, F. & Lin, G. (2024). MoDA: modeling deformable 3D objects from casual videos. International Journal of Computer Vision. https://dx.doi.org/10.1007/s11263-024-02310-5 | Project: | M23L7b0021 | Journal: | International Journal of Computer Vision | Abstract: | In this paper, we focus on the challenges of modeling deformable 3D objects from casual videos. With the popularity of NeRF, many works extend it to dynamic scenes with a canonical NeRF and a deformation model that achieves 3D point transformation between the observation space and the canonical space. Recent works rely on linear blend skinning (LBS) to achieve the canonical-observation transformation. However, the linearly weighted combination of rigid transformation matrices is not guaranteed to be rigid. As a matter of fact, unexpected scale and shear factors often appear. In practice, using LBS as the deformation model can always lead to skin-collapsing artifacts for bending or twisting motions. To solve this problem, we propose neural dual quaternion blend skinning (NeuDBS) to achieve 3D point deformation, which can perform rigid transformation without skin-collapsing artifacts. To register 2D pixels across different frames, we establish a correspondence between canonical feature embeddings that encodes 3D points within the canonical space, and 2D image features by solving an optimal transport problem. Besides, we introduce a texture filtering approach for texture rendering that effectively minimizes the impact of noisy colors outside target deformable objects. | URI: | https://hdl.handle.net/10356/182305 | ISSN: | 0920-5691 | DOI: | 10.1007/s11263-024-02310-5 | Schools: | College of Computing and Data Science | Organisations: | Institute for Infocomm Research, A*STAR | Rights: | © 2024 The Author(s), under exclusive licence to Springer Science+Business Media, LLC, part of Springer Nature. All rights reserved. | Fulltext Permission: | none | Fulltext Availability: | No Fulltext |
Appears in Collections: | CCDS Journal Articles |
SCOPUSTM
Citations
50
1
Updated on Feb 13, 2025
Page view(s)
28
Updated on Feb 16, 2025
Google ScholarTM
Check
Altmetric
Items in DR-NTU are protected by copyright, with all rights reserved, unless otherwise indicated.