SubdSH : subdivision-based spherical harmonics field for real-time shading-based refinement under challenging unknown illumination

Abstract

This paper presents a spatial-varying illumination model for shading-based depth refinement that based on a smooth Spherical Harmonics (SH) lighting field. The proposed lighting model is able to recover shading under challenging unknown lighting conditions, thus improving the quality of recovered surface detail. To avoid over-parameterization, local lighting coefficients are treated as a vector-valued function which is represented by subdivided surfaces using Catmull-Clark subdivision. We solve our lighting model utilizing a highly parallelized scheme that recovers lighting in a few milliseconds. A real-time shading-based depth recovery system is implemented with the integration of our proposed lighting model. We conduct quantitative and qualitative evaluations on both synthetic and real world datasets under challenging illumination. The experimental results show our method outperforms the state-of-the-art real-time shading-based depth refinement system.