Kinect depth map enhancement

Abstract

Kinect is a depth sensor and it is getting popular among the researchers and amateurs due to its cheap cost and reasonable performance. Due to the limitations of the Kinect hardware, some of the depth information is missing and the resultant depth map is too noisy to be useful for many real-life applications. This paper presents two techniques to estimate the missing information. One way to estimate is by applying median filter to the neighbor pixels. Another one is to use segmentation on the RGB image and then estimate the missing information based on the RGB segmentation, based on the observation that nearby pixels with the similar color will have the same depth. The algorithms proposed are able to recover most of the missing information. The segmentation method is able to retain the original object shape while extrapolating and it takes 4.8 seconds to complete. With most of noise removed, one will be easier to identity the objects in the enhanced depth map and it will also be better suited for other applications to leverage the depth information.