Local, global and segment-based stereo matching

Abstract

Stereo matching involves the estimation of depth from two or more images of the same scene taken from different view-points. It finds applications in areas such as dense 3D reconstruction of scenes and object models, depth image based rendering, autonomous robot navigation, satellite imaging and automated cartography. This dissertation explores three classes of algorithms for stereo matching, the first step in recovering depth from stereo. The three classes are local, global and segment-based algorithms, each one outperforming the ones preceding it. The main theme of local matching techniques is presented, an implementation is described and a constraint to keep a check on false matches is illustrated. The basic concepts of global energy minimization in computer vision are presented before describing three global optimization algorithms in the context of stereo matching - graph-cuts, belief propagation and dynamic programming. A general structure of the segment-based approaches is developed and the related work is reviewed. A complete software implementation is developed and some of the issues involved are discussed. 3D point-cloud plots from practical experiments are used to illustrate the main steps of segment-based techniques. For each class of algorithms, experimental results are presented and their qualitative and quantitative evaluation is performed.