Denoising and Refinement Methods for 3D Reconstruction

Abstract

Capturing raw 3D data from the real world is the initial step for many 3D reconstruction pipelines in different computer vision applications. However, due to inaccuracies in measurement and oversimplification in mathematical assumptions during capture, 3D data remain contaminated with significant amounts of errors. In this thesis, we investigate two different types of errors that are invariably encountered in 3D reconstruction. The first type of errors comprises the random measurement error or noise that is inevitably present in raw 3D data obtained from the real world. The second type of errors comprises those that are geometrically-structured. Specifically, we consider non-rigid alignment errors that arise in multi-view scenarios where complete 3D reconstructions of real-world objects are obtained from observations taken from multiple viewpoints. Firstly, we consider random measurement errors, modelled as an additive 3D Gaussian noise. We consider the task of denoising 3D data obtained in two different modalities, i.e. 3D meshes and 3D point clouds and establish the important factors that dictate the quality of denoising. We develop denoising schemes that account for these factors and provide evidence of superior denoising performance on synthetic and real datasets over existing approaches. Secondly, we consider non-rigid errors that are encountered in a multi-view 3D reconstruction pipeline. In particular, we address the problem of multi-view surface refinement for high quality 3D reconstruction, where low quality reconstructions obtained from consumer-grade depth cameras are enhanced using additional photometric information. We show that non-rigid estimation discrepancies that emerge in such tasks are a major issue limiting the quality of reconstruction. We systematically delineate the underlying factors and show that existing refinement methods in the literature do not consider these factors, hence, failing to carry out a proper refinement. Considering these factors, we develop a complete multi-view pipeline for high quality 3D reconstruction. We show the efficacy of our pipeline on synthetic and real datasets, as compared to other existing methods.