On Applications of 3D-Warping and An Analysis of a RANSAC Heuristic

Abstract

In recent years communication of the scene geometry is gaining importance. With development of technologies such as head mounted displays and Augmented Reality (AR) the need for efficient 3D scene communication is becoming vital. Depth sensors are being incorporated into smartphones for large scale deployment of AR applications. 3D-communication require synchronous capture of the scene from multiple viewpoints along with depth for each view, known as Multiview Plus Depth (MVD) data. The number of views required depends on application. Traditionally, it has been assumed that devices are static but for smartphones such an assumption is not valid. The availability of depth modality opens up several possibilities for efficient MVD data compression. In this work we have leveraged depth for better RGB-D data compression and efficient depth estimation. Using the depth information, the RGB-D device motion can be accurately tracked. 3D-warping along with the camera tracking can then be used to generate reference frames to improve compression efficiency of motion vectors. The same mechanism can be used to predict depth in stereo disparity estimation problem. For robust tacking of the motion of camera array, we have used the Random Sample Consensus (RANSAC) algorithm. RANSAC is an iterative algorithm for robust model parameter estimation. A common practice among implementations of RANSAC is to take a few samples extra than the minimum required for estimation problem, but the implications of this heuristic is lacking in literature. We present a probabilistic analysis of this common heuristic. We also present a depth data coding algorithm by employing planar segmentation of depth. While all prior work based on this approach remained restricted to images only and under noise-free conditions, we present an efficient solution for noisy depth videos.