Deep Learning in Computer Vision: Studies in Neuro-image Segmentation and Satellite Image Super-resolution

Abstract

Single image super-resolution (SR) has been a topic of great interest in the computer vision and deep learning community and has found applications in many areas including quality enhancement of satellite images. As the cost of satellite images primarily depends on the sensor quality, super-resolving satellite images captured at a low resolution may substantially reduce the price of image acquisition. However, none of the existing deep CNN based satellite image super resolution techniques takes region-level context information into account and gives equal importance to each image region. Satellite images are typically of very large dimensions and salient object regions often occupy a small portion of the same. This, along with the fact that most state-of-the-art SR methods are complex and cumbersome deep models, the time taken to process very large satellite images can be impractically high. These observations motivate us to propose a context-aware SR pipeline for satellite images. Specifically, in the first work, We, propose to handle this challenge by designing an SR framework that analyzes the regional information content on each patch of the low-resolution image and judiciously chooses to use more computationally complex deep models to super-resolve more structure-rich regions on the image, while using less resource-intensive non-deep methods on non-salient regions. Through extensive experiments on a large satellite image, we show substantial decrease in inference ime while achieving similar performance to that of existing deep SR methods over several evaluation measures like PSNR, MSE and SSIM. Finally, as a direct improvement above this work, we propose a switch-guided hybrid network that is trained to selectively super-resolve salient regions using a deep CNN model and non-salient/background regions via a lightweight SR method such as bi-cubic interpolation. Through experiments on the SpaceNet dataset, we study how the proposed switched SR framework can maintain a balance between computational cost and improvement in image quality.