Perceptual Quality Assessment of Lowlight Restored and Authentically Distorted Images

Abstract

The capability of hand-held devices to acquire high-definition visual content has led to a tremendous increase in the number of images and videos captured daily. However, camera hardware and pipelines are not perfect and lead to multiple distortions in the captured content. This makes quality assessment (QA) imperative to advance the qualitative capability of different devices and the pipelines used. More particularly, the aim of perceptual quality assessment is to quantitatively analyze the perceptual quality of the captured content with respect to the distortions observed by the human visual system. This thesis focuses on two aspects of perceptual quality assessment. Firstly, we focus on the subjective and objective quality assessment of low-light restored images. Then we consider the problem of unsupervised quality assessment methods for authentically distorted images.

The quality assessment of restored low-light images is an important tool for benchmarking and improving low-light restoration (LLR) algorithms. While several LLR algorithms exist, the subjective perception of the restored images has been much less studied. Challenges in capturing aligned low-light and well-lit image pairs and collecting a large number of human opinion scores of quality for training warrant the design of unsupervised (or opinion unaware) no-reference (NR) QA methods. In this part, we study the subjective perception of low-light restored images and their unsupervised NR QA. Our contributions are two-fold. We first create a dataset of restored low-light images using various LLR methods, conduct a subjective QA study, and benchmark the performance of existing QA methods. The lack of good perceptual quality metrics designed explicitly for the low-light scenario is an important limitation in advancing the design of restoration methods. To tackle this, we present a self-supervised contrastive learning technique to extract distortion-aware features from the restored low-light images. We show that these features can be effectively used to build an opinion unaware image quality analyzer. Detailed experiments reveal that our unsupervised NR QA model achieves state-of-the-art performance among all such quality measures for low-light restored images.

The quality assessment of camera captured authentically distorted images is challenging due to the lack of a reference. While there is a plethora of supervised no reference image QA algorithms, there is a need to study unsupervised or opinion unaware algorithms based on their superior generalization performance. We explore self-supervised learning (SSL) for the feature design on authentically distorted images to predict quality without training on human labels. While SSL on synthetic distortions has recently shown promise, there is a need to enrich the feature learning on authentic distortions. We propose a novel two-stage learning approach on synthetic and authentically distorted images with different learning methodologies. We perform contrastive learning with positives and negatives that vary with quality on synthetic data to capture quality features. While learning on authentically distorted images, we only consider positives due to the difficulty in obtaining negatives that vary in quality alone. We employ the SimSiam framework to enrich features by fine-tuning on authentically distorted images. We show that the self-supervised features we learn can be used to make perceptually consistent image quality predictions on authentically distorted images without training on any human opinion scores. We achieve state-of-the-art performance on multiple authentically distorted datasets without training on them.