PhD defence of Andrei Chubarau – Vision Transformers for Image Quality Assessment on High Dynamic Range Displays

Abstract

Image Quality Assessment (IQA) plays a critical role in optimizing visual experiences by approximating human perception of image quality. However, existing IQA methods often fail to generalize across variations in illumination, display properties, and content, which are all too common in real-world scenarios involving modern display technologies. In this thesis, we address these challenges through a series of contributions spanning perceptual studies and applications of deep learning for IQA. First, we conduct a subjective experiment to quantify the influence of ambient illumination on human perception of image quality. To complement these findings, we introduce a framework that extends the applicability of existing IQA methods to a wider range of illumination and display parameters, effectively modeling viewing conditions from complete darkness to bright daylight. Next, we explore the application of vision transformers (ViTs) for IQA, analyzing the feature representations of various pre-trained ViTs to identify architectures better suited for IQA and to gain insights into how these models encode image quality distortions. Building on this analysis, we introduce Vision Transformer for Attention Modulated Image Quality (VTAMIQ), a novel full-reference IQA model that leverages ViTs to capture global dependencies in images and achieves state-of-the-art performance on standard IQA datasets. Finally, while most existing IQA methods and datasets are tailored for Standard Dynamic Range (SDR) imaging, we address the challenges of training deep IQA models on High Dynamic Range (HDR) data by integrating specialized fine-tuning and domain adaptation techniques. Models trained with our approach surpass previous baselines, converge significantly faster, and reliably generalize to HDR inputs. Altogether, our findings offer valuable insights into how viewing conditions influence human perception of image quality and support the development of more robust and generalizable IQA models, enhancing their adaptability and performance in real-world applications.