Hybrid Vision Transformer and Convolutional Neural Network for Super-Resolution Image Quality Assessment
Abstract
With the development of image super-resolution (SR) algorithms, which enhance the resolution and visual quality of images, it has become increasingly important to accurately evaluate the quality of the generated SR images. Full reference (FR) metrics such as SSIM and PSNR have long been used for SR evaluation but have significant limitations. Their reliability is uncertain and a major practical draw-back lies in the dependence on ground-truth images, which are not always available in real-world scenarios. This work focuses on no-reference super-resolution image quality assessment (NR-SRIQA), which evaluates SR images without LR or HR references. In this paper, we propose a novel model to address the SRIQA task by leveraging a hybrid approach that benefits from Vision Transformers (ViTs) and Convolutional Neural Networks (CNNs) to extract both local and nonlocal features from the input SR image …
本文针对超分辨率图像质量评估（SRIQA）任务，提出一种结合视觉Transformer（ViT）与卷积神经网络（CNN）的混合无参考评估模型。该方法利用ViT提取非局部特征，并通过多阶段自注意力处理图像令牌，再将其重塑为特征图，由CNN编码映射为质量分数，从而同时捕捉局部与非局部信息，克服了传统全参考指标依赖真实图像且可靠性不足的问题。