Tutorial 8: Perceptual Metrics for Image Quality Evaluation
Presented by
Sheila Hemami, Cornell University and Thrasos Pappas, Northwestern University
Abstract
We will examine objective criteria for the evaluation of image quality that are based on models of visual perception. Our primary emphasis will be on image fidelity, i.e., how close an image is to a given original or reference image, but we will also discuss no-reference and limited-reference metrics. Our main focus will be on image and video compression and transmission. We will consider realistic distortions that arise from compression and error concealment in transmission over lossy channels.
We will start with a review of the human visual system, including physiology, function, and psychophysical approaches to characterization. We will examine both near-threshold perceptual metrics, which explicitly account for human visual system (HVS) sensitivity to noise by estimating thresholds above which the distortion is just-noticeable, and supra-threshold metrics, which attempt to quantify visible distortions encountered in high compression applications or when there are losses due to channel conditions. We will also consider structural similarity metrics, which model perception implicitly by taking into account the fact that the HVS is adapted for extracting structural information from images, and are thus insensitive to distortions (such as spatial and intensity shifts, contrast and scale changes) that do not change the structure of an image. We will also present a unified framework for perceptual and structural similarity metrics.
Most full-reference quality metrics compare the original image to a distorted image at the same resolution assuming fixed viewing conditions. However, due to the diversity of channel capacities and display devices that is typical of many applications, one may adapt the viewing distance and spatiotemporal resolution of the displayed signal in order to optimize perceived signal quality. We will discuss tradeoffs between resolution/viewing conditions and visibility of compression artifacts, and how they can be evaluated using subjective experiments or objective metrics.
Throughout the tutorial, we will compare and contrast performance of the metrics, including successes and failures. Besides traditional performances (e.g., correlation with subjective scores), we will demonstrate performances in-use, for example, performing R-D optimized compression with a perceptual metric rather than traditional MSE. We will discuss appropriate and inappropriate in-use applications for the various metrics and suggest guidelines for metric selection.
Outline:
- Models for compression/transmission image/video distortions
- Human visual system review
- Near-threshold metrics
- Supra-threshold metrics
- Structural Similarity metrics
- Evaluating image quality across resolutions
- Metric performance comparisons, selection and general use and abuse
