Perceptually based parameter adjustments for video processing operations

ACM SIGGRAPH Talks, 2014
 
Gabriel Eilertsen, Jonas Unger, Robert Wanat, Rafal Mantiuk
C-Research, Media and Information Technology, Linköping University
Bangor University
 

Figure: Top row shows a non-linear filtering of an image, i.e. the result does not vary linearly with the parameter changes. Bottom row shows a perceptually linearized version of the same filter result as one parameter is varied.
 
Abstract:
 
Extensive post processing plays a central role in modern video production pipelines. A problem in this context is that many filters and processing operators are very sensitive to parameter settings and that the filter responses in most cases are highly non-linear. Since there is no general solution for performing perceptual calibration of image and video operators automatically,it is often necessary to manually perform tweaking of multiple parameters. This is an iterative process which requires instant visual feedback of the result in both the spatial and temporal domains. Due to large filter kernels, computational complexity, high frame rate, and image resolution it is, however, often very time consuming to iteratively re-process and tweak long video sequences. We present a new method for rapidly finding the perceptual minima in high-dimensional parameter spaces of general video operators. The key idea of our algorithm is that the characteristics of an operator can be accurately described by interpolating between a small set of pre-computed parameter settings. By computing a perceptual linearization of the parameter space of a video operator, the user can explore this interpolated space to find the best set of parameters in a robust way. Since many operators are dependent on two or more parameters, we formulate this as a general optimization problem where we let the objective function be determined by the user’s image assessments. To demonstrate the usefulness of our approach we show a set of use cases (see the supplementary material) where our algorithm is applied to computationally expensive video operations.
 
Documents:
Paper: Talk abstract (.pdf) (1MB)
 
 
Bibtex
@inproceedings{Eilertsen720373,
author = {Eilertsen, Gabriel and Unger, Jonas and Wanat, Robert and Mantiuk, Rafal},
booktitle = {ACM SIGGRAPH 2014 Talks},
institution = {Link{\"o}ping University, Media and Information Technology},
institution = {Bangor University, United Kingdom},
title = {Perceptually based parameter adjustments for video processing operations},
location = {Vancouver, Canada},
month ={August},
year = {2014}
}