Hide menu

Real-time noise-aware tone mapping

Gabriel Eilertsen1   Rafał K. Mantiuk2,3   Jonas Unger1

1Linköping University, Sweden 2Bangor University, UK 3University of Cambridge, UK

 

Teaser

Video frame from [Fröhlich et al. 2014]

 

 

Abstract


Real-time high quality video tone-mapping is needed for many applications, such as digital viewfinders in cameras, display algorithms which adapt to ambient light, in-camera processing, rendering engines for video games and video post-processing. We propose a viable solution for these applications by designing a video tone mapping operator that controls the visibility of the noise, adapts to display and viewing environment, minimizes contrast distortions, preserves or enhances image details, and can be run in real-time on an incoming sequence without any preprocessing. To our knowledge, no existing solution offers all these features. Our novel contributions are: a fast procedure for computing local display-adaptive tone-curves which minimize contrast distortions, a fast method for detail enhancement free from ringing artifacts, and an integrated video tone-mapping solution combining all the above features.

 

 

Video


 

 

Downloads


 

Overview


We propose a new tone mapping operator (TMO) for video, which controls the visibility of the noise, adapts to the display and viewing environment, minimizes contrast distortions, preserves or enhances image details, and can be run in real-time on an incoming sequence without any preprocessing.

 

TMO overview

Diagram of our video tone-mapping solution. The input is split into a base and a detail layer using an edge preserving filter specifically designed for the purpose of detail extraction in the context of tone-mapping. The base layer is then compressed using a novel method for calculating locally adaptive tone curves that minimize contrast distortions and controls the visibility of image noise. The layers are then combined and converted with an inverse display model into pixel values before display. Click on image to enlarge.

 

As indicated in the figure above, our algorithm can be divided into three main parts, where each part contains novel key features that are central for the goals we set out to achieve:

 

 

Minimum contrast distortion tone curves

We propose a fast method for computing local display-adaptive tone curves. The tone curves minimize contrast distortions introduced by the mapping while controlling the visibility of noise, and their shapes are filtered over time to avoid temporal flickering.

 

Local tone-curve

Global tone-curve

Local tone-curve

Local tone-curves

Tone-mapping using global (left) or local (right) tone-curves. The image was generated assuming high ambient light viewing (low contrast) to better show contrast reduction. Note that other components of the operator, such as local contrast, were disabled for this example. Video frame from [Fröhlich et al. 2014]

 

 

Filter design for base – detail layer decomposition for tone mapping

The filter used for separating image details is constructed by analyzing current edge-preserving filters and their potential problems in the context of tone mapping. From the observations, and a unified formulation of bilateral filtering and anisotropic diffusion, we design a fast edge-stopping non-linear diffusion approximation for detail enhancement without ringing artifacts.

 

Bilateral filter

Bilateral filter

Anisotropic diffusion

Anisotropic diffusion

Non-linear diffusion

Our approach

 

The figures use the same tone curve, but different edge-preserving filters for detail extraction. The details are scaled by a factor 5 to highlight filtering differences and artifacts, and enlarged versions show the areas indicated by the red squares in the images. Video frame from [Fröhlich et al. 2014]

 

Fast detail extraction diffusion

Our approach

Guided filter, small kernel size

Guided filter, 5x5 kernel size

Guided filter, large kernel size

Guided filter, 10x10 kernel size

Comparison with the guided filter [He et al. 2013]. The guided filter shows no ringing artifacts, but either produces details on a too small scale (brick wall in middle image), or tend to show some haloing artifacts (fingers in right image). Video frame from [Fröhlich et al. 2014]

 

 

Noise-aware control over image details

In addition to the use of noise-aware tone curves adaptive to image noise, the visibility of noise is also controlled in the processed detail layer by taking into account the noise characteristics of the original base layer and the tone-mapped base layer. This is to allow the user to preserve or artistically enhance local contrast and details by scaling the detail layer, while still making sure that the noise is kept below the visibility threshold.

 

Noise-aware comparison

Comparison of tone-mapping disregarding noise (top left) with our noise-aware approach (top right), where excessive noise is hidden in darker tones. The bottom row shows the result of complementing the tone mapping with a denoising step (V-BM4D). Click on image to enlarge, for easier comparison of the visibility of noise. Video frame from [Fröhlich et al. 2014]

 

 

Citation


  • Eilertsen, Gabriel, Mantiuk, Rafal K., and Unger, Jonas. 2015. Real-time noise-aware tone mapping. ACM Transactions on Graphics 34(6) (Proceedings of ACM SIGGRAPH Asia 2015).
  • Bibtex:
@article{Eilertsen:2015,
  author = {Eilertsen, Gabriel and Mantiuk, Rafa\l K. and Unger, Jonas}, 
  title = {Real-time noise-aware tone mapping}, 
  journal = {ACM Trans. Graph.}, 
  volume = {34}, 
  number = {6}, 
  year = {2015}, 
  numpages = {15}, 
  url = {http://doi.acm.org/10.1145/2816795.2818092}, 
  doi = {10.1145/2816795.2818092}, 
  publisher = {ACM}, 
  address = {New York, NY, USA}, 
  keywords = {tone mapping, video tone mapping, hdr imaging}, 
}

 



Page manager: gunnar.host@liu.se
Last updated: Wed Oct 14 08:53:30 CEST 2015