Fig. 1. Using our technique to adjust the noise level of photographs to different ISO values. A photograph captured with ISO 100 (left) has its noise level adjusted to ISO 1600 (blue arrow). Likewise, another photograph of the same scene taken with ISO 1600 (right) has its noise level adjusted to ISO 100 (red arrow). The noise distributions for patches Real 1600 and Synthetic 1600 have a Kullback-Leibler (KL) divergence of 0.0617 and the result of their Kolmogorov-Smirnov (KS) test is 0.0878 (with a p-value of 7.81 × 10 −220 ), indicating that the two distributions are very similar. The red, green, and blue histograms underneath patch Real 1600 show the noise distributions corresponding to the R, G, and B channels of patch Real 1600, respectively. The superimposed gray histograms are the corresponding noise distributions for the patch Synthetic 1600. The PSNR value computed for patch Synthetic 100 is 29.13, also indicating a good agreement with patch Real 100.

Synthesizing Camera Noise using Generative Adversarial Networks

Bernardo Henz bhenz@inf.ufrgs.br

Eduardo S. L. Gastal eslgastal@inf.ufrgs.br

Manuel M. Oliveira oliveira@inf.ufrgs.br

We present a technique for synthesizing realistic noise for digital photographs. It can adjust the noise level of an input photograph, either increasing or decreasing it, to match a target ISO level. Our solution learns the mappings among different ISO levels from unpaired data using generative adversarial networks. We demonstrate its effectiveness both quantitatively, using Kullback-Leibler divergence and Kolmogorov-Smirnov test, and qualitatively through a large number of examples. We also demonstrate its practical applicability by using its results to significantly improve the performance of a state-of-the-art trainable denoising method. Our technique should benefit several computer-vision applications that seek robustness to noisy scenarios.

Keywords

Noise model; GANs; Deep learning.

Paper

Pre-print Version (23 MB)

This is the author's version of the work, and it is not for redistribution. The final publication is available via the IEEE Transactions on Visualization and Computer Graphics.

Additional Material

This material provides implementation and evaluation details, plus additional comparison cases for visual inspection.

Additional Material (37.7 MB)

Code

Citation

Bernardo Henz, Eduardo S. L. Gastal, Manuel M. Oliveira. Synthesizing Camera Noise using Generative Adversarial Networks, IEEE Transactions on Visualization and Computer Graphics, 27(3), pp. 2123-2135, 2021.

BibTeX

@article{HenzGastalOliveira_2020,
    author  = {Bernardo Henz, Eduardo S. L. Gastal, Manuel M. Oliveira},
    title   = {Synthesizing Camera Noise using Generative Adversarial Networks},
    journal = {IEEE Transactions on Visualization and Computer Graphics},
    year    = {2021},
    volume  = {27},
    number  = {3},
    pages   = {2123-2135},
    doi     = {10.1109/TVCG.2020.3012120}
    }
  

This work was sponsored by CNPq-Brazil (fellowships and grants 312975/2018-0, 436932/2018-0, 423673/2016-5), and financed in part by the Coordenação de Aperfeiçoamento de Pessoal de Nı́vel Superior – Brasil (CAPES) – Finance Code 001. We would like to thank NVIDIA for donating the GeForce GTX Titan X GPU used for this research.