Abstract

We present a convolutional neural network architecture for performing joint design of color filter array (CFA) patterns and demosaicing. Our generic model allows the training of CFAs of arbitrary sizes, optimizing each color filter over the entire RGB color space. The patterns and algorithms produced by our method provide high-quality color reconstructions. We demonstrate the effectiveness of our approach by showing that its results achieve higher PSNR than the ones obtained with state-of-the-art techniques on all standard demosaicing datasets, both for noise-free and noisy scenarios. Our method can also be used to obtain demosaicing strategies for pre-defined CFAs, such as the Bayer pattern, for which our results also surpass even the demosaicing algorithms specifically designed for such a pattern.

Keywords

Color Filter Arrays; Demosaicing; Computational photography; Convolutional Neural networks.

Downloads

Paper

Pre-print Version (7.3 MB)

This is the author's version of the work, and it is not for redistribution. The final publication is available via the Computer Graphics Forum.

Code

Results

Our Trained 4x4 CFAs

Reconstruction Results

Check our supplementary material to view various examples comparing reconstructions of our models against state-of-the-art techniques, both for noise-free and noisy scenarios.

Reference

Citation

Bernardo Henz, Eduardo S. L. Gastal, Manuel M. Oliveira. Deep Joint Design of Color Filter Arrays and Demosaicing, Computer Graphics Forum, 37(2), pp. 389-399, 2018.

BibTeX

@article{HenzGastalOliveira_2018,
    author  = {Bernardo Henz and Eduardo S. L. Gastal and Manuel M. Oliveira},
    title   = {Deep Joint Design of Color Filter Arrays and Demosaicing},
    journal = {Computer Graphics Forum},
    volume  = {37},
    number  = {2},
    DOI     = {10.1111/cgf.13370},
    ISSN    = {1467-8659},
    pages   = {389--399},
    year    = {2018},
}

Acknowledgments

This work was sponsored by CNPq-Brazil (fellowships and grants 306196/2014-0, 423673/2016-5). We would like to thank NVIDIA for donating the GeForce GTX Titan X GPU used for this research.