Improvement of the visibility of concealed features in misregistered NIR reflectograms by deep learning

Features of Old Master paintings hidden under the upper layer of a painting are often studied using NIR reflectograms; however their interpretability can be reduced due to the visible content. In our previous work [3] we described the possibility of increasing the visibility of concealed features in NIR reflectograms from the painting surface. The method output, enhanced NIR reflectogram, is produced by extrapolating the VIS data to a NIR range reflectogram and subtracting it from the acquired data in the NIR spectral subband. As a result, separated information from the NIR domain is obtiained. This method has a severe limitation, because it requires precise image registration of the VIS and NIR spectral bands. This is often hard to achieve, because DSLR cameras or multiple devices with various optical systems are used for data collection, and the mutual spatial relation of the images is often unknown. Thus, in the original form, the algorithm was applicable only for data acquired using special scanners producing spatially registered images (as in [4]). In this work, we present an extension of the previous algorithm inspired by deep learning. The new concept allows processing of images only partially registered with pixel precision; subpixel accuracy is no longer needed. We suggest an extension of neural network input with neighboring pixels and allocation of extra ANN layers for translation compensation. The results are demonstrated on misregistered images captured by DSLR camera in VIS and NIR. (paper)