Skip to Main content Skip to Navigation

Inter-reflections in computer vision : importance, modeling & application in spectral estimation

Abstract : In this thesis, we study an optical phenomenon often ignored in computer vision, the interreflection phenomenon. Interreflections, which can also be found in the literature under the name mutual illumination happen whenever a concave surface is illuminated. As the name tells, a light ray coming from the light source and hitting a surface point will reflect toward some other point, then another, and so on, before reaching the camera sensor or the eye. Hence, a ray does inter-reflect between the different points of a concave surface. Interreflections lead to color variations, or color gradients, all over the concave surface. These variations are more or less pronounced depending on many factors including, but not limited to, the surface reflectance and its geometry. We will show in this manuscript that these color variations hold some important information which is worth to be used in computer vision. They also play an important role in perception leading to a better color constancy in human vision as demonstrated in our experiments. In order to be able to efficiently use interreflections in some computer vision applications, a spectral infinite-bounce model of interreflections is introduced in the manuscript. This radiometric model allows us to define the relation between the raw RGB values correspondingto the concave surface in the image on one side, and the spectral reflectance and the geometry of this surface, the spectral power distribution of the light and the spectral responses of the camera sensors on the other side. Thanks to this model, we were able to study some applications of interreflections in spectral estimation. We show that a task, such as spectral reflectance estimation form a single RGB image, which is almost impossible without learning even under known illuminant and spectral responses of the camera, is made possible thanks to interreflections. Our results show that, spectral reflectance estimation of a folded surface gives a similar accuracy and sometimes a better one when compared to the state of the art approaches that need three different images of the flat surface taken under three different illuminants. Moreover, interreflections help in proposing a more concrete application of spectral reflectance estimation where a standard light SPD can be used and no pre-calibration for the acquisition settings is needed. Later, we show that interreflections are useful in some applications which need color charts such as camera characterization. The nature of interreflections leads to special colors resulted from raising the spectral reflectance to multiple powers. Using these colors along with the interreflection model helps in introducing some non linearly-related information and thus in obtaining a better spectral characterization. Hence, using 3D color charts is more beneficial than adding new colors to 2D color charts. Finally, we train a convolutional neural network on simulated images of interreflections in order to get an estimation of both the spectral reflectance and the SPD of light from a single RGB image. The experimental results show that our approach is able to get both spectra with a very good accuracy compared to other approaches. In addition, this approach performs very well on real images thanks to the added noises in the training process.
Complete list of metadatas

Cited literature [93 references]  Display  Hide  Download
Contributor : Abes Star :  Contact
Submitted on : Friday, October 4, 2019 - 2:16:06 PM
Last modification on : Monday, January 13, 2020 - 5:46:07 PM


Version validated by the jury (STAR)


  • HAL Id : tel-02305680, version 1


Rada Deeb. Inter-reflections in computer vision : importance, modeling & application in spectral estimation. Other [cond-mat.other]. Université de Lyon, 2018. English. ⟨NNT : 2018LYSES035⟩. ⟨tel-02305680⟩



Record views


Files downloads