Beyond color: in-situ acquisition of monumental ornaments

Introduction: appearance

Appearance is an optical phenomenon seen by a user or captured by a sensor after the interaction of light with an object (fig. 1). The result depends on the incoming light (e.g. spectrum, direction, energy…), the physicochemical composition of the object that can modify the properties of the incoming light (e.g. absorption, diffusion…), and naturally on the surface quality (e.g. roughness). The modelling and acquisition of appearance is thus intrinsically a multidimensional problem. State-of-the-art points up from 12 (fig. 2 and Dong 2013¹) to 14 dimensions in non-free space².

In the heritage field, when it comes to acquiring visual appearance beyond color, the benchmark remains RTIs (Reflectance Transform Imaging³). It is limited compared to a BRDF (Bidirectional Reflectance Distribution Function⁴) acquisition, since it captures only a single viewpoint. It uses the PTM (Polynomial Texture Maps)⁵ approximation of reflectance using a low-degree polynomial leads to an easy but smoothed estimate of a normal field.

In addition, illumination is approximated by a directional source⁶, further reducing the quality of the approximation.

Finally, only one point of view is acquired, which is not suitable for estimating spatial variations in reflection (SVBRDF–Spatially Varying BRDF).

Context

The context of this presentation is the restitution of the Gaston de Saint-Maurice’s residence in Cairo. This building is an excellent example of the reuse of ornaments during the construction of modern Cairo⁷. Built between 1875 and 1879, using decor from Cairo’s monuments, it was demolished in 1937. Some of the decor was re-used for the construction of the new French Embassy in Cairo, which still exists today. For a restitution⁸, we need to acquire the appearance of its decorations. Reproducing the complex appearance allows for greater fidelity of the 3D restitution compared to observations in real conditions.

As the conventional use of a lighting dome (as in RTI) is not possible on site for large, immobile objects, the standard approach relies on the use of black reflecting spheres to estimate light direction⁹. Finally, learning methods (e.g.Volait 2013¹⁰) currently offer no metrological guarantees.

For greater flexibility, we retained an approach based on a simple camera and a light source (a spotlight for photography). This also makes it possible to adapt to on-site acquisition constraints, thanks to their portability and the use of standard photographic equipment, which poses fewer administrative problems when deployed on site.

Methodology

In this paper, we present an outline of the approach, which is detailed in Corentin Cou’s thesis manuscript¹¹. The setup is detailed in fig. 3.

Results

The measurement campaign was carried out in November 2019, in the French Embassy in Cairo (Egypt). Due to the use of standard equipment, we only carried the checkerboard and mirror spheres, as well as a camera. It was a Canon EOS 5D Mark II (definition of 56126×3744 pixels, focal distance–50 mm, aperture–F/6.7, sensibility–ISO 1600, exposition time 1/6 s). The rest of the equipment belonged to the photographer Matjaž Kačičnik, who was in Cairo during our assignment.

The ceiling shown in figure 4 was acquired during the afternoon, including appearance and photogrammetry.

We used a Nikon D850 (definition of 8256×5504 pixels, focal distance–60 mm, aperture–F/7.1, sensibility–ISO 400, sequence of exposition times–1/3 s-0.62 s-1.3 s-2.5 s-5 s-10 s).

The data processing algorithm was implemented in 2023. In order to best calibrate the light spot used during the campaign, the same model was acquired (fig. 3-5–Profoto B10), notably to validate the approximation by a Lambertian disk. This validation has been done based on in-laboratory radiometric measures¹⁸.

Conclusion

We have demonstrated that it is possible, using a simple setup, to capture the appearance of a monumental scenery, more precisely, a SVBRDF. The setup uses only common, off-the-shelf hardware, thus reducing deployment difficulties.

We have also proposed a new methodology to find the position of a projector-type light source.

This method frees us from the constraints of positioning such a source, and contributes to its ease of deployment.

After numerical optimization, the visual results are consistent with the original images. However, we need to go further than simple visual validation to offer an approach tending towards metrology. The points for improvement and research are as follows. From a model point of view, we do not obtain a single normal map, but one per viewpoint. We need to improve the optimization process to resolve this limitation. Then, still on the optimization method, we need to make it more robust when brightness is high. Finally, to aid the study of such sets, it would be useful to be able to relate the BRDF parameters retrieved to the actual physical parameters.