Relationship between reflectance and degree of polarization in the VNIR-SWIR: A case study on art paintings with polarimetric reflectance imaging spectroscopy

Federico Grillini; Lyes Aksas; Pierre-Jean Lapray; Alban Foulonneau; Jean-Baptiste Thomas; Sony George; Laurent Bigué

doi:10.1371/journal.pone.0303018

Abstract

We study the relationship between reflectance and the degree of linear polarization of radiation that bounces off the surface of an unvarnished oil painting. We design a VNIR-SWIR (400 nm to 2500 nm) polarimetric reflectance imaging spectroscopy setup that deploys unpolarized light and allows us to estimate the Stokes vector at the pixel level. We observe a strong negative correlation between the S₀ component of the Stokes vector (which can be used to represent the reflectance) and the degree of linear polarization in the visible interval (average -0.81), while the correlation is weaker and varying in the infrared range (average -0.50 in the NIR range between 780 and 1500 nm, and average -0.87 in the SWIR range between 1500 and 2500 nm). By tackling the problem with multi-resolution image analysis, we observe a dependence of the correlation on the local complexity of the surface. Indeed, we observe a general trend that strengthens the negative correlation for the effect of artificial flattening provoked by low image resolutions.

Citation: Grillini F, Aksas L, Lapray P-J, Foulonneau A, Thomas J-B, George S, et al. (2024) Relationship between reflectance and degree of polarization in the VNIR-SWIR: A case study on art paintings with polarimetric reflectance imaging spectroscopy. PLoS ONE 19(5): e0303018. https://doi.org/10.1371/journal.pone.0303018

Editor: Kristofer Lasko, USACE ERDC: US Army Engineer Research and Development Center, UNITED STATES

Received: September 12, 2023; Accepted: April 17, 2024; Published: May 9, 2024

Copyright: © 2024 Grillini et al. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.

Data Availability: The reflectance hyperspectral images associated to this study and the angles of the linear polarizers placed in front of the cameras are found in the following public repository: https://zenodo.org/records/10814944. DOI 10.5281/zenodo.10814943.

Funding: This research was funded by Partenariat Hubert Curien (PHC) AURORA project number 48341RF. This work was also supported by the ANR JCJC SPIASI project, grant ANR-18-CE10-0005 of the French Agence Nationale de la Recherche. The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.

Competing interests: The authors have declared that no competing interests exist.

1 Introduction

Specular highlights, shadows, and other atmospheric conditions such as haze are extremely important cues that the human visual system uses to resolve a scene [1–3]. They can provide information regarding the direction of the illumination source, the relative location of objects, and can trigger perceptual effects like color constancy [4, 5]. However, in the framework of image-based material analysis, such external agents represent an obstacle in tasks such as characterization and classification [6, 7]. In these instances, the imaging process is affected by a type of noise that results in an ambiguous interpretation of the data, e.g. a naturally light material or a dark material oriented in a way that reflects with specular highlights. In computer vision applications, the task boils down to estimating and discarding the effects of the external agent E in the following Eq 1, in which X represents the recorded image value and P is the true property of the material under study. (1)

In the context of technical imaging of paintings, it is not seldom to encounter challenging glossy targets that display many specular reflections [8, 9]. Usually, the main responsible for the glossy appearance of a painting is the varnish, the outer resin-based layer that is applied with the twofold goal of protection and color enhancement [10]. Specular reflections can also be observed in unvarnished paintings that have been produced with specific pictorial techniques that alter the surface geometry [11]. In impasto, for example, large quantities of paint are applied in wide brushstrokes or with a knife, so that the surface is composed of a wide distribution of planar micro facets, from which a resulting specular reflection can be produced. This specular light, therefore, does not come from a single specular plane of incidence, but from a combination of several [12, 13]. Regardless of what causes the specular reflections, the produced pixel values will show saturation, thus preventing any accurate analysis.

Post-processing solutions have been extensively studied to detect and remove the undesired effects introduced by specular reflections [14, 15]. At the same time, it is possible to act at the root of the problem, by carefully designing imaging techniques that can limit the presence of such image flaws [16, 17].

Polarization is a fundamental property of an electromagnetic wave that describes the direction of the electric field oscillation perpendicular to the direction of propagation. The usage of polarization filters is a well-known solution to characterize and classify surfaces [18–22], or to simply reduce the effect of specular reflections in a scene [23, 24]. When carefully designed, a polarization imaging system can allow the detection and removal of such specular components. At the same time, it offers a gateway into looking at the polarization or depolarization effects that are induced by the reflection or transmission of light [25].

When historical artifacts are analyzed by means of imaging, it is crucial to deploy a sustainable technique that does not harm the ongoing preservation and conservation processes. Hyperspectral imaging (HSI), more formally known as imaging spectroscopy, is a non-invasive and non-destructive imaging technique extensively used to study historical artifacts [26].

The combination of polarization and spectral imaging saw the development of Spectral Polarization Imaging (SPI), a technique to capture at the same time polarization and spectral information from a scene. To our knowledge, a compact sensor for the recording of hyperspectral and polarization does not exist yet, so a variety of experimental protocols have been developed for the capturing of such data combinations [27]. SPI is a relatively new field of research with a lot of potential for applications in the context of Cultural Heritage analysis. Most of the literature on SPI has been focusing on the coupling of polarization imaging with multispectral systems, to compactly collect polarization and wide-band spectral data [28–30].

In one of the first studies on SPI, Le Hors et al. [31] observed a strong inverse correlation between the degree of linear polarization and the reflectance of diffuse materials such as paints and coatings. It is noteworthy that a similar relationship, termed the Umov effect [32] from the Russian astronomer Nikolay Umov who first observed it in 1905, had been observed in the field of astronomy between the albedo of a planet and its degree of linear polarization for large phase angles (intended as the angle between the incident and reflected radiation).

Such correlation is attributed to the individual contributions of surface and volume scattering that take place within the paint layer. The composition of a paint layer can be schematized by a multitude of pigment particles that float in a binding medium (usually a type of oil). In their work, Le Hors et al. [31] deployed polarized light and diffusive media, and provided the following explanation of the phenomenon. When polarized incident radiation impinges a surface, a part of it experiences surface scattering and is reflected along the specular direction, maintaining its polarization state. The remaining part of the radiation is either absorbed by the material or experiences volume scattering and is reflected. In the case of absorbance, the final measurement detects a low reflectance with a definite polarization angle (and thus a high degree of linear polarization) for which surface scattering is responsible. In case volume scattering takes place, the outgoing reflected radiation is depolarized but its contribution is greater than the surface-scattered component. Thus, the measurement will detect a high reflectance and a low degree of linear polarization. In the following research, the same team (Le Hors et al. [33]) could build a Kubelka-Munk-based model for the description of the depolarization phenomena in paints and other diffuse materials.

The correlation between the degree of linear polarization and reflectance in diffuse materials is an interesting property that can be potentially deployed as a feature for the characterization of materials in conservation science, but it is unknown if it depends on the observed spectral range.

In this article, we propose a paradigm for the acquisition of hyperspectral images in the Visible and Near-Infrared (VNIR) and Short-Wave Infrared (SWIR) in combination with polarization information to investigate in more detail the correlation properties of the interaction between the degree of linear polarization and reflectance. The target used for this study is a mockup oil painting with a rather complex surface topology. The first goal of this paper is to assess the presence of the inverse correlation in the visible range in an imaging context (since the original study by Le Hors et al. [31] considered punctual measurements), where the information at the pixel level is affected by several variables. Secondly, we extend the study to the whole available spectrum, and by exploiting the fine spectral sampling we explore how the correlation varies locally through different spectral windows. Finally, we investigate the role of spatial resolution on the computed correlation and try to connect it to the surface properties.

The article is organized as follows. In Section 2, spectral and polarization models and assumptions are provided for a surface exhibiting specular highlights. After deriving predictions on the spectropolarimetric signatures, the acquisition setup and imaging pipeline are presented in Section 3. This will be used as an experiment to confront our predictions with the measurement. We discuss the results in Section 4, before concluding in Section 5.

2 Background

2.1 Spectral and polarization imaging model

The radiation reflected off a surface depends on the spectral power distribution and the direction of the illumination source, the optical properties of the surface, and the angle between the surface normal and the illumination direction. The Bidirectional Reflectance Distribution Function (BRDF) [34] describes how much radiation is captured by a spectral sensor, factoring in all the previous terms, plus the direction of observation. In Eq 2, I_i and I_r are the incident and observed radiation respectively, while ω_i and ω_r are the illumination and observation direction, respectively. (2)

The BRDF is notably a highly complex model, and cannot be accurately estimated without extensive measurements and expensive setups [35]. However, assuming that a surface is Lambertian can simplify the model by discarding the angular-dependent terms. A Lambertian surface is defined as flat, matte, and diffusive. These three attributes intrinsically contribute to making a Lambertian material isotropic and free from fluorescence phenomena. When the angular terms are discarded, the BRDF coincides with the reflectance of the material under examination: (3)

When specular highlights are observed in an image, it is very clear that the Lambertian assumptions do not hold any longer, and a different model should be used to describe the reflectance behavior.

The dichromatic reflectance model [36] assumes that the reflection of light is composed of a diffuse component (sub-scattering and surface roughness) and a specular component (direct surface reflection). The total intensity I_r after a surface reflection can be modeled by the sum of two intensity components [17], such as: (4) where I_d is the diffuse component, assumed to be completely unpolarized, and I_sp is the specular component. The specular reflection has polarization features that depend on the optical properties of the surface interface, i.e. the Fresnel coefficients which are a function of refractive index, wavelength, angle of incidence/reflection, and polarization status of the incident light. In the instance of materials that present multiple sub-surface interactions like paints, it is often assumed that the diffuse component is unpolarized, while the specular component is assumed to be partially polarized. Thus, rotating a linear polarizer with an angle θ in front of a camera leads to an intensity measurement such as: (5) where I_sp,c is the constant specular component relative to the angle of the polarization filter, I_sp,v is the amplitude of a cosine function term of the variable specular component, and ϕ is the angle of linear polarization of light [37]. A visualization of the intensity variation is shown in Fig 1.

Download:

Fig 1. Partially linearly polarized light passing through a linear polarizer after a surface reflection (simulation).

The output intensity describes a phase-shifted sinusoid of phase ϕ, with constant and variable intensity components.

https://doi.org/10.1371/journal.pone.0303018.g001

Most of the related works measure polarization signals in relatively narrow spectral ranges or with wide spectral bands. This leads to a characterization of polarization that is often limited in terms of spectral analysis. In this work, we capture images of the linear polarization of the reflected light from a surface from 400 nm to 2500 nm sampling at a high spectral resolution.

2.2 Stokes vector

The Stokes formalism enables the full description of the polarization state of light with a 4-element vector named the Stokes vector. A conventional Stokes imaging technique combines a camera and a polarizing element to form a Polarization State Analyser (PSA). A linear polarimeter is able to estimate the first three elements of the Stokes vector , corresponding to the linear polarization state of the incoming light. In particular, S₀ describes the total power of the incident beam, while the S₁ and S₂ components express the difference between intensities measured through orthogonal directions of the polarizer.

The response of a single PSA to a particular input Stokes vector is modeled by the response of a pixel by: (6) where I is the pixel response, S is the polarization state of the input light, and is the analyzer vector which embeds the polarizer characteristics of the analyzer components, i.e. transmission, polarizing angle, and extinction ratio coefficient. If ideal transmission and extinction ratio are assumed, the analyzer vector is only a function of the rotation angle, and Eq 6 can be rewritten by: (7)

We consider a PSA with M = 4 discrete polarizer positions, which is enough to estimate S. When it is possible, these four angles are selected to be equally spaced in the interval [0, 180]° [38]. In this work, the polarizer is rotated manually, so the angles θ₁₋₄ are not known. This leads to a vector of intensities I defined by: (8) in which W is the analysis matrix that combines the four analyzer vectors A and θ₁₋₄ are the polarizer angles of the four PSA configurations. The four angles are estimated through a calibration procedure (described in Section 3.3).

Once W is estimated, the Stokes vector can be computed from I for each pixel in the image, such as: (9) in which is the pseudo-inverted PSA matrix estimation. For a polarimeter that has K spectral bands, the Stokes vector is estimated by spectral band k as follows: (10)

A more intuitive set of polarimetric parameters can be computed from Stokes vectors, which are the degree of linear polarization ρ_k and the angle of linear polarization ϕ_k: (11)

The manually rotated PSA has a transmission axis θ (with respect to a reference angle). The transmitted light is thus linearly polarized along this axis but with an intensity attenuated by a specific amount, modeled by a cosine law [19]: I(θ) = I₀ cos² θ.

3 Materials and methods

3.1 Experimental set-up

The hyperspectral cameras deployed in this study are of the type push-broom. They are sensitive to the VNIR (Hyspex VNIR1800, Norsk Elektro Optikk) and SWIR (Hyspex SWIR384, Norsk Elektro Optikk). The VNIR image sensor, manufactured in Silicon (CMOS), captures radiation from 400 nm to 1000 nm with 186 spectral channels and with 1800 pixels on the acquisition line, whereas the SWIR sensor (Mercury-Cadmium-Telluride) is sensitive in the interval 950 nm to 2500 nm with 288 spectral channels and 384 pixels on the acquisition line.

Hyperspectral images in VNIR and SWIR were captured with different but highly similar setups, schematized in a single one in Fig 2a. Due to space limitations, it was not possible to deploy both cameras simultaneously, so we opted to acquire VNIR and SWIR images sequentially.

Download:

Fig 2. Imaging setup.

a) Schematized experimental setup with a legend for the corresponding elements. The illumination source and the elements that form the scene must be kept in place when the imager is changed (from VNIR to SWIR for example), in order to keep the BRDF as close as possible between the two hyperspectral modalities. b) Color rendering of an example scene in VNIR. c) False color infrared of an example scene in SWIR in which the images of the reference polarizers have been stacked in post-processing.

https://doi.org/10.1371/journal.pone.0303018.g002

An analysis polarizer (also called analyzer, represented by θ) is placed before the camera objective and can be rotated manually at each new capture. The VNIR polarizer is a 1” Meadowlark NIR Versalight wide-grid polarizer (VLR-100-NIR), while the SWIR camera is coupled with a 1” Meadowlark IR Versalight wide-grid polarizer (VLR-100-IR) [39]. These linear polarizers are built with aluminum nanowires and their contrast ratio, reported in Fig 3, ensures an efficient usage in a broadband spectral range. Indeed the contrast ratio is constantly higher than 500 for the VNIR polarizer, and constantly above 2000 for the SWIR polarizer, which is an indication of reliability.

Download:

Fig 3. Polarization characteristics of the analysis polarizers placed in front of the hyperspectral imagers.

a) VNIR analysis polarizer. b) SWIR analysis polarizer. The figures are displayed as reported in [39].

https://doi.org/10.1371/journal.pone.0303018.g003

Previous research [40] aimed at characterizing Hyspex VNIR1600 and Hyspex SWIR320m-e (two earlier models of the ones used here) observed a maximum 10% polarization sensitivity in both cameras in the first spectral bands of the individual sensors, with the average sensitivity stabilizing at around 5%. We can assume that this observation is still in place for the sensors deployed in this work, but the reported polarization sensitivity does not impact the measurements significantly, since the polarizers added in front of the cameras are highly efficient compared to the inherent sensitivity.

In this push-broom configuration, the translational stage slides across the field of view of the camera at a speed synchronized with the framerate of the camera, while a halogen light shines a constant unpolarized light flux exactly on the acquisition line of the camera.

A substantial difference with the study by Le Hors et al. [31] is the deployment in our setup of unpolarized light. This implies that it is not possible to strictly talk about depolarization effects and that any polarization that will be recorded by the imagers is induced upon reflection.

The following elements are placed on the translational stage: the object to be captured, a Spectralon (LabSphere) target for spectral calibration, and a reference polarizer. In the case of the VNIR capture, four cutouts of the same linear polarizer sheet (model XP40HT-40, Edmund Optics) are placed at known α₁₋₄ angles separated by 45° from each other. Due to the unavailability of a sheet that could polarize light in the SWIR range, a single polarizer working in the same range (model LPNIRC100-MP2, Thorlabs) was placed on a motorized rotational stage. The spectral ranges in which the deployed reference polarizers are effective are between 400 nm and 700 nm for XP40HT-40, and between 1100 nm and 1800 nm for LPNIRC100-MP2. This means that the spectral ranges of the cameras are much wider, and therefore only those spectral bands that present an acceptable attenuation will be considered for the polarimetric calibration step.

The target object selected for this experiment is a mockup oil painting that presents quite a lot of specular reflections due to its topology. The pre-primed cotton canvas received two additional priming layers of gesso, while seven pigments were bound with linseed oil and applied either in their pure state or mixed combinations. The painting is unvarnished, thus the observed specular reflections are caused by wide brushstrokes that were applied in an attempt of replicating the impasto technique. At the time of the imaging campaign, the mockup painting had aged for two and a half years in a dry and dark environment (not controlled in temperature or relative humidity).

To summarize, both VNIR and SWIR capture processes require the sequential acquisition of four images with a fixed integration time, each corresponding to a new rotation of the analysis polarizer by 45°. The only difference between VNIR and SWIR resides in the fact that at each manual rotation of the SWIR analyzer, four separate acquisitions of the reference polarizer at angles α₁₋₄ are needed in order to replicate the situation of the four reference polarizers lined up in the VNIR setup. The procedure is schematized in Fig 4 while an example of the captured VNIR and SWIR scenes is reported for θ₂ in Fig 2b and 2c. Here, the sequentially captured SWIR images of the reference polarizer are stacked on the top of Fig 2c.

Download:

Fig 4. Schematized acquisition paradigm for combined VNIR-SWIR hyperspectral polarization imaging.

https://doi.org/10.1371/journal.pone.0303018.g004

3.2 Spectral data preprocessing

The hyperspectral cameras readily provide image cubes of raw data that need to undergo a series of correction and calibration steps before obtaining absolute reflectance data. A first geometric correction [41] is performed to account for distortions in the across-track direction that arise from differences in viewing angles for the pixels on the acquisition line. Then, a radiometric correction is performed to transform the data from raw to relative radiance, discarding the effects of constants, and user and camera-dependent parameters.

Flat-fielding correction deploys the reflectance target placed in the scene, but it can be performed also with a non-standardized target, as long as the surface is uniform and diffusive. The light field recorded by the reflectance target can be non-homogeneous due to the manual positioning of the light source, so this step accounts for this type of distortion, transforming the radiance data into relative reflectance. Absolute reflectance data is then obtained by using the provided reflectance values of the standardized Spectralon target. Eq 12 summarizes the procedure: (12) Here, a pixel with radiance value L(x, λ) is divided by the corresponding radiance value on the flat-fielding target η(x, λ) and then multiplied by the ratio between the reference reflectance of the Spectralon target χ(λ) and the relative reflectance (flat-fielded) extracted from the Spectralon in the scene μ(λ).

In the capturing sequence, it was decided to first acquire all images in the VNIR range, and subsequently, all the images in the SWIR range, while the painting mockup was kept in place. This allows us to assume that all images within the same spectral range are co-registered, while there exists a unique geometric transform (homography) that connects the VNIR set to the SWIR set. The homography matrix is learned with a first step of SIFT feature matching [42], followed by a refining step using the methodology proposed in [43]. It is important to point out that only the mockup painting portion of the scene is registered, while the Spectralon tile and the polarization reference filters are cropped out. At this stage, the images are registered at SWIR resolution, therefore downgrading the quality of the VNIR set. Later in the article, we will point out the analysis sections that consider the VNIR and SWIR ranges jointly, and those steps that consider them independently, therefore using the VNIR set at its full spatial resolution.

3.3 Polarimetric calibration

The polarimetric calibration is the estimation of the four analysis angles θ₁₋₄ for each of the two cameras. It is performed in three steps: 1- measure the relative angles of the reference polarizing filters α₁₋₄, 2- take the intensity values corresponding to the reference polarizers in the scene, and 3- fit a cosine function on reflectance data to find θ₁₋₄.

The VNIR reference polarizers are lined up on a supportive sheet (see Fig 2) and their relative angles α₁₋₄ are measured with a high-performance polarization filter array camera from Lucid Vision Labs, featuring an on-chip SONY IMX250 MYR [44] sensor. Due to the usage of an electronically controlled rotational stage, the reference angles corresponding to the SWIR captures are directly known as input.

The following values are the orientations of the fixed reference polarizers:

(from left to right in Fig 2b)
(from top to bottom in Fig 2c)

The cosine law is then fitted with the least square method on the intensity values extracted from the reference polarizers. Then, the analyzer angles are estimated by finding the phase of each fitted curve. Fig 5 displays an example of the fitting at 564 nm for the VNIR camera and at 1497 nm for the SWIR camera.

Download:

Fig 5. Example of fitting of a cosine law from intensity measurements of reference polarizers, for a single spectral band.

This allows to estimate the angles of analysis θ_VNIR and θ_SWIR from the phases. a) band corresponding to 564 nm. b) band corresponding to 1497 nm.

https://doi.org/10.1371/journal.pone.0303018.g005

The analysis angles θ_VNIR and θ_SWIR are estimated in correspondence with the phases of the curves, and their behavior as a function of wavelength is reported in Fig 6.

Download:

Fig 6. Results of estimated analysis angles θ₁₋₄ at each spectral band.

The shaded areas between 440 nm and 700 nm (a) VNIR) and between 1100 nm and 1800 nm (b) SWIR) represent the spectral range in which the reference polarizers are known to have a high extinction ratio, and consequently, the average θ₁₋₄ values are computed from those spectral bands.

https://doi.org/10.1371/journal.pone.0303018.g006

The reliability of the estimation depends on the extinction ratios of the deployed reference polarizers, which are known to be effective from 400 to 700 nm for the VNIR acquisition, and from 1100 to 1800 nm for the SWIR acquisition. Since the first spectral bands of the VNIR camera are more affected by noise, the considered spectral range is limited to 440 to 700 nm. The following θ_VNIR and θ_SWIR are then computed as the average in the high-extinction ratio spectral ranges (red shaded areas in Fig 6):

The consistent results obtained by wavelength in the polarimetric calibration serve as a health check to demonstrate that the procedure, although including elements of human interaction (manual rotation of analysis polarizer), is accurate enough to retrieve the Stokes vectors pixel-wise and spectrally. Nonetheless, the setup would benefit from the implementation of a motorized system for the rotation of the analysis polarizer.

We then compute the analysis matrix W from Eq 6:

The resulting W_VNIR and W_SWIR values are then used, according to Eq 10, to retrieve the Stokes vector images in a band-wise fashion. The respective condition numbers for W_VNIR and W_SWIR (excluding the fourth columns full of zeros) are 1.438 and 1.446, close to the ideal value of . Hence, our measurement process will provide polarimetric estimates with reduced noise [38].

3.4 Spectro-polarimetric splicing

When spectral information is captured with two different sensors in adjacent or overlapping spectral ranges, it is common to observe discrepancies in their responses. This is usually due to a series of factors that include different spectral bandwidths, low signal-to-noise ratio, and misalignments in the imaging setup that cause the BRDF to slightly vary. Spectral splicing is a correction that smoothly connects two spectra affected by spectral jumps [45], and can be extended to connect VNIR and SWIR hyperspectral images [46], like in our instance.

However, the VNIR and SWIR hyperspectral images cannot be readily connected as they are captured, since the polarization information has been collected at different angles of the PSA. We then propose to apply the splicing correction on the two independent Stokes multi-band images, in order to obtain a full range version of S₀ and ρ that is continuous in the range between 400 nm and 2500 nm.

3.5 Correlation between reflectance and degree of linear polarization

Le Hors et al. [31] observed a strong negative correlation between the proportion of reflected radiation and the degree of linear polarization of diffusive media such as paints in the visible spectral range, between 400 nm and 780 nm. One of the goals of this article is to corroborate this observation also in an imaging framework at the pixel level and verify its validity in the spectral range spanning from 400 nm to 2500 nm. As the work by Le Hors et al. sets a precedent, we opted to study the correlation between reflectance and degree of linear polarization, considering the and components of the Stokes vector as intermediate products of challenging interpretability if taken individually. Furthermore, the angle of linear polarization is not considered in the present study, but it can be included in a future phenomenological investigation.

After having computed the Stokes vectors pixel-wise, the first element is selected to represent the spectral reflectance of the pixel, as per its definition, it is proportional to the pixel reflectance. We term as pseudo-reflectance, to distinguish it from the scene reflectance that can be computed in polarimetric systems [47]. From the Stokes vector, the degree of linear polarization ρ is computed with Eq 11.

The global correlation coefficient can be computed as: (13)

By computing the correlation at the pixel level, it is possible to obtain a spatial map of correlation. The polarization of reflected light has a strong dependence on the surface topology. Since the mockup painting presents a distinct roughness, the information regarding the polarization properties of a material can be very unstable in a local neighborhood. We then propose to investigate the behavior of the correlation when subjected to a multi-resolution approach. By doing so, it is possible to pull together groups of neighboring pixels, so that the surface is artificially flattened, in the case where multiple surface normals coexist, or uniformed toward a dominant normal direction.

Typically, multi-resolution approaches make use of techniques such as Gaussian blur or Laplacian Pyramids, but these methods do not consider the spatial structures and patterns that exist in a scene, thus pooling neighboring pixels together indiscriminately. We propose to segment the scene using the SLIC superpixel technique [48]. In this way, similar neighboring pixels are pushed together while the spatial structures of the image are still recognizable. We argue that by doing so, all pixels within a superpixel are constituted of the same material.

Then, for each superpixel, we compute the new corresponding Stokes vector, the degree of linear polarization, and the correlation.

Another property of correlation that we want to investigate is its dependence on the considered spectral range. This can be studied by computing a local measure of correlation [49] within a predefined spectral window of width w as: (14)

4 Results and discussion

4.1 Decision on the number of superpixels

The SLIC algorithm allows to over-segment an image into perceptually similar neighborhoods of irregular shapes. The first question that we need to answer is which image is taken as a reference for the computation of the superpixel masks. Since the SLIC algorithm was originally designed to work on color images, it is more appropriate to work on an RGB representation of the scene. The four VNIR images acquired at the different analysis angles θ₁₋₄ record specular reflections with slightly different intensities and patterns, so generating superpixels from an individual image captured at θ_i might not be able to generalize for the remaining three instances. For this reason, it was decided to generate a new RGB image from a θ-independent version of the scene, represented by the S₀ component of the Stokes vector computed from the RGB intensity values.

The subsequent questions concern the number of multi-resolution steps and the number of superpixels that each step should have. The only inputs required by the SLIC algorithm are an image and the approximate number of superpixels. The algorithm will then output a number of superpixel masks close but not equal to the one provided as input. To find out the maximum number of superpixels, we graphically look for the breaking point of the linear relationship between input/output. This is illustrated in Fig 7.

Download:

Fig 7. Decision on the maximum number of superpixels.

The breaking point of linearity is observed around the input of 18000 superpixels.

https://doi.org/10.1371/journal.pone.0303018.g007

After having selected 18000 as the maximum number, we progressively decrease by a factor of 1.5, thus obtaining the following list of superpixel numbers: 18000, 12000, 8000, 5333, 3555, 2370, 1580, 1053, 702, 468, 312, 208, 139, 93, 62, 41, 27, 18. It is important to point out that for a given application the number of deployed superpixels is highly dependent on the scene content. Moreover, the final decision regarding what edges to preserve is also affected by subjectivity and the purpose of use.

Fig 8 displays some selected results of the over-segmentation by reporting the average color values within the superpixel masks.

Download:

Fig 8. Examples of over-segmentation with SLIC superpixels on the image obtained from the S₀ element of the Stokes vector.

The most relevant edges of the scene are preserved even with a very small number of superpixels.

https://doi.org/10.1371/journal.pone.0303018.g008

4.2 Global correlation: Scale and wavelength dependence

Investigating the properties of an observed correlation can provide further analysis into discovering the underlying relationship between two quantities. In this instance, it was previously observed that the reflectance and the linear degree of linear polarization are strongly correlated (negatively) in the visible range.

The first approach that we propose, is to verify this hypothesis at the pixel level on the scene constituted by the mockup painting, for four different spectral ranges: 400-780 nm, 780-1500 nm, 1500-2500 nm, and the full range 400-2500 nm. Fig 9 illustrates this. The original assumption is promptly verified in the visible range (400—780 nm, Fig 9a), but something interesting happens as we shift towards longer wavelengths. The first part of the infrared range (780—1500 nm, Fig 9b) shows a marked decorrelation between S₀ and ρ, while a more strong negative correlation comes back for a deeper interval in the infrared (1500—2500 nm, Fig 9c). The global correlation in the full range (Fig 9d) seems to provide a sort of average, although correlation is a non-linear quantity and this can sound misleading.

Download:

Fig 9. Global correlation (R_g) maps in different spectral ranges.

Besides the evident variation in correlation magnitude between the different spectral ranges, it is also possible to observe the change in the distribution of highly correlated/decorrelated areas.

https://doi.org/10.1371/journal.pone.0303018.g009

To understand the fluctuation of correlation when moving towards longer wavelengths, it is perhaps necessary to refer to the interaction of infrared radiation with the painting layer. Often, paintings are investigated with infrared radiation to reveal underdrawings and pentimenti [50] underneath the paint layer, exploiting the transmission properties of pigments in this spectral range. In the range from 780 nm to 1500 nm, the radiation gets trapped in a series of volume scatterings and upon exiting the material its degree of linear polarization is very small. An almost zero and constant degree of linear polarization then produces the decorrelation values observed in Fig 9b.

We argue that the observed strengthening of negative correlation in the range between 1500 nm and 2500 nm is counter-intuitive. Longer wavelengths should penetrate deeper into the material and generate more volume scattering, thus displaying an even lower degree of linear polarization. Polarization of the radiation could be observed if the infrared light interacted with the underneath preparatory layer coated with gesso. However, we exclude this possibility as we know by construction that the painting layer is thick enough to absorb all radiation in the considered spectral range.

It is interesting to observe how some areas change their behavior when switching the considered spectral range. Take for instance the area on the right side that is red in Fig 8. This area shows a strong negative correlation in the visible range, then a relatively strong positive correlation in the near-infrared, and finally a decorrelation in the deeper infrared range. This peculiar behavior could be ascribed to a particular pigment, in this instance Vermilion, but further analyses are required.

In Fig 10 we report a similar visualization to Fig 9, this time using the multi-resolution approach displaying four instances of superpixel segmentation.

Download:

Fig 10. Global correlation (R_g) maps in different spectral ranges and for different numbers of superpixels.

https://doi.org/10.1371/journal.pone.0303018.g010

As support to Fig 10, we illustrate the distribution of correlation values for each scene in Fig 11.

Download:

Fig 11. Global correlation distributions from the scenes of Fig 10.

The horizontal lines represent the mean of each scene, with values ranging from -0.77 to -0.84 for the different resolutions in the visible range (400-780 nm), -0.30 to -0.66 in the NIR range (780-1500 nm), and -0.76 to -0.95 in the SWIR range (1500-2500 nm).

https://doi.org/10.1371/journal.pone.0303018.g011

By further studying the original hypothesis formulated by Le Hors, it became soon evident how they are based on assumptions that in real-case scenarios are seldom respected. In the context of imaging a real painting, unlike in the case in which smooth mockups are considered, it is necessary to consider the further level of complexity provided by the local surface topology around a pixel.

Let us consider for instance the first column of Fig 10. Upon visual inspection of the painting, it is possible to observe that the areas showing the strongest negative correlation are those in which the surface is relatively smoother. As a possible explanation for this, we argue that when the complexity of the topology is increased, surface scattering plays a more prominent role, thus keeping both reflectance and degree of linear polarization at high levels and correlated at times.

As we shift towards coarser scale representations, the global correlation becomes ever more negative. Indeed, grouping pixels in a local neighborhood is equivalent to reducing the complexity of the surface topology. However, the spatial resolution in this context can represent a bottleneck for the evaluation of polarimetric properties. A too-fine pixel representation can result in noisy data, whereas a too-coarse grouping can flatten the surface and lead to misinterpretations.

The recorded degree of linear polarization is however dependent on the relative angle between the incident light and the surface normal. We argue that a change in illumination direction, which would fade away from imaging standards in Cultural Heritage digitization, will produce a different distribution of specular reflections, but would not change the macro-observation formulated so far, as the surface of the painting is too complex and the resulting distribution of incidence angles would not be too dissimilar. At the same time, it is plausible that this assumption does not hold for angles that approximate the modalities of image capturing with raking light.

The top row of Fig 12 reports the behavior of the global correlation as a function of the number of deployed superpixels. In this case, R_g is the average of all superpixels (or pixels, in the case of the original image) in the scene. In the bottom row of Fig 12 two groups of pixels are identified in the correlation maps displayed in Fig 9: those with a R_g value lower than the 5^th percentile, and those with a R_g greater than the 95^th percentile. The values of these groups of pixels are then tracked through the various resolution steps and plotted. In both rows, the shaded areas illustrate the interquartile range.

Download:

Fig 12. Global correlation as a function of scale expressed in terms of the number of superpixels in which the scene is segmented.

Top row: median correlation value of all superpixels and relative interquartile range in the shaded areas. Bottom row: median correlation in the groups of pixels that result in the 5^th and 95^th percentiles and corresponding interquartile range in the shaded areas.

https://doi.org/10.1371/journal.pone.0303018.g012

Fig 13 illustrates the local spectral correlation R_l as a function of the wavelength. In order to compute this measure it is important to harmonize the spectral sampling, which is different in the VNIR and SWIR ranges. The decision is to interpolate the sampling in the VNIR range to the spectral resolution of the SWIR range (5.45 nm). The width of the window is set to 19 bands (approximately 104 nm), but it can be selected arbitrarily, keeping in mind that a window too small will output a noisy curve, while as the window size is increased, the global correlation R_g is approached. The curve reports the mean value of R_l(λ) computed for all resolution steps, for the whole image. Similarly to the previous plots, the shaded area represents the interquartile range.

Download:

Fig 13. Mean local correlation R_l(λ) between S₀ and ρ as a function of wavelength for all scales.

The local correlation is computed at each wavelength in a window of 19 data points, which correspond to approximately 104 nm. The shaded area corresponds to the computed standard deviation. The subdivision in Sectors is arbitrarily performed according to the shape properties of the curve except for Sectors V and VI, where the division is dictated by the fact that the nominal operating range of the analysis polarizer stops at 2000 nm.

https://doi.org/10.1371/journal.pone.0303018.g013

The plot of Fig 13 has been segmented into six relevant spectral intervals (numbered I, II, III, IV, V, VI) for ease of discussion.

Sector I, representative of the visible range from 400 nm to 780 nm, shows a mild negative correlation followed by a strong negative correlation until its end. It is likely that the first spectral bands are affected by sensor noise and higher polarization sensitivity, as observed by Lenhard et al. for similar devices [40].

The first bands of the infrared (Sector II, 780 nm to 950 nm) display a progressive decorrelation followed by a dip (Sector III, 950 nm—1050 nm), and then complete decorrelation in Sector IV between 1050 nm and 1600 nm. We argue that the dip observed in Sector III could be an artificial result produced either by the change of analysis polarizer in front of the spectral sensor, by the splicing correction, or both.

The deeper Sectors located in the infrared (V from 1600 nm to 2000 nm, and VI from 2000 nm to 2500 nm) display a varying correlation ranging from 0 to -1. The shape of the curve in these intervals could be related to the more spiky nature of infrared reflectance spectra, which have typically narrow absorption bands. It is noteworthy that the analysis polarizer used in front of the SWIR camera has defined specifics up to 2000 nm (Fig 3, hence the division in Sectors V and VI), and its behavior is unknown after this critical wavelength.

5 Conclusion

We designed an efficient spectro-polarimetric system for the acquisition of polarimetric images in combination with reflectance imaging spectroscopy in the VNIR and SWIR. This work is placed in a broader context that aims at achieving deeper material analysis in the context of Cultural Heritage once a Mueller imaging framework is implemented. The main challenge to achieving low-noise Mueller imaging in a wide spectral range resides in the development of broadband circular polarizers (as most use λ-selective phase delayers) to be deployed in the Polarization State Generator. At the present stage, the main goal was to implement a spectral linear Stokes imaging framework with low noise, as well as to study the correlation between (pseudo) reflectance (the first element of the Stokes vector S₀) and degree of linear polarization in a mockup oil painting.

We observe that the correlation between the degree of linear polarization and the amount of reflected radiation is in general negative in the visible range, while a decorrelation is observed in the first part of the infrared range between 780 nm and 1500 nm. Contrarily to the trend, the second part of the infrared (between 1500 nm and 2500 nm) displays a strengthening of the negative correlation. We ascribe these differences to the changes in relative quantities of specular components, surface scattering, and volume scattering, but a fully dedicated study can only be beneficial to explain the observations we made. Moreover, we observe that the correlation depends on the surface topology, as complex local neighborhoods tend to display weaker correlations. This is corroborated by a multi-resolution analysis in which coarser representations of the scene show stronger correlations, due to the fact that the surface is artificially flattened.

The limitations of our system are related to the usage of two separate imaging sensors that need to be spliced around 1000 nm, and to the possible inaccuracy of the analysis polarizer after 2000 nm. Thus, our observations in these intervals need to be consolidated.

Paintings and historical artifacts typically feature complex surface geometries that represent challenging instances of light-matter interactions. On top of that, experimental setups can adopt various illumination geometries that can make the characterization of surfaces even more intricate. Our study was limited to studying the relationship between reflected radiation and its degree of linear polarization, but many more effects take place. A full phenomenological study can then include factors such as dependencies of acquisition geometries (illumination and observation angles), and characterization of the angle of linear polarization.

Furthermore, we argue that the proposed observations should be verified for other dielectric materials and it is likely that they will not be valid for conductive surfaces.

A future line of work can include the spectro-polarimetric characterization of typical materials used in art paintings for the construction of features to be exploited in operations of spectral unmixing and pigment mapping.

References

1. Di Cicco F, Wijntjes MW, Pont SC. Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes. J Vision. 2019;19(3):7–7. pmid:30897625
- View Article
- PubMed/NCBI
- Google Scholar
2. Hubona GS, Wheeler PN, Shirah GW, Brandt M. The relative contributions of stereo, lighting, and background scenes in promoting 3D depth visualization. ACM T Comput-Hum Int. 1999;6(3):214–242.
- View Article
- Google Scholar
3. Henry RC, Mahadev S, Urquijo S, Chitwood D. Color perception through atmospheric haze. J Opt Soc Am A. 2000;17(5):831–835. pmid:10795630
- View Article
- PubMed/NCBI
- Google Scholar
4. Wong KYK, Schnieders D, Li S. Recovering light directions and camera poses from a single sphere. In: Computer Vision–ECCV 2008: 10th European Conference on Computer Vision; 2008. p. 631–642.
5. Wedge-Roberts R, Aston S, Beierholm U, Kentridge R, Hurlbert A, Nardini M, et al. Specular highlights improve color constancy when other cues are weakened. J Vision. 2020;20(12):4–4. pmid:33170203
- View Article
- PubMed/NCBI
- Google Scholar
6. Lopatin J, Dolos K, Kattenborn T, Fassnacht FE. How canopy shadow affects invasive plant species classification in high spatial resolution remote sensing. Rem Sens Ecol Conserv. 2019;5(4):302–317.
- View Article
- Google Scholar
7. Ahmad A, Quegan S. The effects of haze on the accuracy of satellite land cover classification. Appl Math Sc. 2015;9(49):2433–2443.
- View Article
- Google Scholar
8. Elkhuizen W, Essers T, Song Y, Geraedts J, Weijkamp C, Dik J, et al. Gloss, color, and topography scanning for reproducing a Painting’s appearance using 3D printing. J Comput Cult Herit. 2019;12(4):1–22.
- View Article
- Google Scholar
9. Giachetti A, Ciortan I, Daffara C, Pintus R, Gobbetti E. Multispectral RTI analysis of heterogeneous artworks. In: Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage; 2017. p. 19–28.
10. Colombini MP, Modugno F, Giannarelli S, Fuoco R, Matteini M. GC-MS characterization of paint varnishes. Microchem J. 2000;67(1-3):385–396.
- View Article
- Google Scholar
11. Hasegawa T, Tsumura N, Nakaguchi T, Iino K. Photometric approach to surface reconstruction of artist paintings. J Electron Imaging. 2011;20(1):013006–013006.
- View Article
- Google Scholar
12. Wolff LB, Boult TE. Constraining object features using a polarization reflectance model. Phys Based Vis Princ Pract Radiom. 1993;1:167.
- View Article
- Google Scholar
13. Nayar SK, Ikeuchi K, Kanade T. Surface reflection: physical and geometrical perspectives. IEEE Trans Pattern Anal Mach Intell. 1991;13(7):611–634.
- View Article
- Google Scholar
14. Tao MW, Su JC, Wang TC, Malik J, Ramamoorthi R. Depth Estimation and Specular Removal for Glossy Surfaces Using Point and Line Consistency with Light-Field Cameras. IEEE Trans Pattern Anal Mach Intell. 2016;38(6):1155–1169. pmid:26372203
- View Article
- PubMed/NCBI
- Google Scholar
15. Yang Q, Tang J, Ahuja N. Efficient and Robust Specular Highlight Removal. IEEE Trans Pattern Anal Mach Intell. 2015;37(6):1304–1311. pmid:26357351
- View Article
- PubMed/NCBI
- Google Scholar
16. Dana KJ, van Ginneken B, Nayar SK, Koenderink JJ. Reflectance and Texture of Real-World Surfaces. ACM Trans Graphics. 1999;18(1):1–34.
- View Article
- Google Scholar
17. Torrance KE, Sparrow EM. Theory for off-specular reflection from roughened surfaces. J Opt Soc Am. 1967;57(9):1105–1114.
- View Article
- Google Scholar
18. Wolff LB. Polarization vision: a new sensory approach to image understanding. Image Vision Comput. 1997;15(2):81–93.
- View Article
- Google Scholar
19. Nayar SK, Fang XS, Boult T. Separation of reflection components using color and polarization. Int J Comput Vision. 1997;21(3):163–186.
- View Article
- Google Scholar
20. Wen S, Zheng Y, Lu F. Polarization Guided Specular Reflection Separation. IEEE Trans Image Process. 2021;30:7280–7291. pmid:34403344
- View Article
- PubMed/NCBI
- Google Scholar
21. Martin JA, Gross KC. Estimating index of refraction from polarimetric hyperspectral imaging measurements. Opt Express. 2016;24(16):17928–17940. pmid:27505760
- View Article
- PubMed/NCBI
- Google Scholar
22. Martínez-Domingo MA, Valero EM, Hernández-Andrés J, Tominaga S, Horiuchi T, Hirai K. Image processing pipeline for segmentation and material classification based on multispectral high dynamic range polarimetric images. Opt Express. 2017;25(24):30073–30090. pmid:29221042
- View Article
- PubMed/NCBI
- Google Scholar
23. Groner W, Winkelman JW, Harris AG, Ince C, Bouma GJ, Messmer K, et al. Orthogonal polarization spectral imaging: a new method for study of the microcirculation. Nat Med. 1999;5(10):1209–1212. pmid:10502828
- View Article
- PubMed/NCBI
- Google Scholar
24. Morgan SP, Stockford IM. Surface-reflection elimination in polarization imaging of superficial tissue. Opt Lett. 2003;28(2):114–116. pmid:12656502
- View Article
- PubMed/NCBI
- Google Scholar
25. Tominaga S, Kimachi A. Polarization imaging for material classification. Opt Eng. 2008;47(12):123201.
- View Article
- Google Scholar
26. Fischer C, Kakoulli I. Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Stud Conserv. 2006;51(sup1):3–16.
- View Article
- Google Scholar
27. Sattar S, Lapray PJ, Foulonneau A, Bigué L. Review of spectral and polarization imaging systems. Proc SPIE. 2020;11351:191–203.
- View Article
- Google Scholar
28. Manakov A, Restrepo JF, Klehm O, Hegedüs R, Eisemann E, Seidel HP, et al. A Reconfigurable Camera Add-on for High Dynamic Range, Multispectral, Polarization, and Light-Field Imaging. ACM Trans Graph. 2013;32(4).
- View Article
- Google Scholar
29. Shinoda K, Ohtera Y, Hasegawa M. Snapshot multispectral polarization imaging using a photonic crystal filter array. Opt Express. 2018;26(12):15948–15961. pmid:30114848
- View Article
- PubMed/NCBI
- Google Scholar
30. Sattar S, Lapray PJ, Aksas L, Foulonneau A, Bigué L. Snapshot spectropolarimetric imaging using a pair of filter array cameras. Opt Eng. 2022;61(4):043104.
- View Article
- Google Scholar
31. Le Hors L, Hartemann P, Breugnot S; International Society for Optics; Photonics. Multispectral polarization active imager in the visible band. Proc SPIE. 2000;4035:380–389.
- View Article
- Google Scholar
32. Umov N. Chromatische Depolarisation durch Lichtzerstreuung. Phys Zeitsch. 1905;6(20):674–676.
- View Article
- Google Scholar
33. Le Hors L, Hartemann P, Dolfi D, Breugnot S; International Society for Optics; Photonics. Phenomenological model of paints for multispectral polarimetric imaging. Proc SPIE. 2001;4370:94–105.
- View Article
- Google Scholar
34. Bartell FO, Dereniak EL, Wolfe WL. The theory and measurement of bidirectional reflectance distribution function (BRDF) and bidirectional transmittance distribution function (BTDF). Proc SPIE. 1981;257:154–160.
- View Article
- Google Scholar
35. Guarnera D, Guarnera GC, Ghosh A, Denk C, Glencross M. BRDF Representation and Acquisition. Comput Graphics Forum. 2016;35(2):625–650.
- View Article
- Google Scholar
36. Shafer SA. Using color to separate reflection components. Color Res Appl. 1985;10(4):210–218.
- View Article
- Google Scholar
37. Riviere J, Reshetouski I, Filipi L, Ghosh A. Polarization imaging reflectometry in the wild. ACM Trans Graph. 2017;36(6):1–14.
- View Article
- Google Scholar
38. Tyo JS. Design of optimal polarimeters: maximization of signal-to-noise ratio and minimization of systematic error. Appl Opt. 2002;41(4):619–630. pmid:11993905
- View Article
- PubMed/NCBI
- Google Scholar
39. Meadowlark Optics. Grid VersaLight polarizer;. (2019), https://www.meadowlark.com/wire-grid-versalight-polarizer/.
40. Lenhard K, Baumgartner A, Schwarzmaier T. Independent laboratory characterization of NEO HySpex imaging spectrometers VNIR-1600 and SWIR-320m-e. IEEE Trans Geosci Remote Sens. 2014;53(4):1828–1841.
- View Article
- Google Scholar
41. Khan HA, Mihoubi S, Mathon B, Thomas JB, Hardeberg JY. HyTexiLa: High resolution visible and near infrared hyperspectral texture images. Sensors. 2018;18(7):2045. pmid:29949948
- View Article
- PubMed/NCBI
- Google Scholar
42. Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004;60(2):91–110.
- View Article
- Google Scholar
43. Conover DM, Delaney JK, Loew MH. Automatic registration and mosaicking of technical images of Old Master paintings. Appl Phys A. 2015;119:1567–1575.
- View Article
- Google Scholar
44. Sony Semiconductor Solution Group, https://www.sony-semicon.com/files/62/flyer_industry/IMX250_264_253MZR_MYR_Flyer_en.pdf, 2018.
45. Hueni A, Bialek A. Cause, effect, and correction of field spectroradiometer interchannel radiometric steps. IEEE J Sel Top Appl Earth Obs Remote Sens. 2017;10(4):1542–1551.
- View Article
- Google Scholar
46. Grillini F, Thomas JB, George S. Logistic splicing correction for VNIR–SWIR reflectance imaging spectroscopy. Opt Lett. 2023;48(2):403–406. pmid:36638468
- View Article
- PubMed/NCBI
- Google Scholar
47. Alouini M., Goudail F., Roux N., Le Hors L., Hartemann P., Breugnot S., et al. Active spectro-polarimetric imaging: signature modeling, imaging demonstrator and target detection The European Physical Journal-Applied Physics. 2008; 42(2):129–139.
- View Article
- Google Scholar
48. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. IEEE Trans Pattern Anal Mach Intell. 2012;34(11):2274–2282. pmid:22641706
- View Article
- PubMed/NCBI
- Google Scholar
49. Liu G, Fomel S, Jin L, Chen X. Stacking seismic data using local correlation. Geophysics. 2009;74(3):V43–V48.
- View Article
- Google Scholar
50. Daffara C, Fontana R. Multispectral infrared reflectography to differentiate features in paintings. Microsc Microanal. 2011;17(5):691–695. pmid:21473807
- View Article
- PubMed/NCBI
- Google Scholar

[ref1] 1. Di Cicco F, Wijntjes MW, Pont SC. Understanding gloss perception through the lens of art: Combining perception, image analysis, and painting recipes of 17th century painted grapes. J Vision. 2019;19(3):7–7. pmid:30897625
View Article
PubMed/NCBI
Google Scholar

[2] View Article

[3] PubMed/NCBI

[4] Google Scholar

[ref2] 2. Hubona GS, Wheeler PN, Shirah GW, Brandt M. The relative contributions of stereo, lighting, and background scenes in promoting 3D depth visualization. ACM T Comput-Hum Int. 1999;6(3):214–242.
View Article
Google Scholar

[6] View Article

[7] Google Scholar

[ref3] 3. Henry RC, Mahadev S, Urquijo S, Chitwood D. Color perception through atmospheric haze. J Opt Soc Am A. 2000;17(5):831–835. pmid:10795630
View Article
PubMed/NCBI
Google Scholar

[9] View Article

[10] PubMed/NCBI

[11] Google Scholar

[ref4] 4. Wong KYK, Schnieders D, Li S. Recovering light directions and camera poses from a single sphere. In: Computer Vision–ECCV 2008: 10th European Conference on Computer Vision; 2008. p. 631–642.

[ref5] 5. Wedge-Roberts R, Aston S, Beierholm U, Kentridge R, Hurlbert A, Nardini M, et al. Specular highlights improve color constancy when other cues are weakened. J Vision. 2020;20(12):4–4. pmid:33170203
View Article
PubMed/NCBI
Google Scholar

[14] View Article

[15] PubMed/NCBI

[16] Google Scholar

[ref6] 6. Lopatin J, Dolos K, Kattenborn T, Fassnacht FE. How canopy shadow affects invasive plant species classification in high spatial resolution remote sensing. Rem Sens Ecol Conserv. 2019;5(4):302–317.
View Article
Google Scholar

[18] View Article

[19] Google Scholar

[ref7] 7. Ahmad A, Quegan S. The effects of haze on the accuracy of satellite land cover classification. Appl Math Sc. 2015;9(49):2433–2443.
View Article
Google Scholar

[21] View Article

[22] Google Scholar

[ref8] 8. Elkhuizen W, Essers T, Song Y, Geraedts J, Weijkamp C, Dik J, et al. Gloss, color, and topography scanning for reproducing a Painting’s appearance using 3D printing. J Comput Cult Herit. 2019;12(4):1–22.
View Article
Google Scholar

[24] View Article

[25] Google Scholar

[ref9] 9. Giachetti A, Ciortan I, Daffara C, Pintus R, Gobbetti E. Multispectral RTI analysis of heterogeneous artworks. In: Proceedings of the Eurographics Workshop on Graphics and Cultural Heritage; 2017. p. 19–28.

[ref10] 10. Colombini MP, Modugno F, Giannarelli S, Fuoco R, Matteini M. GC-MS characterization of paint varnishes. Microchem J. 2000;67(1-3):385–396.
View Article
Google Scholar

[28] View Article

[29] Google Scholar

[ref11] 11. Hasegawa T, Tsumura N, Nakaguchi T, Iino K. Photometric approach to surface reconstruction of artist paintings. J Electron Imaging. 2011;20(1):013006–013006.
View Article
Google Scholar

[31] View Article

[32] Google Scholar

[ref12] 12. Wolff LB, Boult TE. Constraining object features using a polarization reflectance model. Phys Based Vis Princ Pract Radiom. 1993;1:167.
View Article
Google Scholar

[34] View Article

[35] Google Scholar

[ref13] 13. Nayar SK, Ikeuchi K, Kanade T. Surface reflection: physical and geometrical perspectives. IEEE Trans Pattern Anal Mach Intell. 1991;13(7):611–634.
View Article
Google Scholar

[37] View Article

[38] Google Scholar

[ref14] 14. Tao MW, Su JC, Wang TC, Malik J, Ramamoorthi R. Depth Estimation and Specular Removal for Glossy Surfaces Using Point and Line Consistency with Light-Field Cameras. IEEE Trans Pattern Anal Mach Intell. 2016;38(6):1155–1169. pmid:26372203
View Article
PubMed/NCBI
Google Scholar

[40] View Article

[41] PubMed/NCBI

[42] Google Scholar

[ref15] 15. Yang Q, Tang J, Ahuja N. Efficient and Robust Specular Highlight Removal. IEEE Trans Pattern Anal Mach Intell. 2015;37(6):1304–1311. pmid:26357351
View Article
PubMed/NCBI
Google Scholar

[44] View Article

[45] PubMed/NCBI

[46] Google Scholar

[ref16] 16. Dana KJ, van Ginneken B, Nayar SK, Koenderink JJ. Reflectance and Texture of Real-World Surfaces. ACM Trans Graphics. 1999;18(1):1–34.
View Article
Google Scholar

[48] View Article

[49] Google Scholar

[ref17] 17. Torrance KE, Sparrow EM. Theory for off-specular reflection from roughened surfaces. J Opt Soc Am. 1967;57(9):1105–1114.
View Article
Google Scholar

[51] View Article

[52] Google Scholar

[ref18] 18. Wolff LB. Polarization vision: a new sensory approach to image understanding. Image Vision Comput. 1997;15(2):81–93.
View Article
Google Scholar

[54] View Article

[55] Google Scholar

[ref19] 19. Nayar SK, Fang XS, Boult T. Separation of reflection components using color and polarization. Int J Comput Vision. 1997;21(3):163–186.
View Article
Google Scholar

[57] View Article

[58] Google Scholar

[ref20] 20. Wen S, Zheng Y, Lu F. Polarization Guided Specular Reflection Separation. IEEE Trans Image Process. 2021;30:7280–7291. pmid:34403344
View Article
PubMed/NCBI
Google Scholar

[60] View Article

[61] PubMed/NCBI

[62] Google Scholar

[ref21] 21. Martin JA, Gross KC. Estimating index of refraction from polarimetric hyperspectral imaging measurements. Opt Express. 2016;24(16):17928–17940. pmid:27505760
View Article
PubMed/NCBI
Google Scholar

[64] View Article

[65] PubMed/NCBI

[66] Google Scholar

[ref22] 22. Martínez-Domingo MA, Valero EM, Hernández-Andrés J, Tominaga S, Horiuchi T, Hirai K. Image processing pipeline for segmentation and material classification based on multispectral high dynamic range polarimetric images. Opt Express. 2017;25(24):30073–30090. pmid:29221042
View Article
PubMed/NCBI
Google Scholar

[68] View Article

[69] PubMed/NCBI

[70] Google Scholar

[ref23] 23. Groner W, Winkelman JW, Harris AG, Ince C, Bouma GJ, Messmer K, et al. Orthogonal polarization spectral imaging: a new method for study of the microcirculation. Nat Med. 1999;5(10):1209–1212. pmid:10502828
View Article
PubMed/NCBI
Google Scholar

[72] View Article

[73] PubMed/NCBI

[74] Google Scholar

[ref24] 24. Morgan SP, Stockford IM. Surface-reflection elimination in polarization imaging of superficial tissue. Opt Lett. 2003;28(2):114–116. pmid:12656502
View Article
PubMed/NCBI
Google Scholar

[76] View Article

[77] PubMed/NCBI

[78] Google Scholar

[ref25] 25. Tominaga S, Kimachi A. Polarization imaging for material classification. Opt Eng. 2008;47(12):123201.
View Article
Google Scholar

[80] View Article

[81] Google Scholar

[ref26] 26. Fischer C, Kakoulli I. Multispectral and hyperspectral imaging technologies in conservation: current research and potential applications. Stud Conserv. 2006;51(sup1):3–16.
View Article
Google Scholar

[83] View Article

[84] Google Scholar

[ref27] 27. Sattar S, Lapray PJ, Foulonneau A, Bigué L. Review of spectral and polarization imaging systems. Proc SPIE. 2020;11351:191–203.
View Article
Google Scholar

[86] View Article

[87] Google Scholar

[ref28] 28. Manakov A, Restrepo JF, Klehm O, Hegedüs R, Eisemann E, Seidel HP, et al. A Reconfigurable Camera Add-on for High Dynamic Range, Multispectral, Polarization, and Light-Field Imaging. ACM Trans Graph. 2013;32(4).
View Article
Google Scholar

[89] View Article

[90] Google Scholar

[ref29] 29. Shinoda K, Ohtera Y, Hasegawa M. Snapshot multispectral polarization imaging using a photonic crystal filter array. Opt Express. 2018;26(12):15948–15961. pmid:30114848
View Article
PubMed/NCBI
Google Scholar

[92] View Article

[93] PubMed/NCBI

[94] Google Scholar

[ref30] 30. Sattar S, Lapray PJ, Aksas L, Foulonneau A, Bigué L. Snapshot spectropolarimetric imaging using a pair of filter array cameras. Opt Eng. 2022;61(4):043104.
View Article
Google Scholar

[96] View Article

[97] Google Scholar

[ref31] 31. Le Hors L, Hartemann P, Breugnot S; International Society for Optics; Photonics. Multispectral polarization active imager in the visible band. Proc SPIE. 2000;4035:380–389.
View Article
Google Scholar

[99] View Article

[100] Google Scholar

[ref32] 32. Umov N. Chromatische Depolarisation durch Lichtzerstreuung. Phys Zeitsch. 1905;6(20):674–676.
View Article
Google Scholar

[102] View Article

[103] Google Scholar

[ref33] 33. Le Hors L, Hartemann P, Dolfi D, Breugnot S; International Society for Optics; Photonics. Phenomenological model of paints for multispectral polarimetric imaging. Proc SPIE. 2001;4370:94–105.
View Article
Google Scholar

[105] View Article

[106] Google Scholar

[ref34] 34. Bartell FO, Dereniak EL, Wolfe WL. The theory and measurement of bidirectional reflectance distribution function (BRDF) and bidirectional transmittance distribution function (BTDF). Proc SPIE. 1981;257:154–160.
View Article
Google Scholar

[108] View Article

[109] Google Scholar

[ref35] 35. Guarnera D, Guarnera GC, Ghosh A, Denk C, Glencross M. BRDF Representation and Acquisition. Comput Graphics Forum. 2016;35(2):625–650.
View Article
Google Scholar

[111] View Article

[112] Google Scholar

[ref36] 36. Shafer SA. Using color to separate reflection components. Color Res Appl. 1985;10(4):210–218.
View Article
Google Scholar

[114] View Article

[115] Google Scholar

[ref37] 37. Riviere J, Reshetouski I, Filipi L, Ghosh A. Polarization imaging reflectometry in the wild. ACM Trans Graph. 2017;36(6):1–14.
View Article
Google Scholar

[117] View Article

[118] Google Scholar

[ref38] 38. Tyo JS. Design of optimal polarimeters: maximization of signal-to-noise ratio and minimization of systematic error. Appl Opt. 2002;41(4):619–630. pmid:11993905
View Article
PubMed/NCBI
Google Scholar

[120] View Article

[121] PubMed/NCBI

[122] Google Scholar

[ref39] 39. Meadowlark Optics. Grid VersaLight polarizer;. (2019), https://www.meadowlark.com/wire-grid-versalight-polarizer/.

[ref40] 40. Lenhard K, Baumgartner A, Schwarzmaier T. Independent laboratory characterization of NEO HySpex imaging spectrometers VNIR-1600 and SWIR-320m-e. IEEE Trans Geosci Remote Sens. 2014;53(4):1828–1841.
View Article
Google Scholar

[125] View Article

[126] Google Scholar

[ref41] 41. Khan HA, Mihoubi S, Mathon B, Thomas JB, Hardeberg JY. HyTexiLa: High resolution visible and near infrared hyperspectral texture images. Sensors. 2018;18(7):2045. pmid:29949948
View Article
PubMed/NCBI
Google Scholar

[128] View Article

[129] PubMed/NCBI

[130] Google Scholar

[ref42] 42. Lowe DG. Distinctive image features from scale-invariant keypoints. Int J Comput Vision. 2004;60(2):91–110.
View Article
Google Scholar

[132] View Article

[133] Google Scholar

[ref43] 43. Conover DM, Delaney JK, Loew MH. Automatic registration and mosaicking of technical images of Old Master paintings. Appl Phys A. 2015;119:1567–1575.
View Article
Google Scholar

[135] View Article

[136] Google Scholar

[ref44] 44. Sony Semiconductor Solution Group, https://www.sony-semicon.com/files/62/flyer_industry/IMX250_264_253MZR_MYR_Flyer_en.pdf, 2018.

[ref45] 45. Hueni A, Bialek A. Cause, effect, and correction of field spectroradiometer interchannel radiometric steps. IEEE J Sel Top Appl Earth Obs Remote Sens. 2017;10(4):1542–1551.
View Article
Google Scholar

[139] View Article

[140] Google Scholar

[ref46] 46. Grillini F, Thomas JB, George S. Logistic splicing correction for VNIR–SWIR reflectance imaging spectroscopy. Opt Lett. 2023;48(2):403–406. pmid:36638468
View Article
PubMed/NCBI
Google Scholar

[142] View Article

[143] PubMed/NCBI

[144] Google Scholar

[ref47] 47. Alouini M., Goudail F., Roux N., Le Hors L., Hartemann P., Breugnot S., et al. Active spectro-polarimetric imaging: signature modeling, imaging demonstrator and target detection The European Physical Journal-Applied Physics. 2008; 42(2):129–139.
View Article
Google Scholar

[146] View Article

[147] Google Scholar

[ref48] 48. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S. SLIC Superpixels Compared to State-of-the-Art Superpixel Methods. IEEE Trans Pattern Anal Mach Intell. 2012;34(11):2274–2282. pmid:22641706
View Article
PubMed/NCBI
Google Scholar

[149] View Article

[150] PubMed/NCBI

[151] Google Scholar

[ref49] 49. Liu G, Fomel S, Jin L, Chen X. Stacking seismic data using local correlation. Geophysics. 2009;74(3):V43–V48.
View Article
Google Scholar

[153] View Article

[154] Google Scholar

[ref50] 50. Daffara C, Fontana R. Multispectral infrared reflectography to differentiate features in paintings. Microsc Microanal. 2011;17(5):691–695. pmid:21473807
View Article
PubMed/NCBI
Google Scholar

[156] View Article

[157] PubMed/NCBI

[158] Google Scholar

Figures

Abstract

1 Introduction

2 Background

2.1 Spectral and polarization imaging model

2.2 Stokes vector

3 Materials and methods

3.1 Experimental set-up

3.2 Spectral data preprocessing

3.3 Polarimetric calibration

3.4 Spectro-polarimetric splicing

3.5 Correlation between reflectance and degree of linear polarization

4 Results and discussion

4.1 Decision on the number of superpixels

4.2 Global correlation: Scale and wavelength dependence

5 Conclusion

References