Fig 1.
The intensity histogram of the original image (a) is matched with that of the reference image (b) in the histogram-matched image (c). The geometric model of (a) is "Julius Caesar" designed by Yousef Mansy (https://pinshape.com/items/25809-3d-printed-julius-caesar-scan-the-world), and that of (b) is "Venus sculpture" designed by SHINING 3D (https://pinshape.com/items/19446-3d-printed-venus-sculpture). The cumulative probability densities of pixels of the original and reference image are shown in (d) and (e), respectively. (f) Pixel intensity of the histogram-matched image plotted as a function of that of the original image.
Fig 2.
(a) The image is decomposed into its intrinsic components: albedo, shading, and specular images. Whereas the reflectance image is independent of the shading image, the specular image is dependent on the shading image. This is because the shading intensity is a function of the incident angle of light, while the specular intensity is a function of the incident and viewing angles of the light. (b) The intensities of the shading image within the highlight regions tend to be uniform because both specular and shading intensity depend on the incident angle of light. (c) Intensity gradient maps and the direction maps of the intensity gradient obtained from input images. The magnitude and direction of the vector are indicated as the hue and saturation of a color map, respectively.
Fig 3.
Stimulus examples used in the image analysis.
The bumpy surfaces were rendered using the BRDFs in the MERL database (100 BRDFs). The left panel shows the images rendered under a point light source with a slant of 0°, and the right panel shows the images rendered under an HDR environment map. Each panel includes 100 images rendered with different BRDFs. The same geometry is used for the rendered images in the left and right panels. From each rendered image, the direction and the magnitude maps of the intensity gradient were computed.
Fig 4.
Probably density distributions of correlation coefficients between all pairs of BRDFs.
(a) The left and right panels show the correlation coefficients on the direction and the magnitude of the intensity gradient of the rendered image, respectively. For the direction condition, the circular correlation was used [29, 30]. Each cell of the panels indicates the correlation coefficient between the BRDFs of rows and columns. The numbers from #1 to #100 indicate the index of MERL BRDFs. (b) The probability density distribution of the correlation coefficients. The top panel indicates the correlation of the direction of the intensity gradient. The bottom panel indicates the correlation of the magnitude of the intensity gradient. When the point light source lit the objects from the viewing direction, the direction of the intensity gradient consistently shows quite a high correlation, whereas the magnitude of the intensity gradient shows correlations that are relatively low and highly variable depending on the comparison pair.
Fig 5.
Probability density distributions under the lighting conditions of (a) 20° and (b) 40°. The displacements of the lighting direction do not significantly change the pattern of the correlation distributions.
Fig 6.
(a) Probability density distributions in the HDR environment map. The correlation in the direction of the intensity gradient is reduced for some material combinations. (b) The correlation of the image in the HDR environment map was calculated after applying the compressive tone-mapping of Eq (1) to the surface image. As a consequence, the correlation in the direction of the intensity gradient is markedly improved.
Fig 7.
Probability density distributions of the correlation between the lighting conditions of 0° and 40°.
The correlations in the direction of the intensity gradient are much lower compared with those under the identical illumination environment condition (Figs 4–6).
Fig 8.
A hypothetical processing scheme that the human visual system may use for simultaneous estimation of a variety of surface properties.
We hypothesize that human shape processing is more sensitive to image features given by the intensity order information than those conveyed by the gradient magnitude information, while the magnitude information is dominantly used for material processing.
Fig 9.
The skewness of the intensity histogram of each object image with highlights was modulated using a standard histogram matching method.
Fig 10.
(a) Rating results for the glossiness judgment plotted as a function of the skewness of the intensity histogram. Error bars indicate ± 1 SEM across observers. (b) The perceived tilt (left) or slant (right) for the histogram-modulated object plotted as a function of those for each original object. Different symbols indicate different skew parameters as shown in the legend.
Fig 11.
Stimuli used in Experiment 2a.
(a) We applied a variety of monotonic and nonmonotonic nonlinear remappings to several object images. When the slope was steep (right), the intensity order of the original image was not disrupted by the modulation regardless of its amplitude. In contrast, when the slope was gentle (left), the intensity order was disrupted by the modulation unless the modulation amplitude was small. When the intensity order of the original image was disrupted (left bottom), the shape of the image was perceived differently from the original one (middle bottom). (b) The three object images used in Experiment 2. The gauge probes on the image show the position measured in the experiment.
Fig 12.
The angular difference between the judgments on tone-mapped and original images is plotted as a function of the amplitude of the sinusoidal modulation. Error bars indicate 95% confidence intervals computed using bootstrap estimates. Different symbols indicate different slopes as indicated in the legend. The orange (solid) horizontal line shows the mean angular difference of matched gauges for the same original object measured in different sessions (i.e., the control condition). Magenta (dashed) horizontal lines indicate 95% confidence intervals of the control condition computed using bootstrap estimates. For slope 1, the intensity order of the original image was disrupted when the sinusoidal modulation was 0.115 or 0.165. For slope 0.5, the intensity order was disrupted when the sinusoidal modulation was 0.065, 0.115 or 0.165. For slope 2, the modulations did not change the intensity order. Results show that the large deviations of the perceived shape were obtained when the intensity order of the original image was markedly disrupted.
Fig 13.
Stimuli used in Experiment 2b.
We applied a variety of monotonic and nonmonotonic remappings to the object images under the lighting conditions of slant 0° (left) and slant 45° (right). When the slope of the remapping was steep (i.e., slope = 2), the intensity order of the original image was not disrupted by the amplitude modulation (0.165) as in Experiment 2. In contrast, when the slope was gentle (i.e., slope = 0.5), the intensity order was disrupted by the modulation. Object 4 was used in the experiment. The six gauge probes as shown in the gauge position figures were used.
Fig 14.
The angular difference between the judgments on tone-mapped and original images is plotted as a function of the amplitude of the sinusoidal modulation. Error bars indicate 95% confidence intervals computed using bootstrap estimates. Different symbols indicate different slopes as indicated in the legend of Fig 12. For both illumination conditions, results show that large deviations in the perceived shape were obtained when amplitude modulation was added to the remapping with a slope 0.5, i.e., when the intensity order of the original image was disrupted.
Fig 15.
Relation between pixel intensities of Lambertian and asperity surfaces.
We assume that the incident and reflected angles of the illumination are the same. Each plot of the LA is scaled from 0 to the max intensity of the Ll. The intensity order of each plot changes with parameter a. For instance, when parameter a is 0.2, the intensity order is preserved in the lower range of Lambertian pixel intensity, while it is reversed in the higher range. When a is 0.02, the intensity order is reversed across most of the intensity range.
Fig 16.
Stimulus examples used in Experiment 4.
Upper images indicate the experimental stimuli with different BRDFs as in the legend. Lower images indicate the directional map of each image.
Fig 17.
(a) The scatter plots of the perceived surface orientation between the normal and reversed Lambertian conditions. Left and right panels show the results for the tilt and slant, respectively. Each plot indicates the averaged judgment across trials within each observer for each gauge position. The circular correlation [29, 30] and Pearson’s correlation coefficients are shown in the tilt and slant panels, respectively. (b) The gauge positions in the stimulus. Red colors in each figure show the positions where the direction of the intensity gradient on the stimulus condition was the same as that on the Lambertian object. Green in each figure shows the positions where the direction of the intensity gradient on the stimulus condition was opposite to that on the Lambertian object. (c) The angular difference between the judgments on the two asperity conditions and the Lambertian condition. The horizontal axis indicates the gauge number, which is shown in the Fig 17(B).
Fig 18.
Stimuli and results of the additional experiment of Experiment 3.
(a) The same stimulus set as Experiment 3 under a different lighting condition (illumination slant = 45) is used in the experiment. The directional distortion of intensity gradients from the Lambertian object image is shown in the bottom of each stimulus on the red-green axis. (b) The angular difference between the judgments on the two asperity conditions and the Lambertian condition. The horizontal axis indicates the gauge number.
Fig 19.
An object with inconsistent highlights was made by combining the diffuse pattern with the rotated and displaced specular pattern (bottom). The skewness of the intensity histogram of the object image was modulated using a standard histogram matching method. The glossy objects with veridical highlights are also shown (upper). They are the same stimuli as used in Experiment 1 (Fig 9).
Fig 20.
(a) Ratings for the glossiness judgment and (b) for the non-uniformity judgment are plotted as a function of the skewness of the intensity histogram. Stimulus conditions are shown as in the legend. Error bars indicate ± 1 SEM across observers. The ratings for the object with veridical highlights in Experiment 1 (Fig 10(A)) are plotted again in the glossiness judgment (red).
Fig 21.
Stimulus conditions in Experiment 5.
(a) Three object images (Stanford bunny, low frequency bump, and 1/f bump) were used. In addition to veridical rendering for the object, object images with inconsistent specular highlights were created. The object images were rendered under (a) the point light source or (b) the HDR environment map.
Fig 22.
Effect of tone-remapping on the appearance of the material and albedo.
We changed the cut-off intensity of compressive tone-remapping applied to several object images with veridical or inconsistent highlights.
Fig 23.
The rating values for the glossiness (top) and non-uniformity (bottom) judgments are plotted as a function of the cut-off point. The rating values for the object condition were pooled across observers. The column panels indicate the lighting condition (point or environment). Error bars indicate ± 1 SEM across observers.
Fig 24.
Demonstration of detecting albedo changes based on the intensity gradient.
(a) Veridical highlights have little effect on the direction map, i.e., the map for the veridical highlight was similar to that for the matte one. In contrast, inconsistent highlights mark an abrupt change in the direction map. (b) When one of the intrinsic image decomposition algorithms [58] was applied to the object images with veridical and inconsistent highlights (left), even for the uniform albedo image with veridical highlights it incorrectly detects the highlights as different albedos. The estimated albedo images are shown at the bottom of the figure. In contrast, when it was applied to the slope-normalized image (right), the image with veridical highlights can be regarded as a uniform albedo.