Fig 1.
Overview of the Spatiochromatic Bandwidth Limited (SBL) model.
The model uses a cone-catch image (a, S1 Appendix), which is filtered by either DoG or Gabor kernels for luminance channels, and DoG kernels for chromatic channels (b). Contrasts are converted to Michelson contrasts (c. showing luminance DoG outputs), then clipping and gain processes are applied with a bandwidth (ε) of 10 (d. Fig 2), and the spatial filters are pooled to create the output (e). Output colours are the model’s internal representation and are not scaled to sRGB space. However, we note that the output image has qualities that combine the effects of an impressionist artist’s take on the scene that compresses the contrasts and highlights chromatic features such as the “carpet of bluebells” that observers describe, but are much weaker in the input image. Also noteworthy is that the model would produce the same overall green scene with blue flowers irrespective of the input image’s white balance.
Fig 2.
Dynamic range clipping and gain adjustment by the SBL model.
a) human luminance and chromatic detection thresholds for sinewave gratings [37]. b) Clipping adjusts contrasts so that they cannot fall below the CSF at each spatial frequency (α, SUBTHRESHOLD), or above the saturation threshold (β, SATURATED). Subthreshold contrasts are subtracted, and signals at each spatial frequency are multiplied by a gain value—denoted by arrow length in (c)—so that on average natural images have equal power at each spatial frequency (whitening). The saturation threshold is calculated from the CSF and channel bandwidth, ε (4 in this example) at each spatial frequency. High and low spatial frequency channels therefore have low contrast sensitivity, but encode a large range of image contrasts, whereas intermediate spatial frequencies have high sensitivity and a low dynamic range. To demonstrate the clipping effects, we show an input image with sinewaves of different spatial frequencies and contrasts (d). (e) shows bandpass spatial filters and (f) highlights regions that are clipped or preserved. The overlap between neighbouring octaves (f) means that where contrasts are saturated for one channel, they are unlikely to be saturated for all neighbouring channels so that contrast differences are detectable even in high contrast scenes. Ultimately this shows how a system with a severely limited neural bandwidth of 15 contrast levels and peak sensitivity of ~200:1 can code for contrasts in natural scenes larger than 10,000:1. Note that the fine lines in these illustrative images suffer from moiré effects when viewed on a monitor, and we have artificially blurred the higher spatial frequencies in the input and output images to mitigate this effect. These effects were not present in the modelling, which did not use spatial frequencies that exceeded the kernel’s peak sensitivity.
Fig 3.
Fitting the SBL model to behavioural and neurophysiological data.
a) fit to Whittle’s crispening data ([8], Fig 9, “25/inc-dec/gray” treatment). Model output is scaled to the same 0–25 range. The best-fitting bandwidth (ε) for DoG filters is 15, and for Gabor (oriented) filters is 3.75, both of which result in a good fit to the raw data. The CIE L* fit specifies lightness in psychophysics and does not account for contrast [80]. b) Model fit to single ganglion response data from Derrington and Lennie ([47], Fig 11B). Fitting used a single free parameter that multiplied the arbitrary SBL model output to match neural firing responses (with zero intercept) by least-squares regression. The SBL model shows a linear contrast response and saturation point that provide a better fit than the authors’ model. The inset excludes the three highest contrast values to highlight the linear relationship prior to saturation.
Fig 4.
Illustration of dynamic range clipping by the SBL model.
(a) for the crispening effect [8]. The three rows of grey levels are identical, with equal step sizes. Against the black background contrasts appear largest for darker squares, whereas the opposite is true for the white background. The SBL model explains this effect through saturation; contrasts near the grey level of the local surroundings are preserved (highlighted with circles), while other contrasts are saturated (blue areas adjacent to the highlighted areas). The graph at the bottom plots differences between adjacent squares in the three rows, showing higher contrasts for dark, middle and light ranges respectively. Illusions such as the Chevreul staircase (b) are also explained in part by clipping. The upper staircase appears to be a series of square steps in grey level. The lower staircase has the same grey levels, but is flipped so that its gradient matches the surround gradient. The SBL model correctly predicts that the upper staircase is seen as square steps in grey level (solid green line) while the lower staircase is a series of gradients (dashed grey line). The plot shows pixel values in arbitrary units measured along each staircase, as highlighted in the output image. The model shows that this effect arises partly because the matched gradients of the lower staircase causes local subthreshold contrasts, and because contrasts are not balanced on each side of the step. (c) Shows the effects of increasing background contrast on two identical targets. At intermediate contrasts (~0.1–0.7) the targets are predicted to show simultaneous contrast effects (the right-hand target appears lighter than the left-hand one), and at higher surround contrasts this is predicted to switch to the White illusion (spreading) effect where the right-hand target becomes darker than the left-hand one.
Table 1.
Summary of phenomena tested with oriented and non-oriented versions of the SBL model, with the parameters, α, β and ε fixed as explained in the text.
All phenomena were qualitatively explained to some degree. For illustrations of specific effects see the S1 Appendix.
Fig 5.
The SBL model can account for colour appearance in complex naturalistic images.
(a) shows the input image (similar to the Lotto cube [81]) where the blue squares on the yellow-tinted side (right) and the yellow squares on the blue-tinted side (left) are physically the same grey (colours are shown in the squares at the top of the image). The SBL model (b) correctly predicts that the squares under both tinting regimes appear yellow and blue, rather than grey. The SBL model also predicts the powerful simultaneous contrast (or shadow) illusion present in this image whereby; the central tiles on top of the cube appear to be darker than the central tiles on the shaded side of the cube (colours shown in squares on the far left and for right hand sides).