https://orcid.org/0000-0001-9071-2594
Figures
Abstract
An object’s colour, brightness and pattern are all influenced by its surroundings, and a number of visual phenomena and “illusions” have been discovered that highlight these often dramatic effects. Explanations for these phenomena range from low-level neural mechanisms to high-level processes that incorporate contextual information or prior knowledge. Importantly, few of these phenomena can currently be accounted for in quantitative models of colour appearance. Here we ask to what extent colour appearance is predicted by a model based on the principle of coding efficiency. The model assumes that the image is encoded by noisy spatio-chromatic filters at one octave separations, which are either circularly symmetrical or oriented. Each spatial band’s lower threshold is set by the contrast sensitivity function, and the dynamic range of the band is a fixed multiple of this threshold, above which the response saturates. Filter outputs are then reweighted to give equal power in each channel for natural images. We demonstrate that the model fits human behavioural performance in psychophysics experiments, and also primate retinal ganglion responses. Next, we systematically test the model’s ability to qualitatively predict over 50 brightness and colour phenomena, with almost complete success. This implies that much of colour appearance is potentially attributable to simple mechanisms evolved for efficient coding of natural images, and is a well-founded basis for modelling the vision of humans and other animals.
Author summary
An object’s brightness and colour are not just due to its own surface properties, but also depend on the colours and patterns of its surrounds. We set out to develop a computational model that could predict colour appearance based on the principle of efficient coding. This takes into account the fact that neural bandwidth is limited (e.g. the fastest rate a neurone can fire might only be ten times its lowest rate), and that none of this valuable bandwidth should be wasted when coding information across different spatial scales in a typical natural scene. We next combined these principles with contrast sensitivity functions (because contrast detection thresholds vary with spatial scale), and used either psychophysical or neurophysiological data to estimate the bandwidth for humans/primates. When we tested the model against a bank of visual phenomena (illusions) we found that the model was able to predict the direction of almost all phenomena. Our model is surprisingly simple and generalisable, with no free parameters, and would be explained by low-level feed-forward neural architecture. This suggests that many complex visual phenomena–that have often attributed to high-level processes–could arise as artefacts of limited bandwidth and efficient coding, offering valuable avenues for future research.
Citation: Troscianko J, Osorio D (2023) A model of colour appearance based on efficient coding of natural images. PLoS Comput Biol 19(6): e1011117. https://doi.org/10.1371/journal.pcbi.1011117
Editor: Wolfgang Einhäuser, Technische Universitat Chemnitz, GERMANY
Received: April 8, 2022; Accepted: April 20, 2023; Published: June 15, 2023
Copyright: © 2023 Troscianko, Osorio. This is an open access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.
Data Availability: An implementation of the SBL model is provided here: https://www.biorxiv.org/content/biorxiv/early/2022/02/23/2022.02.22.481414/DC1/embed/media-1.zip?download=true.
Funding: JT was funded by a NERC IRF (NE/P018084/1). The funder had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.
Competing interests: The authors have declared that no competing interests exist.
This is a PLOS Computational Biology Methods paper.
Introduction
The colour and lightness of objects cannot be recovered directly from the retinal image of a scene, but depend upon neural processing by low-level spatial filters and feature detectors along with long-range and top-down mechanisms that incorporate contextual information and prior knowledge about the visual world [1–4]. Ideally, image processing achieves lightness and colour constancy–allowing us to see colour and form veridically–but inevitably it produces visual effects and illusions, which give insight into the underlying mechanisms. Thus, the surroundings of an object affect its lightness or colour in several ways. For example, assimilation and induction effects shift appearance towards that of neighbouring colours [5], whereas simultaneous contrast increases the difference between an object and the surround, and in contrast induction the surround affects the contrast of a pattern [6,7]. The crispening effect–where contrasts close to the background level are enhanced–encompasses all three of these phenomena [8,9]. Related effects in colour vision include the Abney, Bezold–Brücke, Hunt, and Stevens effects, where colours, colourfulness and contrasts shift with saturation and brightness [10].
Neural mechanisms have been proposed to account for some of the foregoing phenomena, for example Mach Bands can be attributed to lateral inhibition [11], brightness induction to spatial filtering in the primary visual cortex [12], and colour constancy to photoreceptor adaptation [13,14] or to cortical processing [15]–but these accounts are controversial, and some effects are not easily explained [7,8,16]. Moreover, the lack of a comprehensive account of colour appearance limits the accuracy of the models that are used in design, industry and research [10,17].
Although photoreceptor adaptation and lateral inhibition do partly account for colour constancy and simultaneous contrast effects, their primary function is probably better understood as allowing the visual system to efficiently encode images of natural scenes, which have a large dynamic range and a high degree of statistical redundancy. Coding efficiency, which allows the brain to make optimal use of limited neural bandwidth and metabolic energy, is a key principle in early visual processing [18–22]; here we ask how a model based on this principle might account for colour appearance.
The optimal (maximum entropy) code for natural images, as specified by their spatial autocorrelation function (i.e. second-order image statistics), approximates a Fourier transform [23,24], which is physiologically unrealistic. Efficient codes can however be defined for circularly symmetrical Difference of Gaussian (DoG) or oriented Gabor-function filters, which respectively resemble the receptive fields of retinal ganglion cells and the simple cells of mammalian visual cortex [22,25–27]. In early studies, Laughlin and his co-workers [20,28] found that the contrast response functions and the centre-surround receptive fields of fly (Lucilia vicina) large monopolar cell (LMC) neurons—which are directly post synaptic to the photoreceptors—produce an efficient representation of natural images for the noise present the insect’s photoreceptor responses. Specifically, synaptic amplification at the receptor to LMC synapse and lateral inhibition between receptor outputs, give a neural code that quantitatively accords with the methods of histogram equalization and predictive coding that are used by data compression algorithms [28]. The centre-surround receptive fields of vertebrate retinal ganglion cells are comparable to those of fly monopolar cells [29], while the simple cells in visual cortex generate an efficient code for natural image statistics [22,30].
Our aim here is not to simulate biological vision precisely, but to model efficient coding by physiologically plausible spatial filters. We describe a Spatiochromatic Bandwidth Limited (SBL) model of early vision, which uses luminance and chromatic spatial filters at octave separations to cover the detectable range of spatial frequencies (Figs 1–3). Three parameters specify the model, namely the spatial autocorrelation function (power spectrum) of natural images, noise in the retinal signal, and the channel bandwidth–or number of distinguishable response states (Fig 1; [20]). The first of these parameters is given by image statistics, the second by physiological or psychophysical measurements, and the third is estimated from psychophysical data on the crispening effect (Fig 3A; [8]). As the model predicts colour and lightness in naturalistic images, and accounts for various visual phenomena and illusions it offers a framework for understanding neural image processing, and is a starting point for simulating colour appearance for humans and other species.
The model uses a cone-catch image (a, S1 Appendix), which is filtered by either DoG or Gabor kernels for luminance channels, and DoG kernels for chromatic channels (b). Contrasts are converted to Michelson contrasts (c. showing luminance DoG outputs), then clipping and gain processes are applied with a bandwidth (ε) of 10 (d. Fig 2), and the spatial filters are pooled to create the output (e). Output colours are the model’s internal representation and are not scaled to sRGB space. However, we note that the output image has qualities that combine the effects of an impressionist artist’s take on the scene that compresses the contrasts and highlights chromatic features such as the “carpet of bluebells” that observers describe, but are much weaker in the input image. Also noteworthy is that the model would produce the same overall green scene with blue flowers irrespective of the input image’s white balance.
a) human luminance and chromatic detection thresholds for sinewave gratings [37]. b) Clipping adjusts contrasts so that they cannot fall below the CSF at each spatial frequency (α, SUBTHRESHOLD), or above the saturation threshold (β, SATURATED). Subthreshold contrasts are subtracted, and signals at each spatial frequency are multiplied by a gain value—denoted by arrow length in (c)—so that on average natural images have equal power at each spatial frequency (whitening). The saturation threshold is calculated from the CSF and channel bandwidth, ε (4 in this example) at each spatial frequency. High and low spatial frequency channels therefore have low contrast sensitivity, but encode a large range of image contrasts, whereas intermediate spatial frequencies have high sensitivity and a low dynamic range. To demonstrate the clipping effects, we show an input image with sinewaves of different spatial frequencies and contrasts (d). (e) shows bandpass spatial filters and (f) highlights regions that are clipped or preserved. The overlap between neighbouring octaves (f) means that where contrasts are saturated for one channel, they are unlikely to be saturated for all neighbouring channels so that contrast differences are detectable even in high contrast scenes. Ultimately this shows how a system with a severely limited neural bandwidth of 15 contrast levels and peak sensitivity of ~200:1 can code for contrasts in natural scenes larger than 10,000:1. Note that the fine lines in these illustrative images suffer from moiré effects when viewed on a monitor, and we have artificially blurred the higher spatial frequencies in the input and output images to mitigate this effect. These effects were not present in the modelling, which did not use spatial frequencies that exceeded the kernel’s peak sensitivity.
a) fit to Whittle’s crispening data ([8], Fig 9, “25/inc-dec/gray” treatment). Model output is scaled to the same 0–25 range. The best-fitting bandwidth (ε) for DoG filters is 15, and for Gabor (oriented) filters is 3.75, both of which result in a good fit to the raw data. The CIE L* fit specifies lightness in psychophysics and does not account for contrast [80]. b) Model fit to single ganglion response data from Derrington and Lennie ([47], Fig 11B). Fitting used a single free parameter that multiplied the arbitrary SBL model output to match neural firing responses (with zero intercept) by least-squares regression. The SBL model shows a linear contrast response and saturation point that provide a better fit than the authors’ model. The inset excludes the three highest contrast values to highlight the linear relationship prior to saturation.
The model
The SBL model is comparable to other models of early vision that have been proposed to account for lightness and colour perception. These include MIRAGE [31], which uses non-oriented DoG filters, the oriented difference of gaussians (ODOG) model [12], the brightness induction wavelet model (BIWaM) [32], and neurodynamical brightness induction model [33], which use orientation-sensitive filters. The SBL model differs from these predecessors in that to achieve efficient coding of natural images the gain and dynamic range (i.e. contrast response function) of neural channels vary with spatial frequency–as specified by the contrast sensitivity threshold–with gain normalised to natural scene statistics, so that on average the output has equal power in each spatial channel. These gain functions are hard-wired and feed-forward, which contrasts with divisive gain models that use feedback loops to make efficient use of neural bandwidth [34].
The model is implemented as follows (Figs 1 and 2). i): The image is filtered with a set of spatial filters at one octave separations. These filters are either circularly symmetrical difference of Gaussian (DoG) functions [26] or Gabor functions at four orientations [25]. The filtering process differs from convolution in that it applies a Michelson contrast to centre versus surrounds; this approach follows the method proposed by Peli [35], who demonstrated that contrast in complex scenes should be scaled relative to the local luminance. The three spectral classes of filter correspond to those in human vision, namely achromatic/luminance with centre and surround receiving the same spectral inputs, blue–yellow, and red–green with centre and surround receiving opposite spectral inputs. ii): The lower threshold (α) for the filter is set by the psychophysical contrast sensitivity at the filter’s centre frequency (based on contrast sensitivity functions, [CFSs], [36,37], parameters shown in Table A in S1 Supporting Information). α is subtracted from image contrasts, which is consistent with human psychophysics [38]. The filters’ contrast response function is linear over a limited dynamic range to an upper threshold (β), which is a fixed multiple, ε, of α. ε corresponds to the number of contrast levels that can be encoded (i.e. channel bandwidth or response states; [20]); (Figs 1 and 2). Thus, for ε = 10, the contrast saturation threshold β is 10 times the activation threshold α for each filter. As ε is equal for all channels, high sensitivity filters encode a smaller range of image contrasts than low sensitivity filters (Fig 2B and Fig B in S1 Supporting Information). We estimated ε by fitting the model to Whittle’s [8] psychophysical measurement of the crispening effect (Figs 3A and 4). iii) Signal power in each channel is normalised to that of the filter’s response to a natural scene, thereby whitening the average spatial frequency power spectrum of the output [39]. This “neural gain” at each spatial frequency remains fixed (in contrast to divisive gain models). iv) Filter outputs are summed to recover their representation of the original image, which can be compared to human perception of the image.
(a) for the crispening effect [8]. The three rows of grey levels are identical, with equal step sizes. Against the black background contrasts appear largest for darker squares, whereas the opposite is true for the white background. The SBL model explains this effect through saturation; contrasts near the grey level of the local surroundings are preserved (highlighted with circles), while other contrasts are saturated (blue areas adjacent to the highlighted areas). The graph at the bottom plots differences between adjacent squares in the three rows, showing higher contrasts for dark, middle and light ranges respectively. Illusions such as the Chevreul staircase (b) are also explained in part by clipping. The upper staircase appears to be a series of square steps in grey level. The lower staircase has the same grey levels, but is flipped so that its gradient matches the surround gradient. The SBL model correctly predicts that the upper staircase is seen as square steps in grey level (solid green line) while the lower staircase is a series of gradients (dashed grey line). The plot shows pixel values in arbitrary units measured along each staircase, as highlighted in the output image. The model shows that this effect arises partly because the matched gradients of the lower staircase causes local subthreshold contrasts, and because contrasts are not balanced on each side of the step. (c) Shows the effects of increasing background contrast on two identical targets. At intermediate contrasts (~0.1–0.7) the targets are predicted to show simultaneous contrast effects (the right-hand target appears lighter than the left-hand one), and at higher surround contrasts this is predicted to switch to the White illusion (spreading) effect where the right-hand target becomes darker than the left-hand one.
For the red-green and blue-yellow chromatic channels we make the assumption, consistent with neurophysiology [40,41], that the filters are less orientation selective than for luminance channels and use only DoG filters (but see [42]). The bandwidth of the red-green channel equals that of the luminance DoG signal, which produces plausible results (Fig 1 and below). However, if the blue-yellow channel has the same bandwidth (ε), its low contrast sensitivity (Fig 2A) means that it fails to saturate in natural scenes. We therefore reduced ε to give an equal proportion of saturated pixels in natural images for red-green and blue-yellow channels.
Further details of the model are presented in S1 Supporting Information. An implementation of the SBL model is provided for use with ImageJ, a free, open-source image processing platform [43] and the micaToolbox [44,45]. The code is accessible here.
Results
Estimation of the bandwidth, ε
Channel bandwidth (ε) is estimated by fitting the model to human psychophysical data from Whittle’s [8] investigation of the crispening effect. Whittle described how perceived lightness varies with luminance, and how contrast sensitivity depends on contrast and background luminance, by asking subjects to adjust target luminances to make equal-interval brightness series (Figs 3A and 4A). We created images simulating the viewing conditions in Whittle’s experiment, including the spatial arrangement and luminance of the grey patches that he used to create perceptually uniform equal-contrast steps. Raw data (Fig 3A) were extracted from figures using WebPlotDigitiser [46]. Based on least squares fitting, ε is 15 for the circularly symmetrical version of the SBF model (DoG, R2 = 0.994), and 3.75 for the oriented version of the model (Gabor, R2 = 0.995). These bandwidths are within the range encoded by single neurones [20,24]. Critically, the model recreates the characteristic inflection point around the background grey value. Lowering the bandwidth, and thereby increasing the proportion of saturated channels, produces a more extreme crispening effect, which suggests that crispening is due to saturation rather than to loss of contrast sensitivity with increasing contrast between targets and the background (Fig 2), which is the usual interpretation of Fechner’s law [8].
Interestingly, the model with ε derived from Whittle’s [8] crispening data accurately predicts the responses of primate retinal ganglion cells to sinewave gratings [47] (Fig 3B). The model fit (R2 = 0.972) is better than the authors’ own function (R2 = 0.952), although the difference in residuals is non-significant (T-test, t = 0.68, p = 0.50). Both the psychophysical crispening effect and bottom-up neural responses suggest that at around 4 cpd the saturation threshold for the human vision and macaque retinal ganglion cells (βϖ = 4) is approximately 0.2.
Model performance
We tested the SBL model’s ability to account for approximately 52 perceptual phenomena that could plausibly be explained by low-level visual mechanisms [16,48,49], first for the version with oriented luminance filters, and secondarily for DoG filters (chromatic filters were always non-oriented, see above). Both versions of the model correctly predict the direction of almost all effects and, where relevant, their controls (Table 1 and Fig 4 and S1 Appendix). The main exceptions were the DoG (non-oriented) model’s inability to predict illusory spots and bars in the Hermann grid and Poggendorff illusions, comparatively weak performance with one control for the Chevreul staircase [50], failure in some of the brightness induction effects presented by Zaidi et al. [51] (though mostly with the comparatively weak illusions), and the enhanced assimilation of colour created by bars in patterned chromatic backgrounds [52]. Nevertheless, this performance was achieved with no free parameters (Figs 1–3), and the model can be adjusted to predict all effects.
All phenomena were qualitatively explained to some degree. For illustrations of specific effects see the S1 Appendix.
Discussion
The Spatiochromatic Bandwidth Limited model of colour appearance described here at least qualitatively predicts the appearance of a wide variety of images that are used to demonstrate colour and lightness perception (Table 1 and Fig 4 and S1 Appendix). These include ‘illusions’ that have been explained by high-level interpretations of 3D geometry, lighting, atmospherics, or mid-level principles of perceptual organisation [53,54]: for example White-Munker, shadow, Koffka ring and haze illusions. It is therefore parsimonious to suggest that many aspects of object appearance can be attributed to mechanisms adapted for–or consistent with–coding efficiency [19]. Other accounts of the same phenomena invoke specialised mechanisms (e.g. [12,55]) or top-down effects, which imply that multiple sources of sensory evidence and prior knowledge are used to infer the most likely cause of the stimulus [7,16,56,57]. Neither does the SBL model invoke light adaptation or eye movements, which implies that colour constancy is largely independent of the adaptation state of the photoreceptors–provided that they are not saturated. By comparison the models used by standard colour spaces, such as CIE LAB/CIE CAM implement the von Kries co-efficient rule [14], which assumes that photoreceptor responses are adapted to the global mean for a scene, even though chromatic adaptation is affected by both local and global colour contrasts [58]. Retinex [55] and Hunt models do normalise receptor signals to their local value [17], but the weightings of global and local factors are poorly understood. Recent work has successfully modelled some lightness phenomena by simulating edge integration with asymmetric gain for centre-on and centre-off pathways [59,60], but the mechanisms underlying colour constancy are less clear [58]. Moreover, the adjustments required for colour constancy are largely complete within about 25ms [61], which is too fast for receptor adaptation, but consistent with the purely feed-forward character of the SBL model (the Rudd & Zemach model is similarly feed-forward [59,62]). Fig 5 shows how the SBL model can account for colour appearance in a naturalistic image under variable illumination. More generally, the feed-forward architecture of the SBL model explains why many other visual phenomena appear without any delay, whereas existing models require feedback loops for normalisation [12,17,32,55]. Thus, Brown and MacLeod [7] comment that the distribution of surround colours affects colour appearance almost immediately, leaving little time for feedback or adaptation. Likewise, as suggested by [6], contrast induction is explained without requiring the feedback invoked by Nassi et al. [63]. This is because, according to the SBL model, low contrast surrounds allow all spatial bands to operate within their dynamic ranges, whereas high contrast surrounds saturate some spatial bands, resulting in under-estimates of brightness contrast or chromaticity (Figs 3A and 4). Geier and Hudák [50] argue that lateral inhibition cannot explain effects such as the Chevreul staircase, and this indeed is where the SBL model fails to replicate one of their control conditions (see S1 Appendix). However, the SBL model’s performance with the other control, combined with its use of CSFs (grounding the model’s performance at low spatial frequencies) does at least address many of Geier and Hudák’s reasons for rejecting low-level models. Likewise, Zaidi et al. [51] show how brightness induction can be explained through t-junction based models, and while our models do not predict all effects perfectly, we show that low-level models can predict all of their effects under some conditions (e.g. White illusions and Benary cross). Future work should test the model’s performance against humans quantitatively in key tasks such as this. The model also reconciles contrast constancy with a visual system that varies dramatically in contrast sensitivity and contrast gain across spatial frequencies, allowing suprathreshold contrasts to have a similar appearance at different distances [64]. Contrasts are predicted to be most constant where they are saturated across multiple spatial frequencies, e.g. where the blue regions in Fig 2F overlap. Pooling across spatial scales might explain the Abney effect, which is a shift in hue that occurs when white light is added to a monochromatic stimulus [65], because the colour stimulus may be below-threshold at some spatial bands, but above threshold for others, however we require specific data to estimate the bandwidth of chromatic channels (equivalent to Whittle’s [8] luminance crispening data). As noted above (Model, Figs 1 and 2A), we assume that the bandwidth of the red-green signal equals the luminance DoG signal, but the blue-yellow signal has reduced the bandwidth, which produces plausible results when processing natural scenes, but future work should measure the chromatic bandwidth functions and determine whether the SBL model can account for the Abney effect quantitatively. Further developments of the chromatic SBL model should also investigate whether performance could be improved by modelling both single-opponent and double-opponent pathways. The latter are sensitive to both spatial frequency and orientation, and has been suggested to play a role in suprathreshold colour appearance [42]. However, we were able to simulate the same spatial-frequency/saturation effects with the non-oriented version of the SBL model (S1 Appendix).
(a) shows the input image (similar to the Lotto cube [81]) where the blue squares on the yellow-tinted side (right) and the yellow squares on the blue-tinted side (left) are physically the same grey (colours are shown in the squares at the top of the image). The SBL model (b) correctly predicts that the squares under both tinting regimes appear yellow and blue, rather than grey. The SBL model also predicts the powerful simultaneous contrast (or shadow) illusion present in this image whereby; the central tiles on top of the cube appear to be darker than the central tiles on the shaded side of the cube (colours shown in squares on the far left and for right hand sides).
The Circularly Symmetric Version of the SBL Model and Animal Vision
Whereas the oriented version of the SBL model uses orientation selective achromatic filters and circularly symmetrical chromatic filters (see above), the circularly symmetrical version uses DoG filters for all channels. For the visual phenomena that we have tested the oriented version of the SBL model predicts lightness and colour at least as well as the circularly symmetrical version (Table 1). It might therefore seem logical to consider only the former, but whereas the visual systems of all animals probably have circularly symmetrical receptive fields (e.g. [28]), there is limited evidence for orientation selective cells other than in mammalian visual cortex. Also, the differences between the two versions of the SBL model seem to us to be surprisingly small. For example, both predict White effects, which might be expected to depend on orientation selective mechanisms (S1 Appendix; [12,49]), but only the oriented model correctly predicts the presence of illusory spots in the Hermann grid, and elimination of these spots in the wavy grid [66]. Similarly, the oriented version of the model predicts Koffka rings and the Chevreul staircase (Fig 4B) more accurately than the circularly symmetrical version. The bandwidth, ε, for the non-oriented filter is approximately 15, which matches neurophysiological measurements from primate retinal ganglion cells (Fig 3; [47]). By comparison the bandwidth of the oriented version is estimated to be about four-fold lower than that of the non-oriented model, which is consistent with the low spike rates of neurons in the primary visual cortex [24]. For a given spike rate partitioning the information into multiple channels allows a correspondingly reduced integration time. The DoG model highlights an asymmetry between positive and negative contrasts in natural scenes; in order to code natural scenes efficiently and with equal bandwidth for positive (centre-on) and negative (centre-off) contrasts, our model assumes that negative contrasts require a larger dynamic range in order to use the same bandwidth efficiently (see Fig A in S1 Supporting Information). This may reflect the observed asymmetries in primate LGN ON and OFF pathways, where the OFF pathway has a larger dynamic range, and smaller receptive field size [67], and also in primate and cat cortical pathways, where the ON pathways also have a smaller dynamic range than OFF pathways [68]; our models suggest that these asymmetries may be adaptations to natural image statistics.
The SBL model is useful for non-human animals because coding efficiency is a universal principle, and contrast sensitivity functions are known for many species (Fig 2A; [69]), whereas psychophysical and neurophysiological data on visual mechanisms in non-primates is limited. Current research into non-human colour appearance typically uses the receptor noise limited (RNL) model [70,71], which also assumes that early vision is constrained by low level noise. Others have sought to control for acuity and distance dependent effects [45,69,72], but surprisingly few studies have utilised contrast sensitivity functions [73], and behavioural validation of the models is difficult [74,75]. As with human vision, the SBL model may reconcile a number of key effects. For example, in a bird (blue tit, Cyanistes caeruleus) chromatic discrimination thresholds depended on the contrast of the surround [74], which resembles chromatic contrast induction [7] and is simulated by the SBL model. Shadow-illusion effects have also been demonstrated in fish [76]. Aside from predicting colour appearance the SBL model highlights comparatively unexplored trade-offs in visual systems, with contrast sensitivity potentially linked to dynamic range and to other factors such as low-light vision and temporal acuity. For example, birds have poor luminance contrast sensitivity, but high temporal acuity consistent with a low neural bandwidth in the SBL model [77–79].
Supporting information
S1 Appendix. This file contains a comprehensive list of visual phenomena tested with both versions of the model and assessments of the models’ performance.
https://doi.org/10.1371/journal.pcbi.1011117.s001
(PDF)
S1 Supporting Information. This text file contains additional information regarding the model.
https://doi.org/10.1371/journal.pcbi.1011117.s002
(PDF)
Acknowledgments
We thank Jenny Bosten, Nick Scott-Samuel and Roland Baddeley for their constructive feedback on earlier drafts.
References
- 1. Wandell BA. Foundations of Vision Sinauer Associates. Inc Sunderland MA. 1995.
- 2. Brainard DH, Freeman WT. Bayesian color constancy. JOSA A. 1997;14: 1393–1411. pmid:9203394
- 3. Witzel C, Valkova H, Hansen T, Gegenfurtner KR. Object knowledge modulates colour appearance. i-Perception. 2011;2: 13–49. pmid:23145224
- 4. Bloj MG, Kersten D, Hurlbert AC. Perception of three-dimensional shape influences colour perception through mutual illumination. Nature. 1999;402: 877–879. pmid:10622251
- 5. White M. A new effect of pattern on perceived lightness. Perception. 1979;8: 413–416. pmid:503772
- 6. Chubb C, Sperling G, Solomon JA. Texture interactions determine perceived contrast. PNAS. 1989;86: 9631–9635. pmid:2594791
- 7. Brown RO, MacLeod DIA. Color appearance depends on the variance of surround colors. Current Biology. 1997;7: 844–849. pmid:9382808
- 8. Whittle P. Brightness, discriminability and the “Crispening Effect.” Vision Research. 1992;32: 1493–1507. pmid:1455722
- 9. Kane D, Bertalmío M. A reevaluation of Whittle (1986, 1992) reveals the link between detection thresholds, discrimination thresholds, and brightness perception. Journal of Vision. 2019;19: 16–16. pmid:30694300
- 10.
Fairchild MD. Color appearance models. John Wiley & Sons; 2013.
- 11.
Ratliff F. Mach bands: quantitative studies on neural networks. San Francisco, CA: Holden-Day; 1965.
- 12. Blakeslee B, McCourt ME. A unified theory of brightness contrast and assimilation incorporating oriented multiscale spatial filtering and contrast normalization. Vision Research. 2004;44: 2483–2503. pmid:15358084
- 13. Judd DB. Hue Saturation and Lightness of Surface Colors with Chromatic Illumination. J Opt Soc Am, JOSA. 1940;30: 2–32.
- 14. Foster DH. Color constancy. Vision Research. 2011;51: 674–700. pmid:20849875
- 15. Roe AW, Chelazzi L, Connor CE, Conway BR, Fujita I, Gallant JL, et al. Toward a Unified Theory of Visual Area V4. Neuron. 2012;74: 12–29. pmid:22500626
- 16.
Adelson EH. Lightness Perception and Lightness Illusions. 2nd ed. The new cognitive neurosciences. 2nd ed. MIT Press, Cambridge, MA; 2000.
- 17.
Hunt RWG. The reproduction of colour. John Wiley & Sons; 2005.
- 18. Atick JJ, Redlich AN. What does the retina know about natural scenes? Neural computation. 1992;4: 196–210.
- 19. Barlow HB. Possible principles underlying the transformation of sensory messages. Sensory communication. 1961;1.
- 20. Laughlin S. A Simple Coding Procedure Enhances a Neuron’s Information Capacity. Zeitschrift für Naturforschung C. 1981;36: 910–912. pmid:7303823
- 21. Ruderman DL, Cronin TW, Chiao C-C. Statistics of cone responses to natural images: implications for visual coding. JOSA A. 1998;15: 2036–2045.
- 22. Simoncelli EP, Olshausen BA. Natural image statistics and neural representation. Annual review of neuroscience. 2001;24: 1193–1216. pmid:11520932
- 23. Bossomaier T, Snyder AW. Why spatial frequency processing in the visual cortex? Vision Research. 1986;26: 1307–1309. pmid:3798766
- 24. Baddeley R, Abbott LF, Booth MCA, Sengpiel F, Freeman T, Wakeman EA, et al. Responses of neurons in primary and inferior temporal visual cortices to natural scenes. Proceedings of the Royal Society of London Series B: Biological Sciences. 1997;264: 1775–1783. pmid:9447735
- 25. Daugman JG. Uncertainty relation for resolution in space, spatial frequency, and orientation optimized by two-dimensional visual cortical filters. JOSA A. 1985;2: 1160–1169. pmid:4020513
- 26. Enroth-Cugell C, Robson JG. The contrast sensitivity of retinal ganglion cells of the cat. The Journal of physiology. 1966;187: 517–552. pmid:16783910
- 27. Marĉelja S. Mathematical description of the responses of simple cortical cells. JOSA. 1980;70: 1297–1300. pmid:7463179
- 28. Srinivasan MV, Laughlin SB, Dubs A, Horridge GA. Predictive coding: a fresh view of inhibition in the retina. Proceedings of the Royal Society of London Series B Biological Sciences. 1982;216: 427–459. pmid:6129637
- 29. Tadmor Y, Tolhurst DJ. Calculating the contrasts that retinal ganglion cells and LGN neurones encounter in natural scenes. Vision research. 2000;40: 3145–3157. pmid:10996617
- 30. Field DJ. Relations between the statistics of natural images and the response properties of cortical cells. Josa a. 1987;4: 2379–2394. pmid:3430225
- 31. Watt RJ, Morgan MJ. A theory of the primitive spatial code in human vision. Vision Research. 1985;25: 1661–1674. pmid:3832590
- 32. Otazu X, Vanrell M, Alejandro Párraga C. Multiresolution wavelet framework models brightness induction effects. Vision Research. 2008;48: 733–751. pmid:18241909
- 33. Penacchio O, Otazu X, Dempere-Marco L. A Neurodynamical Model of Brightness Induction in V1. PLOS ONE. 2013;8: e64086. pmid:23717536
- 34. Heeger DJ. Normalization of cell responses in cat striate cortex. Visual Neuroscience. 1992;9: 181–197. pmid:1504027
- 35. Peli E. Contrast in complex images. J Opt Soc Am A. 1990;7: 2032. pmid:2231113
- 36. Mullen KT. The contrast sensitivity of human colour vision to red-green and blue-yellow chromatic gratings. The Journal of Physiology. 1985;359: 381–400. pmid:3999044
- 37.
Kim KJ, Mantiuk R, Lee KH. Measurements of achromatic and chromatic contrast sensitivity functions for an extended range of adaptation luminance. Human vision and electronic imaging XVIII. International Society for Optics and Photonics; 2013. p. 86511A.
- 38. Kulikowski JJ. Effective contrast constancy and linearity of contrast sensation. Vision research. 1976;16: 1419–1431. pmid:1007022
- 39. Carandini M, Heeger DJ. Normalization as a canonical neural computation. Nature Reviews Neuroscience. 2012;13: 51–62.
- 40. Solomon SG, Lennie P. The machinery of colour vision. Nature Reviews Neuroscience. 2007;8: 276–286. pmid:17375040
- 41. Conway BR. Spatial structure of cone inputs to color cells in alert macaque primary visual cortex (V-1). Journal of Neuroscience. 2001;21: 2768–2783. pmid:11306629
- 42. Shapley R, Nunez V, Gordon J. Cortical double-opponent cells and human color perception. Current Opinion in Behavioral Sciences. 2019;30: 1–7.
- 43. Schneider CA, Rasband WS, Eliceiri KW. NIH Image to ImageJ: 25 years of image analysis. Nat Methods. 2012;9: 671–675. pmid:22930834
- 44. Troscianko J, Stevens M. Image calibration and analysis toolbox–a free software suite for objectively measuring reflectance, colour and pattern. Methods Ecol Evol. 2015;6: 1320–1331. pmid:27076902
- 45. Berg CP van den, Troscianko J, Endler JA, Marshall NJ, Cheney KL. Quantitative Colour Pattern Analysis (QCPA): A comprehensive framework for the analysis of colour patterns in nature. Methods in Ecology and Evolution. 2020;11: 316–332.
- 46. Rohatgi A. WebPlotDigitizer: Version 4.4. 2020. Available from: https://automeris.io/WebPlotDigitizer.
- 47. Derrington AM, Lennie P. Spatial and temporal contrast sensitivities of neurones in lateral geniculate nucleus of macaque. J Physiol. 1984;357: 219–240. pmid:6512690
- 48.
Shapiro AG, Todorovic D. The Oxford Compendium of Visual Illusions. Oxford University Press; 2016.
- 49. Bertalmío M, Calatroni L, Franceschi V, Franceschiello B, Gomez Villa A, Prandi D. Visual illusions via neural dynamics: Wilson–Cowan-type models and the efficient representation principle. Journal of Neurophysiology. 2020;123: 1606–1618. pmid:32159409
- 50. Geier J, Hudák M. Changing the Chevreul Illusion by a Background Luminance Ramp: Lateral Inhibition Fails at Its Traditional Stronghold—A Psychophysical Refutation. PLOS ONE. 2011;6: e26062. pmid:22022508
- 51. Zaidi Q, Spehar B, Shy M. Induced Effects of Backgrounds and Foregrounds in Three-Dimensional Configurations: The Role of T-Junctions. Perception. 1997;26: 395–408. pmid:9404490
- 52. Monnier P, Shevell SK. Large shifts in color appearance from patterned chromatic backgrounds. Nat Neurosci. 2003;6: 801–802. pmid:12872129
- 53. Adelson EH. Perceptual organization and the judgment of brightness. Science. 1993;262: 2042–2044. pmid:8266102
- 54. Gilchrist A. A Gestalt Account of Lightness Illusions. Perception. 2014;43: 881–895. pmid:25420329
- 55. Land EH, McCann JJ. Lightness and Retinex Theory. J Opt Soc Am, JOSA. 1971;61: 1–11. pmid:5541571
- 56. Gregory RL. Knowledge in perception and illusion. Philosophical Transactions of the Royal Society of London Series B: Biological Sciences. 1997;352: 1121–1127. pmid:9304679
- 57. Yuille A, Kersten D. Vision as Bayesian inference: analysis by synthesis? Trends in cognitive sciences. 2006;10: 301–308. pmid:16784882
- 58. Kraft JM, Brainard DH. Mechanisms of color constancy under nearly natural viewing. PNAS. 1999;96: 307–312. pmid:9874814
- 59. Rudd ME. Neurocomputational Lightness Model Explains the Appearance of Real Surfaces Viewed Under Gelb Illumination. jpi. 2020;3: 10502-1–10502–16. pmid:36968520
- 60. Rudd ME. A feedforward model of spatial lightness computation by the human visual system. Electronic Imaging. 2022;34: 1–7.
- 61. Rinner O, Gegenfurtner KR. Time course of chromatic adaptation for color appearance and discrimination. Vision research. 2000;40: 1813–1826. pmid:10837828
- 62. Rudd ME, Zemach IK. Quantitative properties of achromatic color induction: An edge integration analysis. Vision Research. 2004;44: 971–981. pmid:15031090
- 63. Nassi JJ, Lomber SG, Born RT. Corticocortical feedback contributes to surround suppression in V1 of the alert primate. J Neurosci. 2013;33: 8504–8517. pmid:23658187
- 64. Georgeson MA, Sullivan GD. Contrast constancy: deblurring in human vision by spatial frequency channels. J Physiol. 1975;252: 627–656. pmid:1206570
- 65. Burns SA, Elsner AE, Pokorny J, Smith VC. The abney effect: Chromaticity coordinates of unique and other constant hues. Vision Research. 1984;24: 479–489. pmid:6740967
- 66. Geier J, Bernáth L, Hudák M, Séra L. Straightness as the Main Factor of the Hermann Grid Illusion. Perception. 2008;37: 651–665. pmid:18605141
- 67. Chichilnisky EJ, Kalmar RS. Functional Asymmetries in ON and OFF Ganglion Cells of Primate Retina. J Neurosci. 2002;22: 2737–2747. pmid:11923439
- 68. Kremkow J, Jin J, Komban SJ, Wang Y, Lashgari R, Li X, et al. Neuronal nonlinearity explains greater visual spatial resolution for darks than lights. Proceedings of the National Academy of Sciences. 2014;111: 3170–3175. pmid:24516130
- 69. Caves EM, Johnsen S. AcuityView: An r package for portraying the effects of visual acuity on scenes observed by an animal. Methods in Ecology and Evolution. 2018;9: 793–797.
- 70. Vorobyev M, Osorio D. Receptor noise as a determinant of colour thresholds. Proceedings of the Royal Society of London Series B: Biological Sciences. 1998;265: 351–358.
- 71. Renoult JP, Kelber A, Schaefer HM. Colour spaces in ecology and evolutionary biology. Biological Reviews. 2017;92: 292–315. pmid:26468059
- 72. Barnett JB, Michalis C, Scott-Samuel NE, Cuthill IC. Distance-dependent defensive coloration in the poison frog Dendrobates tinctorius, Dendrobatidae. PNAS. 2018; 201800826. pmid:29866847
- 73. Melin AD, Kline DW, Hiramatsu C, Caro T. Zebra Stripes through the Eyes of Their Predators, Zebras, and Humans. PLOS ONE. 2016;11: e0145679. pmid:26799935
- 74. Silvasti SA, Valkonen JK, Nokelainen O. Behavioural thresholds of blue tit colour vision and the effect of background chromatic complexity. Vision Research. 2021;182: 46–57. pmid:33596523
- 75. Berg CP van den, Hollenkamp M, Mitchell LJ, Watson EJ, Green NF, Marshall NJ, et al. More than noise: context-dependent luminance contrast discrimination in a coral reef fish (Rhinecanthus aculeatus). Journal of Experimental Biology. 2020;223. pmid:32967998
- 76. Simpson EE, Marshall NJ, Cheney KL. Coral reef fish perceive lightness illusions. Sci Rep. 2016;6: 35335. pmid:27748401
- 77. Potier S, Mitkus M, Kelber A. High resolution of colour vision, but low contrast sensitivity in a diurnal raptor. Proceedings of the Royal Society B. 2018;285: 20181036. pmid:30158305
- 78. Ghim MM, Hodos W. Spatial contrast sensitivity of birds. J Comp Physiol A. 2006;192: 523–534. pmid:16404602
- 79. Boström JE, Dimitrova M, Canton C, Håstad O, Qvarnström A, Ödeen A. Ultra-Rapid Vision in Birds. PLoS One. 2016;11. pmid:26990087
- 80.
Commision Internationale de l’Eclairage. Recommendations on uniform color spaces, color-difference equations, psychometric color terms. Paris: CIE. 1978.
- 81. Purves D, Lotto RB, Nundy S. Why We See What We Do. Scientific American. 2002;90: 236–243.