Fig 1.
Same signal but different noise.
Observers were asked to discriminate a vertical Gabor target signal (A) from a non-target signal (blank in F). Four different noise types were added to both target and non-target signals. In the 2D noise condition (B,G) each pixel was assigned a random Gaussian modulation. In the 1D condition (C,H) noise only varied along the horizontal dimension in the form of bar-like Gaussian modulations. In the orientation (Θ) condition (D,I), noise consisted of the sum of a set of Gabor patches spanning the entire orientation range (K), each patch taking on a randomly assigned contrast value (see Methods). Spatial frequency (SF) noise (E,J) was generated using a similar procedure, except the underlying patch set varied across SF (L) rather than orientation. The green (alternatively red) profile in A displays a horizontal (alternatively vertical) slice through the target surface. Green labels in K (alternatively L) point to target location along Θ (alternatively SF) dimensions.
Fig 2.
Perceptual filters used by human observers to detect/discriminate stimuli.
A-D show aggregate (across observers) perceptual filters (PF) for detecting the vertical Gabor target (icons to the left of A) returned by reverse correlating different noise types: 2D (A), 1D (B), Θ (C) and SF (D; see Fig 1 for image samples of all four classes). Red/blue lines in A show positive/negative contours through a Gabor fit to the data. Green trace in B and green lines in C-D indicate target signal. Grey shaded regions in B-D show ±1 SEM. Panels E,G show results corresponding to A,C (2D and Θ noise probes) for discriminating vertical from horizontal Gabor targets (icons to the left of E). F plots match between 2D PF’s and target/non-target signals on y/x axes (non-target as specified in the discrimination task) for both detection (solid) and discrimination (open) across different observers (different data points). H plots filter amplitude of orientation-tuned PF’s (C,G) at 0 (peak, y axis) and ±π/2 (trough) using similar conventions. Error bars (±1 SEM) in F,H are sometimes smaller than symbols and therefore not visible. The centre value of 2D PF’s (A,E) is effectively undefined because occluded by the fixation marker (see Methods).
Fig 3.
Nonlinear tests return no deviations, i.e. compliance with template matching.
Panels B-F,H plot the same as Fig 2A–2E and 2G but separately for target-present and target-absent PF’s (see Methods). The two descriptors match, as predicted by the linear-nonlinear (LN) cascade. The 1st-order nonlinear test is designed to quantify potential differences between the two first-order descriptors (see Methods). G plots the outcome of this test for individual observers (one data point per observer per condition) on the x axis for the 2D (red, solid/open for detection/discrimination) and 1D (orange) conditions, versus the outcome of the 2nd-order nonlinear test designed to quantify potential modulations within second-order PF’s (these should be featureless for the LN cascade [10]). Both tests scatter around 0 (indicated by dashed lines). I plots the same for Θ (blue) and SF (magenta) conditions. Error bars in G,I show ±1 SEM; gray ovals are centred on mean across all data points, with axes tilted along linear fit and spanning ±2 SD of projected data values. A plots distributions for both tests across all conditions/observers, together with a similar descriptor (orange) quantifying modulations within full first-order PF’s (i.e. computed from all trials; this additional analysis is presented here to demonstrate that the adopted test metric is sensitive to structure when present).
Fig 4.
Human intrinsic variability is compatible with a linear model followed by a late additive noise source.
The absolute efficiency predicted by the LN model (x axis in A) matches corresponding human estimates (y axis). Inset to A plots distribution of log-ratios between measured and predicted values (0 when not different). Trial-by-trial agreement between human responses and LN model (y axis in B) falls within the optimal range (indicated by green shading; green dotted line shows midpoint between upper and lower boundaries); inset shows similar results for gain-control model (diagram in Fig 6A). Symbol size in B reflects associated decoupled baseline (small size for baseline≤chance, big for baseline>chance; see S1 Text for details on how baseline was computed). Axes have been stretched to map linearly in d′ units. Internal noise estimates (see Methods) plotted on y axis in C (see also side histogram) are broadly consistent with measurements from previous studies [27] (range indicated by green shading) and do not correlate with sensitivity (x axis), as expected of an additive noise source. Red horizontal line marks cut-off point (value of 5) beyond which internal noise estimates represent estimation failures [27]. Estimates for 2D (x axis in D) versus 1D noise from the detection task are mildly correlated across observers (dashed lines show ±95% confidence intervals on linear fit) but do not differ (data points scatter around solid unity line), consistent with a late common noise source (an earlier noise source may be expected to scale with noise dimensionality [28]). Black/blue symbols in D show estimates obtained by computing the percentage of correct responses from double-pass blocks only (black) or all blocks (blue) to demonstrate that this choice had little impact on the resulting trend. In all remaining panels, solid/open symbols refer to detection/discrimination tasks and different colours refer to different noise types (red for 2D, yellow for 1D, blue for orientation, magenta for SF). Diamond shape refers to the symmetric variant of the discrimination experiment. Each symbol refers to data from an individual observer in the condition specified by its colour/shape characteristics. Error bars show ±1 SEM in all plots (omitted in inset to B to avoid clutter); ovals are centred on mean across data points, with axes tilted along linear fit and spanning ±1 SD of projected data values.
Fig 5.
No single LN model is able to account for the entire dataset.
B-E,G,I plot human PF’s shown in Fig 2A–2E and 2G together with simulated PF’s returned by a single 2D implementation of the matcher (LN) model (blue in C-E,I) or the push-pull model (red in C-E,I; see diagrams in A,F for schematics of the two models). Error regions around simulated PF’s show ±1 SD across simulations. H,J show normalized root-mean-square distance (see Methods) between simulated and human PF’s for matcher model (x axis) versus push-pull model (y axis) using the plotting conventions adopted in Fig 3G and 3I.
Fig 6.
Gain-control model accounts for the entire dataset.
Plotting conventions as in Fig 5. Diagram in A summarizes structure of the gain-control model that generated the PF’s in B-F,H.
Fig 7.
Nonlinear tests return no deviations for gain-control model, but detect nonlinear behaviour exhibited by push-pull model.
Black histograms plot distributions for the 1st-order nonlinear test, grey histograms for the 2nd-order nonlinear test (see caption to Fig 3 for brief description of these two tests), orange histograms for 1st-order full PF’s (reflecting linear component of LN models when applicable), across all 4 noise conditions (different rows) and for both gain-control (A) and push-pull model (B). Left insets plot target-present/target-absent (red/blue) first-order PF’s (similar to Fig 3B–3E); right insets plot second-order PF’s for the 1D condition. The second-order PF associated with the push-pull model displays substantial modulations; in the 1D condition, this model returns clearly positive values for both nonlinear tests. All simulated first-order PF’s display measurable structure (see orange distributions) except for the push-pull model in the 2D condition (top right, see also Fig 5B and 5G).
Fig 8.
Symmetric variant of discrimination task engages two elementary operations with orthogonal preference.
The central probe was flanked by additional non-target templates (indicated by green in A). B-C plot overall (labelled ‘Human’) as well as target-present/absent PF’s for 2D and Θ conditions (similar plotting conventions to Fig 3B and 3D). F plots the overall 2D PF from the opponent variant of the LN model (left) and target-present/absent PF’s from the opponent variant of the gain congrol model (right; see Methods for detailed description of these variants). G plots overall Θ PF’s from these two models (magenta and yellow respectively) together with the human PF replotted from C (black). D plots target/non-target match values (similar to Fig 2F) from overall (black), target-present (red) and target-absent (blue) 2D PF’s; light colouring refers to the observer who had also participated in the original discrimination experiment without non-target templates. E plots peak/trough amplitudes of Θ PF’s (similar to Fig 2H) using similar colour-coding conventions. H plots normalized root-mean-square distance between simulated and human PF’s using the conventions adopted in Figs 5H and 5J and 6G and 6H. I plots outcome of 1st- and 2nd-order nonlinear tests using the conventions of Fig 3G and 3I across observers and noise masks from the original discrimination experiment (black) and its symmetric variant (green).
Fig 9.
The sense in which vision is linear, and the sense in which it is not.
Our results can be intuitively summarized with relation to the simple notion of piecewise linear approximation to a nonlinear function, although this parallel should not be taken literally: the sensory process is far more complex than can be represented here using a single trajectory. It is nevertheless useful to imagine this process as the pictured 3D pipe-like structure, spanning all 4 different spaces within which we defined and perturbed the visual stimuli. When the process is probed within each individual stimulus space, its characteristics can be adequately approximated by a linear model. A comprehensive account of its collective properties spanning all 4 spaces, however, cannot be achieved using a linear approximator: the system may appear linear every time it is probed from a different direction, yet its underlying structure remains highly nonlinear.