Figure 1.
A. A white noise image free of spatial correlations between pixel gray values. B. A natural image. In the present work, we study sensitivity to local regularities in natural images.
Table 1.
Natural image model features and likelihood estimates.
Figure 2.
The left texture contains model samples, and the right texture contains only true natural image samples. Each texture is a square tiling of 64 samples, where each sample is pixels in size. The observer's task is to indicate the texture made only of natural image samples. Feedback was given, and a short training sequence was performed before every experiment.
Figure 3.
Generating model samples using ICA.
A. A set of 64 pixel natural image patches,
. B. The coefficients of the first two (non-DC) ICA components are plotted against each other for all 64 patches along with their marginal distributions. C. Histogram of the 64 patches' norms in the ICA basis. D. To apply the ICA independence assumption to
, we shuffle the ICA coefficients across samples separately for each component. Shown are the resulting matched model patches,
. E. The coefficients of the first two (non-DC) ICA components of
. The marginal distributions are the same as those of
shown in B. F. Histogram of the coefficient norms of the 64 patches in
. Applying the ICA assumption has changed the radial distribution so that the variance is much lower than that of the original distribution shown in C.
Figure 4.
Image patch examples from Experiment 1.
In Experiment 1, we tested six models in one session (RND, ICA, L2, LP, IPS, GPS) and the four mixture models in a separate session (MEC2, MEC4, MEC8, MEC16). Shown are example textures for each model. The 64 samples comprising each model texture are matched to the 64 natural image samples shown on the left. Patch size here is pixels. On any single trial, observers viewed only one set of natural image samples and one set of model samples (e.g. as shown in Figure 2).
Figure 5.
Discriminability estimates with 95% binomial confidence intervals are shown by model as a function of patch size, where data are pooled over subjects. Sixteen subjects participated in session one with RND, ICA, L2, LP, IPS, and GPS, and 12 participated in session two with the MEC models. Each subject performed 30 test trials per data point in the plot. Therefore, each data point for session one is based on trials, and each for session two is based on
trials.
Table 2.
Experiment 1 average discriminability for all models.
Figure 6.
Model discriminability and likelihood.
A. Discriminability estimates with 95% binomial confidence intervals plotted in order of increasing model likelihood. Data is pooled over subjects and patch sizes ,
, and
pixels. Each data point for RND, ICA, L2, and LP contains 1,440 trials, and 1,080 trials for the MEC models. MEC models are identified by the number of mixtures. Chance performance was 50%. B. Discriminability estimates with 95% binomial confidence intervals for one subject who performed 5 sessions of a four alternative choice version of the experiment. Each data point for RND, ICA, L2, and LP contains 576 trials, and 144 for the MEC models. Chance was 25%. C. Discriminability ranks of the models from most difficult to easiest are plotted against likelihood ranks from lowest likelihood to highest. Diamonds show group average data from A, and circles show the individual subject's data from B. The group data contain more trials and show a clear decrease in discriminability with increased likelihood. The same order is shown in the individual subject data within the range of the 95% confidence intervals, which overlap for L2, LP, and MEC
.
Figure 7.
Experiment 2 texture scrambles.
Here we show example textures for each model tested in Experiment 2: RND, ICA, L2, LP, IPS, GPS, and MEC16. Both A and B are scrambled versions of the corresponding model stimuli shown in Figure 4. On any single trial the observer viewed only one texture based on natural image samples and one texture based on samples from a single model. A. Global scrambles, where the pixels of each texture were scrambled as a final post-processing step. B. Sample scrambles, where the pixels of each image patch were scrambled individually to preserve variations in luminance histograms across samples.
Figure 8.
A. Discriminability estimates with 95% binomial confidence intervals are shown by model as a function of patch size. Three subjects participated, and each performed 30 test trials per model per patch size per condition, so each data point is based on trials. We did not measure discriminability for MEC
with
pixel patches as observers were at chance with them in Experiment 1. The solid line shows these observers' data in Experiment 1, i.e. with unperturbed stimuli, the dotted line shows performance for global scrambles, and the dashed line for sample scrambles. B. Discriminability estimates averaged over patch size for each model are plotted in order of increasing likelihood. The colored bars are the data from Experiment 1, the translucent bars with dashed edges are for the global scrambles, and the bars with solid edges are for the sample scrambles. In all three conditions, the ordering is the same: higher likelihood is linked with lower discriminability.
Figure 9.
Experiment 3 contrast fluctuation matched model samples.
The contrast fluctuations of each model sample set have been artificially matched to the contrast fluctuations across the natural samples by matching the distribution of grayscale pixel norms to that of the natural samples. Each texture is the fluctuation matched version of the corresponding stimulus in Figure 4.
Figure 10.
Discriminability estimates are plotted with 95% binomial confidence intervals. Nine subjects participated, and each performed 36 test trials per model per condition per patch size, so each data point in A and B is based on trials. MEC, IPS, and GPS were not included in the experiment because they perfectly captured the contrast fluctuation cue in Experiment 2. A. Results from the unperturbed stimulus condition. B. Results from the contrast fluctuation matched stimulus condition. C. Discriminability estimates pooled over patch sizes and plotted in order of increasing model likelihood. The unfilled bars are for the unperturbed stimulus data in A, the filled bars for the data in B. As expected the model ordering for the data in A are the same as in Experiment 1, but the model ordering changed for the contrast fluctuation matched data, showing that
brought performance closest to chance out of all models whereas ICA was near ceiling with the unperturbed stimuli.
Figure 11.
Experiment 4 high contrast stimuli.
To focus on regions of natural images containing shape information, we automatically selected high contrast natural image patches for use as stimuli. A. Grayscale stimuli for the 8 models we tested: RND, ICA, L2, LP, MEC2, MEC4, MEC8, MEC16. B. The binary version of A where the number of on and off pixels are held equal. On any only trial, the observer viewed only one set of natural image samples and one set of samples from a single model.
Figure 12.
Discriminability estimates with 95% binomial confidence intervals are shown by model in order of increasing likelihood where data are pooled over subjects and patch sizes (,
,
, and
). Seven subjects participated, and each performed 36 test trials per model per patch size per condition, so each data point is based on
trials. Unfilled bars are for the grayscale high contrast stimuli, and filled bars the binary version. Within the range of the error bars, the estimates for the grayscale stimuli followed the same ordering as in Experiment 1, yet the data for the binary stimuli show no ordering.