2.1 Firing rates throughout visual cortex are modulated by spatial predictability

We first asked whether neural responses were modulated by overall spatial predictability. To compute spatial predictability for every patch in every image, we masked out the image patch that spans the receptive field of an image, then use a deep generative model to predict or “fill-in" the missing image patch, and compare the predicted image patch with the actual image patch to compute overall predictability. While this method is designed to quantify predictability in complex natural scenes, it can also recapitulate classic contextual effects with simple parametric stimuli (see S1 and S2 Figs). Critically, this predictability calculation was performed for every classical receptive field of every unit of every image individually (see Fig 1c and Methods). Cortical responses were assessed as firing rate activity time-courses (expressed as fold-change with respect to baseline) for every neuron, for every stimulus.

To isolate the influence of image patch unpredictability from known drivers of visual cortical activity, we first fitted a robust baseline regression model to the firing rates at every timepoint. This baseline model incorporated a series of non-prediction-related variables, including running speed and a comprehensive set of low-level image statistics (e.g., local contrast energy across spatial frequencies, orientation content, and circular variance, indexing contour homogeneity; detailed in Methods 4.6). To evaluate whether unpredictability significantly modulated firing rate, we then evaluated whether adding our image patch unpredictability metric could explain unique, additional variance beyond this comprehensive baseline. To test this, we computed —the increase in cross-validated prediction performance when unpredictability was added to the baseline model (Fig 2a, 2b). Note that this cross-validated metric provides a conservative test for the presence of an effect, rather than a direct estimate of the effect size magnitude.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. a) Cross-validated prediction performance of the baseline model across the visual cortical areas, recapitulating the hierarchical organization established in [1].

b) Average increase in cross-validated performance between 75-150 ms—see dotted lines in panel a)—when adding unpredictability to the baseline model. Small dots indicate individual animals; large dots with error bar indicate mean across animals and bootstrapped 95% CI. Note, provides a conservative test for the presence of an effect; for effect size, see β coefficients in panels c-d. c) Effect of unpredictability and contrast on average response in V1. Curves show the average response (blue curve), and the average response plus β coefficients of unpredictability (red) and contrast (gray). Note that firing rate is expressed as fold-change with respect to baseline (0-20 ms; see Methods). d) Same as panel c) but showing just the modulation function (β coefficients), rather than effect on average response. In both panels, curves show average coefficient across neurons for each animal, averaged across animals; shading shows bootstrapped 95% CI across animals. Note that in all regressions, predictor features are z-scored, making coefficients reflect standardised modulation strength.

https://doi.org/10.1371/journal.pcbi.1013136.g002

This revealed robust effects of unpredictability, both in V1 (mean , 95% CI [0.0026,0.0046], bootstrap t-test with Bonferroni correction: p < 0.0001), and across downstream visual areas (VISl: mean , 95% CI , p = 0.006; VISrl: mean , 95% CI [0.0010,0.0034], p = 0.0012; VISal: mean , 95% CI [4.47 × 10⁻⁴,0.0026], p = 0.0144; VISam: mean , 95% CI [9.77 × 10⁻⁴,0.0025], p < 0.0001), with the exception of VISpm where the effect was not significant (mean , 95% CI [−4.52 × 10⁻⁴,0.0025], p = 0.282). Together, this shows that cortical responses are indeed sensitive to spatial unpredictability, across multiple cortical regions.

Having established that spatial unpredictability impacts the neural response, we next examined the coefficients of the regression model to characterize the nature of this effect. The coefficients of this regression model can be interpreted as ‘modulation functions‘ that quantify how a variable modulates the firing rate over time. As can be seen from Fig 2c, 2d, we observed a clear positive modulation by unpredictability (average coefficient between 75–150 ms: M = 0.077, 95% CI [0.059,0.096], p < 0.0001, bootstrap t-test across animals). This indicates that more predictable image patches are associated with weaker neural responses (expectation suppression). For reference, the effect of unpredictability can be compared to the effect of local contrast energy in a unit’s preferred contrast channel. This reveals that the effect of contrast is stronger than the effect of unpredictability (paired difference between 75–150 ms: M = 0.132, 95% CI [0.108,0.157], p < 0.0001). Critically, the contrast effect appears to modulate the neural response earlier (Fig 2c, 2d; average latency to 50% peak: M = 71.0 ms, SE = [68.8,73.2] vs. unpredictability: M = 77.3 ms, SE = [75.1,79.6]), in line with the notion that predictability effects rely more on recurrent processing.

2.2 Opposite tuning for stimulus features and feature predictability

After confirming that overall spatial predictability modulated responses across mouse visual cortex, we next assessed the representational content of these predictions. This was done by estimating the neural effect not just of overall predictability, but of predictability at multiple levels of abstraction [3,16]. These multi-level predictability estimates were obtained by comparing the actual and predicted patch at increasingly abstract feature spaces. For this we used the convolutional layers of AlexNet [18], a shallow CNN with a good fit to mouse visual cortex [19]. Because early CNN layers extract simple features (e.g. edges) and higher levels more complex ones (e.g. textures), performing the comparison at each layer allows to quantify predictability at multiple levels of abstraction (Fig 3a, 3b).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. a) Multi-level predictability analysis.

To compute unpredictability at multiple levels, the predicted RF-patch and actual input RF-patch were fed into a CNN to extract a series of increasingly abstract features. The predicted and actual input patch were then compared for each level separately, to separately quantify the predictability of low- and high-level features. b) Dissociation between high-level and low-level predictability. Some illustrative example image patches (of stimuli from [1,6]) at four quadrants of the high-level vs low-level unpredictability axes. One common example are textures (grass, moss, sandstone), for which the exact composition of low-level features (edges and orientations) are highly unpredictable, but the higher-level structure (the texture) is perfectly predictable; hence right top corner. See Methods for more details, and S3 Fig for more examples. c) Selectivity for visual features (blue) and visual feature predictability (red) in V1. Image feature encoding (blue) shows normalised encoding, i.e. correlation coefficient divided by the maximum encoding performance per unit, before averaging across units. Unpredictability sensitivity is defined as average coefficient in the 75-150 ms timewindow (see Fig 2). Dots show mean across animal-level averages; error bars show bootstrapped 95% confidence intervals. Importantly positive trend for unpredictability sensitivity was absent in a control analysis using synthetic noise images, confirming it reflects neural tuning (see S7 Fig).

https://doi.org/10.1371/journal.pcbi.1013136.g003

Focusing on V1 first, we computed the unpredictability sensitivity as the average coefficient between 75-150 ms, for each level of unpredictability. This revealed a clear positive relationship (see Fig 3c): firing rates in V1 were least sensitive to low-level unpredictability, and most sensitive to high-level unpredictability; and this pattern was highly consistent across animals (mean slope = 0.149, 95% CI [0.096, 0.210], bootstrap: p < 0.0001), suggesting that early visual cortex predicts higher rather than lower-level visual features. While this is in line with recent observations ([3,20]) the pattern is striking since the unpredictability-sensitivity to higher versus lower-level features is the opposite of the well-characterised feature-sensitivity of V1. Indeed, when we examined the tuning profile of the neurons, by performing a more standard encoding-modelling based feature-sensitivity analysis—i.e. predicting firing rates as a function of CNN image features, rather than the unpredictability—we found that V1 was tuned to low-level, instead of high-level, features. Namely, V1 firing rates were best predicted by low-level image features (from early CNN layers), and progressively worse by higher-level image features. This negative relationship recapitulates traditional bottom-up feature tuning, and was again highly consistent across animals (mean slope = –0.034, 95% CI [–0.039, –0.027], bootstrap test across animals: p < 0.0001).

This dissociation between feature-sensitivity (preferring low-level features) and unpredictability-sensitivity (preferring high-level features) provides clear evidence that the high-level unpredictability effects are distinct from V1’s basic tuning to simple image features (see Discussion). Interestingly, when we repeated the same analysis for other downstream areas in the mouse visual cortex, we found largely the same pattern of effects: feature sensitivity was highest low-level image features, but unpredictability sensitivity was highest for high-level features—although we also found that, for encoding, later level visual areas are more sensitive to higher-level features than early visual areas (see S4 and S5 Figs). The relatively modest differences in selectivity between V1 and higher visual areas is in line with earlier encoding analyses of mouse visual cortex [19].

As a final confirmatory control, we repeated the predictability-sensitivity analysis, but now using cross-validation: quantifying predictability effects as the increase in regression model fit when adding each unpredictability estimate to the baseline model. This reveals a very similar positive relationship for V1 (S6a Fig), that is highly consistent across animals (mean slope across = , 95% CI [, ], bootstrap t-test: p < 0.0001). Indeed, we find again a similar pattern across all cortical areas, where unpredictability effects are strongest for higher-level unpredictability and weakest for low-level unpredictability (S6b Fig).

Finally, we explored whether the timing of these effects also varied across the different levels of unpredictability. However, a dedicated latency analysis did not reveal any clear or consistent patterns (see S9 Fig). This null-result should be interpreted with caution, as the analysis may not have been sensitive enough to detect more subtle latency differences between the conditions.

Together, we observed that predictability sensitivity does not simply follow the neuron’s bottom-up feature sensitivity. Instead, we found a distinct preference for unpredictability of higher-level features. This results in a highly consistent pattern of opposite tuning: neurons are most sensitive to lower-level image features, but higher-level feature predictability.

2.3 Predictability effects are strongest in superficial layers of primary visual cortex

A distinguishing feature of predictive coding models is that they postulate two distinct populations of neurons: error neurons that signal the mismatch between predictions and sensory input, and prediction or representation neurons that encode internal models of the environment [8–10,21]. In hierarchical predictive coding models, predictions flow downward through the cortical hierarchy while prediction errors propagate forward. Because in cortex, feedforward signals originate from superficial layers (2/3) while feedback signals stem from deep layers (5/6), this anatomical asymmetry maps directly onto the functional asymmetry in predictive coding. Namely, superficial-layer neurons should primarily signal prediction errors, whereas deep-layer neurons should encode predictive representations of sensory input.

The Visual Coding dataset allowed to test this hypothesized laminar organization, as it provides simultaneous recordings across all cortical layers of the visual system (Fig 4a). We took this opportunity by classifying neurons as either superficial (layer 2/3) or deep (layer 5/6) based on their precise recording depths (see Methods for detailed classification criteria). Then, we tested whether predictability effects (assuming they reflect sensory prediction errors) were indeed strongest in superficial layers of cortex. To this end, we quantified the modulation of firing rates (75-150 ms) by overall spatial predictability. Strikingly, in V1, we indeed identified this expected difference: the modulation by spatial predictability was consistently stronger for neurons in superficial layers compared to those in deeper layers (mean summed coefficient difference = 0.336 , 95% CI [0.049,0.605], p = 0.0173). Importantly, this laminar difference was specific to predictability and not observed for contrast energy (mean difference = –0.294, 95% CI [–0.949,0.255], p = 0.2774; interaction, difference of difference: mean = 0.630, 95% CI [0.054,1.347], p = 0.011).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. a) Neuropixels probe cortical activity across layers of the entire visual system, allowing to dissociate units in superficial (2/3) and deep (5-6) cortical layers.

b) V1: effect of overall spatial predictability and contrast energy (average coefficient between 75-150ms), in neurons in superficial and deep cortical layers. Small dots with connecting lines indicate individual animals; big dots with error bars indicate mean across animal-averages, and bootstrapped 95% confidence interval. * indicates p < 0.05. c) Same as in b, but for the multi-level predictability estimates; dots indicate mean across animals, error bars indicate 95% confidence interval around the mean. d) V1: effect of overall spatial predictability over time (as in Fig 2d), split out for units in superficial versus deep layers. Solid line indicates mean modulation function across animals; shaded area indicates bootstrapped 95% confidence interval around the mean. Black line indicates time points with significant differences (TFCE-corrected p < 0.05). For a similar plot for other cortical areas, see S8 Fig.

https://doi.org/10.1371/journal.pcbi.1013136.g004

Although the observed laminar difference is subtle, it appears robust across methodological variations, as we found the same patterns not just for overall predictability, but for each of the multi-level predictability estimates (Fig 4c), and across time (Fig 4d), where FWER-corrected comparison reveals significant differences between 85 and 110 ms, showing the effect was not contingent on our specific, pre-defined time-window.

However, because the analysis involved subdividing the already sub-selected units into those classified as ‘superficial’ and ‘deep’ (and discarding others), the analysis could only be feasibly conducted in V1; we did not have enough sensitivity in other areas to meaningfully perform the comparison (see S8 Fig).

2.4 No effect of short-term experience on predictability modulations

Finally, we asked whether the predictability effects were dependent on short-term experience. One possibility would be that these effects reflect more long-term, ‘hard-wired‘ priors about the structure of the natural visual world, largely independent of recent experience. However, the stimuli were repeated 50 times, so the animals built up extensive experience with the images. Since recent experience and familiarity are known to shape cortical processing in mouse visual cortex [22–24], such extensive repetition might enhance the predictability effect—or may even be required for it to emerge in the first place.

To adjudicate between these possibilities, we estimated the effect separately for the first and second half of the dataset. Unsurprisingly, the average response was lower (Fig 5) in the second half of trials (±25 repetitions), compared to the first (difference between 75-150 ms: mean = 0.076, 95% CI [0.028, 0.119], p = 0.0124), potentially reflecting general adaptation. There was also a small but significant difference for contrast energy (mean = 0.024, 95% CI [0.002, 0.051], p = 0.0182), indicating that contrast energy modulated the neural response less strongly in the second half of trials. Strikingly, however, we observed no difference for the unpredictability effect (mean = 0.004, 95% CI [–0.006, 0.015], p = 0.4006; see Fig 5). Together, this suggests the spatial prediction effects we observed appear independent of recent experience.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. a) Primary visual cortex: average response (coefficient of time-resolved intercept) in first vs second half of the trials.

b) Same but for β coefficient of contrast energy in the preferred contrast channel. c) Same but for image patch unpredictability. d) Difference waves (early-late) for all three effects. More extensive exposure leads to a reduced average response and reduced contrast effect, but no difference for unpredictability. In all panels, solid lines are mean coefficients across the animal-level means, shaded area indicates bootstrapped confidence interval around the across-animal mean.

https://doi.org/10.1371/journal.pcbi.1013136.g005

4.1 Stimuli and data

We analysed the ‘Brain Observatory 1.1’ subset of the Allen Institute Visual Coding Neuropixels dataset [1]. This is an openly available dataset that surveys spiking activity across the mouse visual system, recorded using 6 simultaneously inserted Neuropixels probes. Mice viewed a range of full-field stimuli, presented monocularly to the right eye. We focus on two stimulus sets: Gabors—for Receptive Field (RF) mapping—and Natural Scenes. The latter comprises 118 scene images, presented for 250 ms, for 50 repetitions, resulting in 5900 trials (Fig 1A).

For analysis, raw spike times were binned into 10 ms intervals to compute time-resolved firing rates. To enhance signal-to-noise ratio, firing rates were smoothed across time using a uniform moving average filter with a 30 ms window. Because different neurons exhibited widely varying firing rate ranges, we normalised the firing rates to decrease the variance between neurons and avoid that the population or animal-level averages would be dominated by high-firing-rate neurons, We normalised firing rates as fold change relative to baseline, defined as the mean activity during the initial 20 ms of stimulus presentation. This normalisation was computed as (FR-baseline)/baseline, with a small pseudo-baseline value (of 0.5 Hz) added to avoid division by extremely small numbers.

We further enhanced the signal-to-noise ratio by averaging responses across multiple presentations of the same stimulus. Specifically, the 50 repetitions of each natural scene were divided into 5 chunks, with each chunk representing the average response to approximately 10 presentations. This chunking approach preserved stimulus-specific response patterns while reducing noise, providing more reliable inputs for subsequent regression analyses. Finally, to minimise the influence of eye movements and blinks, we implemented quality control on each trial. Trials were excluded (i.e. removed before the averaging into chunks) if they contained blinks, or outlying horizontal or vertical eye positions, defined as those exceeding 2 standard deviations from the mean.

4.2 Receptive field mapping and unit selection

RFs were modelled by fitting a 2d-Gaussian on the smoothed 2d spike histograms of the positions where gabors were presented. This resulted in RF for every identified unit. Subsequently, we selected all units for which this ideal RF that was on-screen and had a size between 150 and 750^∘2; we did this because for much larger RFs the spatial predictability analysis did not work as the RFs became too large to inpaint. Moreover, we required that the “ideal" RF was a good fit, i.e. that the raw (smoothed) 2d histograms could be well approximated by the gaussian (). After applying these RF criteria and combining with the quality assurance criteria reported in [1], this resulted in 3176 analysed cortical neurons in total that were located at the 6 pre-defined cortical areas of interest: 1173 neurons for V1, 354 neurons for VISl, 364 neurons for VISrl, 525 neurons for VISal, 253 neurons for VISpm, and 507 neurons for VISam. For each unit, we used its recorded Common Coordinate Framework (CCF) coordinates to putatively identify its corresponding cortical layer in the Allen Mouse Brain Atlas. Units were classified as “superficial" if they resided in cortical putative layers 1-3 and “deep" if they resided in putative layers 5-6.

4.3 Spatial predictability inpainting model

Our inpainting model employed a PConvUNet architecture with partial convolutions, following Liu et al. [2]. The encoder consisted of up to eight blocks, with the first using a 7×7 kernel followed by 5×5 kernels in the second and third blocks, and 3×3 kernels in subsequent blocks. Each encoder block combined a partial convolution with batch normalization (except the first layer) and ReLU activation. The partial convolution technique progressively filled missing regions by updating a binary mask after each operation, normalising feature values based on the ratio of valid pixels.

The decoder mirrored the encoder with eight blocks, each containing nearest-neighbor upsampling followed by concatenation with corresponding encoder features, partial convolution, batch normalization, and LeakyReLU activation. This skip-connection structure preserved spatial information lost during downsampling. During fine-tuning, encoder batch normalization layers were frozen to maintain pretrained feature extraction capabilities while allowing decoder parameters to adapt to the new dataset characteristics.

The loss function combined reconstruction, content, and style components with weighted hyperparameters. Reconstruction loss incorporated Fourier domain metrics, while content and style losses compared VGG-16 activations and their Gramian matrices, respectively, minimising perceptual artifacts common in generative models.

4.4 Model training and fine-tuning

We used a pretrained model initialized with VGG-16 weights and first trained on the Places2 dataset [57] for 60k iterations with a batch size of 64, focusing on large circular masks to improve performance on receptive field-sized regions. We employed the Adam optimizer with a learning rate of 0.0005 and default momentum parameters (, ).

For the final adaptation stage, we fine-tuned the model for 1500 steps on the Van Hateren Natural Images dataset [6], using a learning rate of 9.5e–5, excluding images used in the Allen Institute Dataset. This step specifically optimized the model for grayscale natural image statistics matching our experimental stimuli.

We adapted the loss function from the implementation of Liu et al. [2], which combined multiple weighted components:

where and L_hole are per-pixel reconstruction losses on non-masked and masked regions respectively, is the total variation loss, measures content across VGG layers, and compares Gram matrices capturing texture statistics. Following Liu et al.’s implementation, we used these loss coefficients during fine-tuning to balance structural accuracy and perceptual quality. Complete mathematical derivations of individual loss components are provided in Liu et al. [2].

4.5 Spatial predictability analysis

For each neuron, we defined an RF mask as the area within the full-width at half-maximum of the gaussian receptive field. This mask was removed from the image and then in-painted by the model. For each image, the filled-in (predicted) RF patch was compared with the actual patch to compute predictability (Fig 1A), for this we took a rectangular mask spanning the RF circle with a dimension of times the diameter of the RF. Unpredictability was defined as the distance between the observed and predicted patches at each convolutional layer of AlexNet, a shallow CNN with a good fit to mouse visual cortex [19]. We compute the discrepancy between predicted and actual patch in the feature space of a deep network because it captures perceptually relevant differences while remaining insensitive to pixel-level artifacts or distortions such as noise and intensity variations that don’t affect perceptual content, thus providing a more robust measure of perceptual distance and hence predictability [58].

In our analyses, we use two types of spatial predictability estimates: the overall spatial predictability (e.g. in Fig 2 and elsewehere where indicated), and multi-level spatial predictability. The overall spatial predictability is meant to serve as an omnibus metric, for the research questions where we were not interested in feature-specific predictability. To reduce researcher degrees of freedom, we simply defined it as the average of the distance for the individual layers. The multi-level predictability estimates, by contrast, where simply the layer-specific ℓ2 distances. Because early CNN layers extract simple features (e.g. edges) and higher levels more complex ones (e.g. textures), performing the comparison at each layer allows to quantify predictability at multiple levels of abstraction (Fig 3a).

While the predictability estimates at different levels are correlated, they also partially and systematically dissociate. Post-hoc inspection confirms they dissociate in predictable and interpretable ways. Some key illustrative examples are included in Fig 3. Perhaps the most common stimulus patch that dissociates high and low-level predictability are textures, where the exact spatial configuration of low-level features (edges, orientations, and spatial frequencies) is highly unpredictable, but the higher-level statistical structure is perfectly predictable. This phenomenon aligns with the seminal work of Portilla and Simoncelli [59], who demonstrated that textures can be mathematically defined and synthesised by matching higher-order correlations among low-level features (while having the low-level features themselves random and unpredictable). This principle explains why texture regions in our dataset occupy the upper-right quadrant in Fig 3B, exhibiting high low-level unpredictability but low high-level unpredictability. At other patches, both the high and low-level features are highly unpredictable, typically occurring at boundaries between distinct objects or scenes where contextual information from the surroundings provides insufficient constraints to predict either low-level features or higher-order structure within the receptive field.

4.6 Control variables

To control for low-level statistics extracted in the feedforward sweep, we implemented a biologically-inspired contrast model that captures how early visual neurons respond to images, by computing both first-order and second-order contrast [4,5,60]. The model computes local contrast using quadrature-phase Gabor filters [61], which mimic the orientation and spatial frequency selectivity of V1 simple cells. Specifically, we constructed two filter banks: a first-order or contrast energy (CE) bank with spatial frequencies spanning 5 octaves (0.02-0.32 cycles per degree) and a second-order contrast or spatial coherence (SC) bank with slightly lower frequencies (0.015-0.24 cycles per degree). For each filter bank and spatial frequency, we computed the energy response E(x,y) by combining quadrature-phase filter responses F₀ and according to . This energy-based formulation quantifies contrast along specific orientations (8 orientations, spanning 0-π radians). We then applied divisive normalization to these energy maps using local statistics: , where S(x,y) represents the local coefficient of variation (standard deviation divided by mean) computed over a Gaussian window. This step models the suppressive surround observed the visual system and enhances the representation of contrast boundaries.

From these normalised contrast maps, we extracted several summary statistics for each neuron’s receptive field across all 118 natural scenes. Due to the limited number of unique images, we could not fit the contrast energy of an entire gabor pyramid to the neural response, and instead had to capture relevant contrast information with a minimal set of regressors. For each neuron, we first identified its preferred spatial frequency and orientation from the Gabor stimulus responses. We then computed the following sets of metrics within each receptive field: (1) five first-order contrast energies at five spatial frequency scales (0.02-0.32 cpd), representing the average normalised energy within the receptive field; (2) five second order contrast statistics (SC) at corresponding scales (0.015-0.24 cpd), computed as the inverse coefficient of variation () of the receptive field of the SC image; (3) preferred contrast energy, capturing energy specifically at each neuron’s preferred spatial frequency and orientation; (4) circular variance, which quantifies orientation homogeneity within the receptive field, allowing us to capture neural responses to extended contours [62]; (5) relative orientation energies, indexing contrast at orientations from the preferred orientation of the specific neuron, averaging across octaves; and (6) root-mean-square (RMS) contrast within the receptive field. Additionally, we incorporated running speed as a non-visual control variable, averaged across trial chunks, to account for locomotion-dependent modulation of visual responses. Altogether, this comprehensive set of regressors enabled us to control for low-level feedforward processing while maintaining a statistically tractable model.

4.7 Regression analysis

To assess the effect of spatial predictability on neural responses, we performed time-resolved regression analyses on the normalised firing rates. For each neuron i and time point t, we fitted separate linear regression models to the normalised firing rate across all observations:

Baseline model:

(1)

Extended model:

(2)

where is the vector of neural responses across observations, and the design matrix contains the intercept plus baseline variables

(3)

The extended model augments this with spatial unpredictability: . Here, denotes first-order contrast energy at spatial frequency scale k ( corresponding to 0.02–0.32 cycles per degree), denotes second-order contrast (inverse coefficient of variation, ) at scale k (0.015–0.24 cpd), is the contrast energy at neuron i’s preferred spatial frequency and orientation, is circular variance, and represent contrast energies at orientations and relative to the preferred orientation (averaged across spatial frequency octaves), is root-mean-square contrast, v is running speed, and P is the spatial unpredictability metric. All spatial features were computed within neuron i’s receptive field (and neuron-specific tuning properties) resulting in a unique design matrix , and separate model estimation, for each neuron. Individual regression models were estimated for each neuron at each timepoint (10 ms bins), allowing us to track the temporal evolution of predictability effects throughout the visual response. We constructed separate models for each predictability metric rather than including multiple predictability estimates simultaneously, since the correlations between different predictability metrics were often high, complicating the interpretation of the coefficients and .

We employed two complementary regression approaches. First, we estimated the full model coefficients () to perform inference on the relationship between predictability and neural responses. This allowed us to quantify the magnitude and timing of predictability effects across different visual areas and cortical layers. Because in every regression we fit, all predictor variables were normalised (z-scored), the resulting standardised β coefficients from these models revealed how strongly neural activity was modulated by predictability compared to low-level features. Second, we used cross-validation to evaluate model performance. For each neuron, we implemented a shuffle-split cross-validation procedure (10 splits with 75% training, 25% testing) to compute correlation coefficients between predicted and actual neural responses. By comparing the performance of models with and without the predictability term, we calculated a difference in correlation coefficients (), which quantified the unique contribution of predictability to explaining neural responses beyond what could be accounted for by low-level image statistics and behavioural variables alone.

4.8 Encoding-based feature sensitivity analysis

To systematically quantify feature sensitivity across visual cortical areas, we implemented an encoding approach that assessed the predictive power of different feature levels on neural responses. Following [19], we employed a pretrained AlexNet CNN to extract hierarchical visual features from the stimulus set. For each unit, we constructed encoding models using feature maps from each AlexNet layer, trained to predict trial-averaged neural responses to natural scenes, with data partitioned into chunks to enhance signal-to-noise ratio while preserving stimulus-specific patterns. To accomodate the high dimensionality of CNN features relative to our stimuli (118 scenes), we employed Partial Least Squares regression with 25 components, capturing predictor space variance while avoiding overfitting. Model performance was evaluated using shuffle-split cross-validation (10 splits, 75% training, 25% testing) to compute correlation coefficients between predicted and actual neural responses, generating layer-specific encoding accuracies for each unit.

Because our primary interest was in comparing relative preferences for stimulus features, we normalizeised cross-validated correlation scores by dividing by the maximum correlation achieved across layers for each neuron. This prevented high-SNR units from dominating population-level analyses while preserving relative layer preference patterns. The resulting normalised profiles enabled direct comparison of feature selectivity across different visual areas and cortical layers, providing complementary evidence to our spatial predictability analysis.

4.9 Statistical testing

Statistical testing was performed using multi-level inference, first computing averages across units in one animal, and then performing statistics across animals. For simplicity, we ignored the fact that different animals had different amounts of (selected) neurons per region, and weighted the contribution of each animal equally.

To compute p-values we used data-driven, non-analytical bootstrap t-tests, that involve resampling a null-distribution with zero mean (by removing the mean), counting across bootstraps how likely a t-value at least as extreme as the true t-value was to occur. Each test used at least 10⁴ bootstraps; p-values were computed without assuming symmetry (equal-tail bootstrap; [63]). Confidence intervals (in the figures and text) are also based on bootstrapping, with 10⁴ resamples.

For time-resolved paired comparisons, we corrected for multiple comparisons using a temporal clustering permutation test. This was implemented using the Threshold-Free Cluster Enhancement (TFCE) algorithm, which integrates cluster size and height information across a range of thresholds [64]. The analysis was performed using the implementation and all default parameters in MNE-Python [65].

4.10 Synthetic stimuli validation

To validate the predictability metric, we generated two sets of synthetic parametric stimuli (384x384 pixels). The first set, simulating classical end-stopping (S1 Fig), consisted of a horizontal bar composed of fine-grained alternating stripes. The bar was presented in three configurations relative to a central circular mask: confined within the mask (‘No Context’), extending 20% beyond it (‘Some Context’), or spanning the full image width (‘Full Context’). The second set, designed to dissociate feature-level predictability (S2 Fig), used a constant vertical grating as the surround. The central masked region was filled with one of three ground-truth patterns: a continuous vertical grating (no mismatch), an orthogonal grating (‘Shifted’), or a grating with added phase noise (‘Noisy’). Since the model’s input (the masked image) was identical for these three conditions, it produced a single reconstruction, allowing us to measure how unpredictability changed as a function of the specific mismatch between the reconstruction and each ground truth. In all cases, the computed loss was normalised (by dividing across the max loss of each layer) to ensure the content losses are at the quantitative same scale for visualisation.