High-level visual prediction errors in early visual cortex
Fig 3
Convergence of DNN and visual cortical representations showing a gradient from low- to high-level features in prediction-free contexts.
(A) RSA of visual responses shows that (feedforward) visual responses during the prediction-free localizer were best explained by a gradient of low-level to high-level visual features going up the ventral visual hierarchy. EVC responses aligned more closely with early DNN layers, indicative of low-level visual feature processing (depicted in cold colors: purple to blue). HVC areas, like the fusiform gyrus, show a greater correlation with late DNN layers, representing high-level visual feature processing (illustrated in warm colors: yellow to red). Analysis was masked to visual cortex and thresholded at z > 3.1 (p < 0.001, uncorrected) of the RSA. (B) ROI masks, depicting voxels included in the anatomically and functionally defined masks. Color indicates the ROI: Blue = V1, Purple = LOC, Orange = HVC. Opacity indicates the proportion of participants whose individual masks included the voxel. For visualization, full opacity corresponds to a proportion of 0.1, with voxel inclusion proportions reaching up to ~0.7. Data and code that support these findings are available at: https://doi.org/10.34973/8e49-2012. DNN, deep neural network; EVC, early visual cortex; HVC, higher visual cortex; ROI, region of interest; RSA, representational similarity analysis; V1, primary visual cortex.