Concept manifold geometry and few-shot classification error

Consider two concepts, a and b. A key insight of the theory of Sorscher et al. is that even though the manifolds for concepts a and b may have quite complex structure, the error of a linear few-shot classifier that is trained to discriminate between the concepts depends upon a small number of intuitive geometric properties: the distance between the centers of the manifolds (“geometric Signal”); the radius of manifold a relative to the radius of manifold b (“Bias”); the number of independent directions of variation that are utilized by a manifold (“Dimensionality”); the alignment of principal manifold directions with each other (Noise-Noise Overlap); and the alignment of the principal manifold directions with the Signal vector that connects the centers of the manifolds (Noise-Signal Overlap).

Sorscher et al. introduced a critical quantity, the geometric Signal-to-Noise ratio (), that relates all of these geometric properties to m-shot classification error:

(1)

where and are the geometric Signal and Bias terms respectively. The denominator consists of all the nuisance factors: the Noise-Signal Overlap for manifold i and the Noise-Noise Overlap that capture the overlap between all of the manifolds’ directions (Fig 1A). For all elements, the subscript indices indicate that the element describes the property of concept a relative to concept b. These quantities are generally asymmetrical with respect to the concept manifolds, meaning it is possible that . The m-shot error rate is related to the by a Gaussian tail function (dashed black curves in Fig 4). Finally, D_a is the participation ratio dimension (referred to here as “Dimensionality”) of concept a.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Measuring geometric properties of concept manifolds in the space of human brain activity.

(A) Concept manifolds (shown here as ellipses) for two concepts (blue and green, respectively) in the space of brain activity (each axis of the space corresponds to the activity in a single voxel). To construct an m-shot (m = 5 in this illustration) classifier, prototypes (open circles) are estimated by computing the mean of m exemplar activity patterns (filled circles). At test, activity patterns are projected onto the vector (thick red line) connecting the concept prototypes, and thresholded via distance to the mid-point (thin red line) between prototypes. According to the theory of Sorscher et al., the error of such an m-shot classifier is determined by several geometric properties of the concept manifolds: geometric Signal is distance (dashed red line) between the true manifold centroids (dark x-marks) relative to the radii (lines with feet) of the principal directions (thick green and blue arrows); Bias is the relative radius of each manifold; Noise-Noise Overlap measures the alignment of the two manifolds’ principal directions; Noise-Signal Overlap is the alignment of the manifolds’ principal directions and the line (dashed red) connecting the centroids; Dimensionality is a measure of how uniformly the variance in concept exemplars is distributed across the axes of the space of neural activations. Collectively, these properties determine a quantity called the “geometric SNR” (inset grayscale matrix) that, per the theory, predicts m-shot error for any pair of concepts. In this illustration the m-shot error will be non-zero, since at least some of the exemplars (in the striped area) will be mis-classified. (B) Inferring geometric properties from fMRI data. Geometric properties and few-shot error are inferred from activity patterns (colored matrices), shown as exemplars (filled circles) and prototypes (open circles) in the space of brain activity measured using fMRI (right; pink) or output from an encoding model (left; red). In our case the encoding model is a deep neural network consisting of a feature extractor (trapezoid) and voxel-wise read-out heads (rectangle) and trained end-to-end to predict brain activity. (C) Once trained, an encoding model can be used to generate estimates of activity patterns (matrix) for exemplars (rows) of any pair of visual object categories (blue / green bars) in the ImageNet database. From these, geometric properties of pairs of concept manifolds (green/blue ellipses) are computed. All images are for illustration purpose only (photographers: K.K. and G.S.-Y.).

https://doi.org/10.1371/journal.pcbi.1013416.g001

As the equation shows, classification error will be small when geometric Signal and Dimensionality are large and Noise-Noise and Noise-Signal Overlap are small. In other words, visual representations support accurate few-shot classification of concepts a and b when their manifolds are far apart, when the concepts utilize multiple directions of variation, and when the principal directions of the manifolds do not align with each other or with the Signal vector.

It is important to note that the expression for geometric SNR does not reveal any explicit dependencies between the geometric properties. Thus, as far as the expression for geometric SNR is concerned, there are many different patterns of co-variation in geometric properties across visual cortex that can induce an increase or decrease in SNR. Variation in geometric SNR could emerge as a result of “coordinated” co-variation, in which variation in all of the constituent geometric properties affect geometric SNR in the same way, or as a result of “antagonistic” co-variation, in which variation in the constituent properties have opposing effects on geometric SNR (as seen in the example below; Fig 2B). Also consistent with the theory is an interesting special case of antagonistic co-variation, in which variation in all geometric properties cancel each others’ effects, resulting in potentially profound variation in manifold geometry across visual cortex, but no variation in geometric SNR. From the limited perspective of understanding concept classification, such a profile of co-variation in geometric properties could be considered “non-functional”.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Illustration of two concept manifolds for brain regions of interest across human visual cortex.

Projections of voxel activity patterns in (A) V1 and (B) PHC corresponding to two concepts: “sport”-related things (each blue dot is one activity pattern) and “food”-related things (green dots). Activity patterns are projected onto the Signal direction (x-axis), i.e., the vector connecting the centroids (squares) of each concept manifold, and the direction orthogonal to the signal direction (y-axis) with maximum variance for the two concepts. The centroids for other concepts (large circles) are displayed for reference. Images associated with extremal activity patterns (triangles) which indicate some of the image features that vary along the signal direction can be found at https://github.com/styvesg/nsd_manifolds. (C) Summaries of sport-things and food-things manifolds (blue and green ellipses, resp.; axes same as in A,B) manifold orientations (greeen and blue lines) and distance between manifold centroids (purple arrows) are shown for the V1 and PHC ROIs.

https://doi.org/10.1371/journal.pcbi.1013416.g002

To gain an intuition for how these various geometric quantities might vary across human visual cortex, we visualized 2D projections of concept manifolds using brain activity patterns in the Natural Scenes Dataset (NSD), a massive sampling of blood-oxygenation-level-dependent (BOLD) activity in eight subjects using ultra-high field fMRI (7T, 1.8-mm resolution). Subjects each viewed 9000–10,000 natural scenes (sampled from the Microsoft Common Objects in Context database [18]) presented (3-s exposure) repeatedly (three times typically), yielding 22–30K trials for individual subjects and a total of 213K trials across subjects.

We visualized manifolds for the concepts “has something to do with sports” and “has something to do with food” in visual regions of interest (ROIs; V1, V2, V3, V4, V3a, V3b, lateral occipital complex (LOC), ventral occipital cortex (VO), intraparietal sulcus (IPS), middle temporal area (MT), medial superior temporal area (MST) and peri-hippocampal gyrus (PHC)) that collectively span most of visually-responsive cortex (Fig 2). As expected, the concept manifolds in higher areas appeared to be more disentangled than concept manifolds in lower visual areas, in the sense that they overlap less in the 2D plane used here for visualization (Fig 2). The visualizations offered some clues about what particular geometric properties make the manifolds for these specific concepts more disentangled in higher than in lower visual areas. For example, in PHC the manifold centers are farther apart (relative to their respective radii) than in V1 (purple annotations in Fig 2C), suggesting that manifolds in PHC are more disentangled than in V1 due to an increase in geometric Signal.

Interestingly, in PHC the “sport things” manifold was compressed towards a principal direction that was clearly rotated toward the Signal vector, indicating a decrease in Dimensionality and an increase in noise-signal Overlap in PHC relative to V1 (blue and green line annotations in Fig 2C indicating the first PC of that manifold). Per the theory of Sorscher et al., linear classification error increases as Dimensionality decreases, and as Noise-Signal Overlap increases; thus, manifolds in PHC were more disentangled than in V1 in spite of a relative decrease in Dimensionality and increase in Noise-Signal Overlap. This is an illustration of “antagonistic” co-variation because a change in one geometric property (Signal) effected an increase in SNR, while a change other geometric properties (Dimensionality and Overlap) mitigated the effect.

Having established these intuitions, we used the NSD to conduct a systematic exploration of few-shot classification error and manifold geometry across visual cortex.

Geometric properties of concept manifolds precisely determine the error of few-shot classifiers estimated from both human brain activity patterns and the outputs of an encoding model

Before estimating concept manifold geometry, we first empirically measured how the error of few-shot classifiers for thousands of visual concepts varied across the visual cortex. To control for the effects of inter-trial variability (i.e., the variability of activity in response to multiple presentations of the same stimulus), and to expand the concept repertoire, we first investigated synthetic brain activity patterns output by a brain-optimized encoding model. The encoding model, known as GNet [16], is a deep neural network that was trained end-to-end using measured fMRI brain activity patterns in the NSD. GNet outputs a predicted brain activity pattern in response to each input image. Using GNet, we were able to construct few-shot classifiers from noiseless predicted brain activity patterns for roughly one million concept pairs from the ImageNet database. For each region of interest (ROI) and pair of concepts, we resampled exemplars to construct multiple 5-shot linear classifiers, computed their errors (the fraction of incorrect classifications on a held-out test sample), and then averaged errors across the classifiers.

In each ROI, error decreased with the semantic distance (number of “hops” between concepts on the WordNet graph [19]) between the concept pairs (Fig 3A). Across ROIs, error induced an unsurprising posterior to anterior ordering that was consistent across most levels of semantic distance (Fig 3B). Under the ordering induced by the largest semantic distance (darkest curve in Fig 3B) classification error decreases rapidly across early and intermediate visual areas V1–V4, then more subtly across a collection of more anterior lateral (MST, MT, LO), dorsal (V3AB) and ventral (PHC, VO) ROIs, and terminates with an ROI spanning much of the intraparietal sulcus (IPS), suggesting that much of the geometric variation that supports accurate linear classification in high-level areas occurs across early and intermediate visual areas.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Variation in few-shot classification error across predicted human visual cortex.

All results are based on the outputs of our encoding model. (A) Average 5-shot error (1 - fraction of correctly classified exemplars; y-axis) as a function of WordNet path distance (x-axis; see inset) between the concept pairs in all ROIs (colored curves) considered here. (B) 5-shot error across ROIs (x-axis) for pairs of concepts across a range of WordNet distances (colors). (C) Top left: 5-shot success (1 - error; color bar) matrices for V1 and PHC. Top right: WordNet distance matrix for all concept pairs. (D) Correlation (y-axis) between 5-shot success and WordNet path distance for all pairs of concepts in ImageNet across ROIs (x-axis). (E) Symmetry of the 5-shot error matrices across ROIs as in (D).

https://doi.org/10.1371/journal.pcbi.1013416.g003

As error decreased across this sequence, the concept-by-concept matrix of classification accuracy (1 - error) increasingly resembled concept-by-concept semantic distances in WordNet (Fig 3C). Note that although semantic distances are symmetric on the WordNet graph, classification errors need not be. For example, a classifier that discriminates between “dog” and “cat” may correctly identify “dog” exemplars more often than it correctly identifies “cat” exemplars. Interestingly, classification error matrices became more symmetric with progression along the sequence of ROIs (Fig 3C, inset), and the similarity between few-shot accuracy and hierarchical distance continues to increase even after symmetry plateaus. This suggests that the increase in alignment between 5-shot error and WordNet distance is not fully explained by an increase in the symmetry of 5-shot errors alone.

Having calculated the average few-shot classification error for each ROI, we then confirmed that the expression for geometric SNR (Eq 1) correctly predicted classification error in our particular setting—i.e., concept manifolds that are sets of brain activity patterns measured in the human visual cortex using fMRI. We first confirmed the theory for measured brain activity patterns. To do this, we divided images in the NSD into 12 super-category concepts (see legend in 2B). For each concept, we selected 580 activity patterns and used these to calculate, for each pair of concepts, the geometric SNR as specified by Eq 1. We found that geometric SNR was an extremely accurate predictor of 5-shot error in our setting (Fig 4A, left and right).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Predicting few-shot classification error from geometric properties of concept manifolds.

(A) (left) Theoretical relationship (dashed curve) between 5-shot error (y-axis) and geometric SNR (x-axis) compared to empirical measurements of each for the concept pairs {“person”, “food”} (orange), {“person”, “vehicle”} (green), {“person”, “sport”} (red), and “person” with all other categories (blue) calculated directly for brain activity patterns (dots) in response to presentation of images containing instances of the various concepts. (right) All pairs of concepts listed in Fig 2A. (B) Same format as A where the geometric SNR and 5-shot error has been calculated using synthetic brain activity patterns output by an encoding model trained to predict brain activity in visual cortex. (C) SNR for pairs of concepts using activity predicted by the encoding model (x-axis) and measured by fMRI (y-axis) across all ROIs (colored curves). (D) Binned average of empirical 5-shot error (x-axis) as a function of average geometric SNR per the theory of Sorscher et al. (dashed lines) and for each ROI (colors). Curves have been shifted upward by 10% in order to disambiguate them. The diamond symbol shows the mean 5-shot error and mean SNR[5] over all concept pairs.

https://doi.org/10.1371/journal.pcbi.1013416.g004

We then confirmed the theory for synthetic brain activity patterns output by the GNet encoding model and found that, again, geometric SNR predicted the variation in 5-shot error across many pairs of concepts. In all areas, geometric SNR derived from activity patterns output by our encoding model was strongly correlated with–but much higher than–geometric SNR derived from measured activity (Fig 4C). We interpret the increase in geometric SNR for the encoding model outputs as evidence that the encoding model is more effective at removing trial-to-trial variability in the brain activity than simple averaging, and that reducing trial-to-trial variability in the brain activity measurements increases geometric SNR (see S6 Fig). For this reason, in subsequent analyses we foreground results based on the outputs of our encoding model. Finally, in what follows we will often consider the average of the classification error and other geometric properties across many pairs of concepts within a single ROI; thus, we confirmed that averaged geometric SNR predicted the average classification error (Fig 4D).

Variation in few-shot classification accuracy across visual cortex depends primarily upon variation in geometric Signal

Having confirmed the predictive validity of the expression for geometric SNR in our particular setting, we next studied the profile of variation in geometric SNR and all other geometric properties identified by the theory of Sorscher et al. across our brain ROIs (Fig 5). For each ROI, we averaged geometric SNR across pairs of ImageNet concepts. The ordering induced by average geometric SNR was identical to the ordering induced by 5-shot classification error from concepts separated by more than 23 “hops” on the WordNet graph. We also averaged geometric SNR across pairs of COCO super-categories, inducing a largely consistent (although not identical) ordering of ROIs.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Variation in the geometry of concept manifolds across human visual cortex and layers of deep neural networks.

(A) The average (black dots) geometric SNR, Signal, Bias, combined Overlap and Dimensionality for each subject (colors; thick black curve is cross-subject average; maximum width of each violin shape indicates modal value, extrema indicate 5th and 95th percentiles) estimated from predicted brain activity patterns and 1000 ImageNet categories for all ROIs ordered by ascending SNR. (B) Same format as in (A), but for COCO super-categories. (C) Geometric properties estimated from activity patterns in deep neural networks for layers ordered according to relative depth (the final readout layer has relative depth of 1). All panels indicate geometric properties estimated directly from images pixels (gray violin). Note the change in scale (y-axis) across panels.

https://doi.org/10.1371/journal.pcbi.1013416.g005

To understand effects of geometric properties on geometric SNR, we calculated average geometric properties in each ROI and plotted them in order of ascending SNR. Per the theory, for geometric SNR to increase across any transition between ROIs, at least one of the following must happen: geometric Signal increases, Dimensionality increases, or geometric Overlap decreases (assuming geometric Bias does not change). We found that no single geometric property could explain the increase in SNR across all transitions in this ordering of ROIs. However, we found that variation in geometric Signal had a strong and significant tendency to increase SNR (correlation of between geometric Signal and SNR, p < 0.001), while variation in Dimensionality had a tendency to decrease SNR (, p < 0.05). This means that geometric Signal is the only geometric property whose variation across visual cortex has a tendency to effectively increase geometric SNR. In addition, Signal was greater in each ROI than in pixel space (gray distribution on the left in each panel of Fig 5), while Dimensionality was smaller and Overlap was greater in each ROI than in pixel space. This means that only geometric Signal can explain why few-shot classifiers trained on brain activity patterns are more accurate than few-shot classifiers trained directly on image pixel values.

Importantly, we found that the largest Dimensionality for any concept manifold was no greater than 30, which is far less than the total number of voxels in any ROI studied here. In general, although manifold Dimensionality did show a tendency to increase with ROI size across subjects, the average number of voxels in each ROI alone could not explain average Dimensionality (Fig S1B). For example the average Dimensionality of IPS and LO are about the same (∼4), but IPS has 4 times more voxels than LO (on average). Thus, variation in Dimensionality across ROIs was not a simple consequence of the differences in size between ROIs (the same goes for 5-shot classification error (S1A Fig)).

To determine if the dependence of geometric SNR upon geometric Signal was unique to human brain activity, we investigated the profile of variation in geometric SNR for our COCO supercategories across layers of deep neural networks optimized to perform n-way classification of ImageNet object categories. We found that average geometric SNR increased nearly monotonically across network depth. Geometric properties varied smoothly and monotonically across early and intermediate layers, then abruptly changed direction. For example, geometric Signal decreased smoothly from the pixel layer up through 50% of the layers in AlexNet and 70% of the layers in ResNet50 before abruptly increasing across the final layers. Similar abrupt transitions were seen in the late layers for Dimensionality and Overlap. In contrast to what we observed in the brain, variation in geometric Signal was uncorrelated () with geometric SNR across early layers of the neural networks (first 50% of layers), while Dimensionality was positively correlated (). Thus, the variation of manifold geometry across early and intermediate layers of neural networks was, by our particular analysis, exactly the opposite of what was observed in the brain.

We concluded that increases in geometric SNR (and, therefore, few-shot concept classification accuracy) across visual cortex depends most strongly upon increases in geometric Signal and that, while this dependence is not unique to the human brain, it is also not inevitable: neural networks achieve increases in geometric SNR across early layers using a very different strategy than that observed in our human brain data, or in the later layers of the networks themselves.

Variation in manifold geometry is structured by strong correlations between geometric signal, dimensionality, and overlap

Inspection of the profiles of variation in geometry across brain ROIs and network layers revealed that when average Signal increased, average Dimensionality tended to decrease, and average Overlap tended to increase. Similarly, when average Dimensionality increased, average Overlap tended to decrease. These observations inspired us to investigate correlated variation in geometric properties across pairs of concepts within brain ROIs (and DNN layers) and variation in average geometric properties across brain ROIs (and DNN layers).

Perhaps most strikingly, we found that that variation in geometric Overlap across pairs of concepts with brain ROIs and DNN layers was almost perfectly explained by variation in geometric Signal and Dimensionality. Specifically, a log-linear transformation of geometric Signal and Dimensionality explained 97% of the variation in geometric Overlap across concept pairs for both brain and DNN manifolds (Fig 6A).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. Correlations between geometric properties in brains and neural networks.

All estimates geometric properties in brain ROIs are based on the outputs of an encoding model. (A) Combined Overlap predicted (y-axis) by a simple log-linear transform of Signal and Dimensionality vs. actual measured Overlap (x-axis; dots and colors as in panel D, all brain ROIs are blue dots). The simple model accounts for 97% of the variance (R²) in Overlap across all concept pairs, ROIs, subjects and models. (B) Dimensionality (D_a, log scaled y-axis) vs. geometric Signal (S_a,b, log scaled x-axis) for all pairs of COCO supercategories concepts (dots) for all subjects and in all ROIs (colors, panel D legend; large dots indicate ROI-wise average). (C) Same as panel B but with the concept pairs colored according to geometric SNR (color bar in panel D); lines of constant SNR value (“iso-SNR lines”) are drawn. (D) Average log Signal (x-axis) and Dimensionality (y-axis) for ROIs (box; same data as box in panel C) and neural network layers (“-R” indicates networks with randomized weights) on a map of SNR (colorbar). Iso-SNR lines are drawn; trained and untrained networks are labeled.

https://doi.org/10.1371/journal.pcbi.1013416.g006

Across ROIs and layers, average geometric properties were strongly correlated. Specifically, average Dimensionality and average Overlap showed a strong negative correlation (Fig 5) (, p < 0.001). This was an example of “coordinated” covariation, since it indicated that changes in average Dimensionality and average Overlap tended to have the same effect on average SNR. In contrast, average Dimensionality and average geometric Signal, which were strongly and significantly negatively correlated (, p < 0.001), revealed a form of “antagonistic” covariation, since increases in average Dimensionality (that would increase SNR if all other geometric properties remained fixed) tended to co-occur with decreases in average Signal (that would decrease SNR if all other geometric properties remained fixed; see S5 Fig for a complete table of correlations between geometric properties).

To further investigate the patterns of covariation between geometric properties, we visualized the values of geometric properties in the Signal, Dimensionality (S,D)-plane. This visualization revealed quite clearly the strong negative correlation in the ROI-wise averages of geometric Signal and Dimensionality mentioned above, (larger circles in Fig 6B), while revealing that within a single ROI, the Signal and Dimension for pairs of concept manifolds vary more independently from pair to pair (colored dots in Fig 6B). These observations show clearly that the correlation between Dimensionality and Signal is not an inevitable relationship between geometric properties of manifolds, but rather a relationship between average geometric properties of manifolds that holds across ROIs.

The nearly deterministic relationship of Overlap to Signal and Dimensionality, and the lack of change in Bias, means that the average geometric SNR an any brain ROI and DNN layers can be computed from average geometric Signal and Dimensionality. We leveraged this finding by mapping geometric SNR across the (S,D)-plane (Figs 6C,D).

The map of geometric SNR across the (S,D)-plane showed very clearly that the largest SNR increases occur across the earliest transitions, from pixel space to V1 and V2. There was almost no increase in SNR across intermediate areas or high-level areas (MST, V3ab, PHC, VO, LO, IPS), where variation in the (S,D)-plane was parallel to lines of fixed geometric SNR (“iso-SNR” lines in Figs 6C,D). Thus, because of the antagonistic co-variation between Signal and Dimensionality, most of the variation in the average geometric properties of concept manifolds across higher visual cortex has little impact on classification error.

Different profiles of geometric variation for human visual cortex and deep neural networks

To visualize how the geometry of concept manifolds in human visual cortex and deep neural networks might differ, we also overlaid the average Signal and Dimensionality of each layer in several task-optimized DNNs and randomized DNNs onto the map of geometric SNR in the (S,D)-plane.

This visualization highlighted the differences in brain and DNN geometry noted earlier. The increase in SNR with the transition from pixel space (black square in Fig 6D) to the early layers of optimized DNNs comes with an abrupt increase in Dimensionality, and no net increase in Signal. In contrast, the increase in SNR with the transition from pixel space to the early visual cortex was driven entirely by increased Signal, and occurred in spite of a decrease in Dimensionality. Interestingly, our visualizations show that after the initial increase of Dimensionality across early layers, networks abruptly switched to a geometric profile similar to what we observed in the brain: across later layers geometric SNR increases were driven entirely by increased Signal, and occurred in spite of decreased Dimensionality. Thus, the profile of geometric variation across early network layers appears to be an anomaly that distinguishes it from later layers and has no counterpart in human visual cortex.

For randomized DNNs we found that, as with trained DNNs and visual cortex, Signal and Dimensionality of pairs of concept manifolds jointly determined Overlap, while Signal and Dimensionality were negatively correlated, so that most of the variation in concept manifold geometry was along the iso-SNR lines. This suggests that the “antagonistic” covariation of average Signal and Dimensionality is a common property of both optimized and random networks. Unlike trained DNNs and visual cortex, the randomized networks did not achieve a positive net change in SNR across layers, as expected.

Ethics statement

All data acquisition procedures were approved by the University of Minnesota Institutional Review Board (IRB ID 1508M77463). Formal written consent was obtained from each participant.

Dataset acquisition

All models were trained on the Natural Scenes Dataset (NSD [17]). The NSD dataset consists of between 22K and 30K fMRI image-responses per subject (8 subjects). Images were sampled from the Common Objects in Context (COCO) database [18] and displayed at . The experimental design specified that each of the eight participants would view 10K distinct images (3 presentations each), and a special subset of 1K images would be shared across participants (8 subjects × 9K unique images + 1K shared images = 73K unique images); however, not all participants completed the full acquisition, so the final numbers are somewhat smaller. All fMRI data in the NSD were collected at ultra-high field (7T) using a whole-brain, 1.8-mm, 1.6-s, gradient-echo, echo-planar imaging (EPI) pulse sequence.

The image-responses are expressed in terms of betas obtained from a general linear model (GLM) analysis. For this paper, we used GLM results provided with the NSD data release, specifically, the 1.8-mm volume preparation of the data and version 3 of the GLM betas (betas_fithrf_GLMdenoise_RR). This GLM version involves estimating the hemodynamic response function for each voxel, using the GLMdenoise technique for denoising [24], and using ridge regression to improve the estimation of single-trial betas. Betas indicate BOLD response amplitudes evoked by each stimulus trial relative to the baseline signal level present during the absence of a stimulus (“gray screen”). The betas for each voxel in each session were separately z-scored and all sessions were concatenated.

Identification of visual regions of interest (ROIs)s

Retinotopic visual ROIs (e.g., V1,V3,V3,V4) were identified using a separate population receptive field (pRF) retinotopic mapping experiment, as documented in [17]. Retinotopic areas (more generally described as ‘regions of interest’ (ROIs)) were manually drawn based on results of the pRF experiment. These ROIs consist of V1v, V1d, V2v, V2d, V3v, V3d, and V4, and extend from the fovea ( eccentricity) to peripheral locations that exhibit sensible responses in the pRF experiment given the limited stimulus size (the diameter of the pRF mapping stimulus was ).

Higher visual ROIs were labeled with the Kastner altas [25] and provided as a reference as part of NSD. If there was any discrepancy, we gave priority to the more accurate pRF mapping labels.

Due to hardware limitation during the joint encoding model optimization, only voxels that were labeled as described above were selected. The total number of voxels (cumulative over ROI) was 11838, 10325, 11356, 9470, 9565, 11827, 9162, 10178 for subjects 1–8 respectively, totaling 83721 voxels.

General encoding model architecture

Brain-optimized encoding models consisted of a shared feature extractor (a DNN) and multiple read-out heads as detailed in the following sections and described in Allen et al. [17] and in greater details in St-Yves et al. [16].

Read-out heads.

Read-out heads convert features output by the feature extractor into predictions of brain activity, , for each voxel v. These predictions can be expressed as a linearized model

(2)

where

(3)

and where denotes the concatenation along the feature axis . is the value of the “pooling field” for voxel v applied at feature maps in layer l at pixel (i,j). Pooling fields generalize the population receptive field [26] to arbitrary feature maps. All pooling field elements are positive-valued and normalized such that their sum equals to unity.

and were both set to the fully differentiable nonlinearity . This nonlinearity has several interesting and desirable characteristics: 1) it has an expansive and a compressive regime, 2) it is differentiable everywhere, with no discontinuity and 3) it does not plateau over a large range.

Objective and training.

The brain-optimized encoding model was trained jointly on a large section of visually responsive cortex that totalled 83721 voxels across all 8 subjects.

The pre-filtering network was initialized from a task-optimized AlexNet and held constant for the first phase of training (see fine-tuning details in St-Yves et al. [16]. All parameters of the deep feature extractor and the read-out heads are learned jointly via gradient descent with the ADAM optimizer (, , ). The training steps of the deep feature extractor and the read-out heads are alternated to promote stability of the training procedure.

To account for the noise profile and the discrepancies in voxel predictability, we used the following L2-norm weighted loss function

(4)

where is the batchwise Pearson-like correlation of every voxel with a floor of 0.1 to always permit some contribution of the yet-to-be-predicted voxels (i.e. those with low correlation at the onset of training) in the voxel ensemble V. If the model can be trained voxelwise, then this weighting could be ignored but this could affect the learning dynamics (learning rates).

Further details regarding the regularization procedure and fine-tuning can be found in St-Yves et al. [16].

Details on concept manifold geometry calculation

To evaluate concept manifold geometry under the theory of Sorscher et al. [6] we adapted code provided with the original manuscript. Briefly, we represent every set of activity patterns (fMRI, encoding model, or neural network) corresponding to P examplars from a concept a in a V-dimensional feature space as a P-by-V matrix X^a. We calculate the centroid and perform principal component analysis to calculate the variance explained a_i along the principal component direction for . Likewise for any other concept b.

From there, we calculate the radii , average concept radius , normalized pairwise centroid distance with respect to concept a, and normalized direction matrices

(5)

(6)

from which all other terms involved in Eq 1 are expressed. The geometric Signal and Bias are given by and , respectively. The concept-wise Dimensionality is defined by the participation ratio . The geometric Overlap terms are given by , and where is the Frobenius matrix norm, which is the sum of the square of every entry. We did not consider any terms in the theory involving power-of-m greater than 1, where m is the number of examplars defining the concept prototype under m-shot learning.

Datasets, geometry and empirical few-shot accuracy

We initially performed the geometry estimation in the native voxel space and found that the highest participation ratio dimension was of the order of 20–30. Based on this result, and to facilitate future geometry estimation, we followed [6] and randomly projected all ROI voxel populations into a 300-dimensions random subspace. Given the relatively large number of dimensions of this subspace w.r.t. the dimensions of interest, we found that all geometry estimates were accurately recovered as predicted by the theory (see S3 Fig).

Once the encoding model had been trained, we compared geometric estimates derived from synthetic brain activity patterns output by our encoding model to estimates derived from directly measured brain activity (which was used to train the model, incidentally). For this analysis, visual concepts were the 12 ‘thing’ super-categories of the COCO dataset (Rev. 2017). The COCO super-categories are: ‘appliance’, ‘accessory’, ‘kitchen’, ‘person’, ‘indoor’, ‘outdoor’, ‘vehicle’, ‘electronic’, ‘sports’, ‘food’, ‘furniture’ and ‘animal’. Each of these super-categories contains several constituent COCO sub-categories. For example, the ‘outdoor’ super-category includes the sub-categories ‘traffic light’, ‘fire hydrant’, ‘stop sign’, ‘parking meter’ and ‘bench’. These are only loosely associated, and it should be understood that the super-category ‘outdoor’ here stands for this collection of constituents, not to a broader concept of ‘outdoor’ (e.g., like whether the image depicts an outdoor scene). This is not an issue for our analysis since the concepts are arbitrary and only serves as probes for the representational geometry. For more details, see S2 Fig or refer to the 2017 COCO things annotation files found at https://cocodataset.org/. Note that images in the COCO dataset may contain multiple objects corresponding to different super-categories. If any of pixel in a COCO image (cropped as described in Allen et al. [17]) depicted a super-category, it was labeled as belonging to that super-category. Therefore, any one image in the COCO dataset may contributed to multiple concept manifolds. This explains the decrease of geometric SNR of COCO supercategory concept manifolds relative to ImageNet concept manifolds.

When comparing geometric estimates derived from encoding models and direct brain activity measurements, we did not need to use the same training/validation dataset split used for training of the encoding model, since the question here is not one of cross-validation but of characterization. However, we did need to validate the geometric theory. In order to do so, we performed multiple folds of geometry estimation and empirical few-shot error calculation. For each subject, the roughly 10k image responses are randomized and divided into two groups of roughly 90% and 10% of samples. The geometry is calculated on each concept member in the larger group and the smaller group is randomized and further divided into a group of m sample per concept and those remaining. The m-shot prototype is calculated and used to classify the remaining samples of the smaller group according to the few-shot accuracy classification rule. This latter randomization is performed 16 times and the result is averaged. The larger randomization loop is also performed 16 times and the average and standard deviation of the geometric characteristic, SNR and empirical m-shot accuracy is calculated. Fig 4 shows the average empirical 5-shot accuracy plotted against the average SNR calculated in this manner. Fig 5 shows the average geometric SNR components: the Signal, Bias, noise-signal Overlap and noise-noise Overlap calculated in this manner, averaged over pairs of concepts.

We also estimated geometric properties for concept manifolds corresponding to the 1,000 classes of the ImageNet 2012 database [27]. Note that this was only possible using an encoding model, as ImageNet stimuli were not included in the NSD. For this analysis, we kept the original training and validation image separation. The geometry was calculated over the larger training set samples for each category. We randomly selected P = 700 images from each category to estimate the geometry. The ImageNet validation set was used to calculate a multi-fold 5-shot accuracy in the manner described above, with the number of folds set to 16. We also performed the geometry estimation in the original space in each ROI. The results highlighted the invariance of the geometry estimates under sufficiently high dimensional random projections.

Extracting layerwise DNN concept geometry

We looked at two networks’ internal representations at various stages along the network hierarchy. In most cases, We extracted network activities after the activation function following either a convolution or fully connected layer, which occurs in any network at relatively regular intervals. These internal representations (features) were randomly sampled into a lower dimensional space of 50K dimensions (or the full space if the dimension was lower) and then randomly projected onto 2,000 dimensions. This was done in order to reduce the computational (and memory) cost of the random projection for very large layers. Comparison with and without this random projection in case where not applying it could be afforded showed extremely comparable values for all the geometric estimates. Likewise, the choice of the number of projection dimension was such that all estimates were sufficiently close to convergence. In all cases, stability of the relevant estimates was the deciding factor for any embedding dimension.

All models parameters were imported from the pyTorch (v1.10.0+cu113) torchvision (v0.11.1) model library. For all models, we performed the analysis of concept geometry evoked by the NSD COCO images with the trained model weights (trained) and with a randomized version of the trained weights where all network parameters have been shuffled (untrained).

The geometry estimation was performed 9 times for each extracted layers with a new shuffling of the weights for the untrained variant, and different randomly sampled features and random projection each time the analysis was performed. The values plotted in Fig 5 and Fig 6 show the mean and std. dev. over these 9 estimates. The “trained” model had std. dev. on their estimate of Dimensionality and Signal smaller than the symbol size (Fig 6), indicating the high degree of consistency of these geometric properties w.r.t the feature sampling procedure. Therefore, the sampling step that introduced the greatest source of variance in this procedure appears to be the shuffling of the parameters in the “untrained” model estimate. All other sampling offered very robust estimates.

Combined geometric overlap term

We initially computed signal-noise Overlap and noise-noise Overlap as defined by Sorscher et al. [6]. However, in all network and brain manifolds, these properties were highly correlated. Therefore, in Fig 5 we described the combined geometric Overlap term

(7)

and later in Fig 6 we predicted this term as a function of Signal and Dimension. Note that this combined term depends on the number of examplar m in the concept prototype. Therefore, the regression parameters will be dependent on m since Signal and Dimensionality are independent of m.

Relating signal, dimensionality, and overlap

After performing principal component analysis on the space of representational geometry Signal, Bias, Dimensionality and combined Overlap), we observed empirically that two factors accounted for most of the variance (greater than 92% of variance) in all the representations that we measured (brains and DNNs). These factors are in the plane of Signal and Dimensionality which implies that all other properties can be relatively accurately described as a linear combination of these two properties. The Bias term being essentially independent and constant, this leaves only the combined Overlap term which we write as

(8)

A 2-dimensional linear regression of this functional form for all model estimate in Fig 6A jointly yielded a coefficient of determination R² = 0.970. By comparison, a linear regression of the combined Overlap term onto the Signal alone produced a correlation of 0.56.

The fitted values of enable one to write SNR as a function of Signal and Dimensionality only. We indicate SNR using using color gradients and isolines in Fig 6A. Note that the isolines are slightly curved in log-log space. The regression can be performed independently for concept manifolds in brain ROIs and DNN layers indepedently, or as a whole. While there are some deviations from one model to another, the results are largely consistent; this finding warrants a comparison of the models in a single common (S,D)-plane as shown in Fig 6D. Fig 6C shows that the relationship between Signal, Dimensionality and SNR also holds for individual concept pairs. Concept pairs whose geometry has been estimated in all brain ROIs have been combined in this plot which shows that the 2D determination is obeyed at the concept level in all ROIs simultaneously.

Symmetry of 5-shot error matrices

Fig 3C shows the correlation between the ImageNet hierarchical distance matrix and the few-shot success rate matrices. The ImageNet hierarchical distance matrix is always symmetric, whereas the few-shot accuracy matrix need not be. To measure the symmetry of few-shot error matrices, we calculated the correlation between the lower and upper triangular component of the few-shot accuracy matrices (ROI-wise). The result is shown in the inset of Fig 3C.

Correlations of SNR and SNR components

While the geometrical SNR is determined by Eq 1, the geometric Signal, Bias, Overlap and Dimensionality all depends on common factors which induces correlations in these quantities, therefore a simple look at the dependence of geometric SNR on its components does not reveal the net influence that changing these properties have on geometric SNR.

Therefore, we calculated the empirical Pearson correlations between the geometric components and SNR for the mean of ROI (layer for networks) estimate and for individual concept manifold pairs, for all ROIs (layers) and subjects (samples) concatenated together (referred to in the text and shown in S5 Fig). For the networks, we concatenated layers from AlexNet and ResNet50 together in a single estimate since both showed similar trajectories. Significance was estimated by performing 1,000 two-sided permutation test.