How well do models of visual cortex generalize to out of distribution samples?
Fig 3
Assumptions on brain-model correspondence affect generalization in neural predictivity.
A: Neural predictivity scores from unit activity in each layer of ResNet50 architecture for individual neuronal sites recorded during example sessions from different animal subjects. From top, rows correspond to M-S1, N-S1 and S-S1. Colors correspond to the neural predictivity score on natural (green) and synthetic (blue) domains. Different shades correspond to different neuronal site in the same animal. Bold lines correspond to the average predictivity score in each domain across all neuronal sites within that animal’s session; B: Number of neurons with highest neural predictivity in a given layer corresponding to the same subplot in a. Colors are the same as in a; C: Distribution of the difference between the layer number in ResNet50 neural network where each neuronal site is best predicted in-distribution and out-of-distribution. The difference is calculated as the best ID layer number—the best OOD layer number. The distribution spans a wide range but has a slightly negative mean (-1.29); D: Comparison of neural predictivity scores on natural and synthetic domains when a brain-model correspondence follows a Layer-Area (LA) or Layer-Neuron (LN) mapping assumption.