How well do models of visual cortex generalize to out of distribution samples?
Fig 2
A: Scatter plot of normalized measured responses and their corresponding predicted values from a layer in ResNet50 model for two sample neuronal sites. The neuronal model’s generalization capability is highly variable across neurons. Left and right plot show two examples neuron with high and low generalization respectively. B: Scatter plot of Nat. and Syn. predictivity scores for a neuronal model based on ResNet50 unit activations for all neuronal sites with high internal consistency (larger than 0.7). The corner histogram shows the distribution of the difference between in- and out-of-distribution predictivity scores across neuronal sites; C: Neural predictivity score of the ResNet50 neuronal model and internal consistency of the neural data in naturalistic and synthetic domains for neural data collected from different animals (M, N, and S) and different recording sessions (S1–4); D: Bar plot of Nat. and Syn. predictivity scores as well as the neural predictivity gap for 7 different neural network models; E: Comparison of Nat. and Syn. predictivity scores for ResNet50 model when the regression model was fitted on naturalistic data (left) and synthetic data (right). The regression model fitted to the synthetic domain shows worse generalization to the naturalistic domain. All error bars denote the variance across 5 repetitions of each analysis.