Fig 1.
Tests of invariant and configural brain representation.
A: Varying degrees of change in neural encoding as a function of a change in context. With a change in context, context-invariant representations do not change at all, whereas context-specific representations change completely, with a continuum between both extremes. Tests in the literature focus on evidence against one of the two extremes. B: Tests of context invariance and specificity. Steps 1 and 2 are common to all tests. Different tests differ on how invariance/specificity is evaluated in step 3. The figure depicts distributions of classifier decision variables and the areas of these distributions on which each test focuses (in gray). C: Representations are transformed from the space of neural activities to the space of voxel measurements. Context-invariant representations (top) cannot be transformed to decrease their invariance and increase their specificity, whereas context-specific representations (bottom) can be transformed to increase their invariance and decrease their specificity. D: Example highlighting the differences between spatially smooth versus fine-grained encoding schemes, and a particular combination of the two schemes that produces false-positives in a voxelwise analysis. Each column represents a voxel containing neurons (small circles), each with selectivity for one of two values of the target property (red and yellow). The multivoxel pattern of activity is the same for both levels of the context dimension (spatially smooth encoding), but completely different populations of neurons encode each level (fine-grained encoding). This figure includes public domain clipart and all other parts are original: https://commons.wikimedia.org/wiki/File:Gaussian_distribution.svg https://www.wpclipart.com/signs_symbol/arrows/BW_arrows/arrow_BW_thick_left.png.html.
Table 1.
Lookup table summarizing how joint tests against specificity and invariance should be interpreted.
Fig 2.
Stimuli were composed of oriented gratings (dimension 1) presented in a windowed spatial position (dimension 2). Each trial consisted of a single combination of oriented gratings and spatial position. Training runs were composed of stimuli presented only in top-right and bottom-left spatial positions (highlighted through red and blue boxes). Testing runs included all sixteen stimulus combinations.
Fig 3.
Classification accuracy results with test data for the decoding of spatial position.
Each row represents a complete analysis for a single subject. Columns represent different levels of the context dimension (orientation) held fixed during training and the dotted line represents chance performance. Group descriptive statistics (mean and standard errors) are presented in the bottom row. Also shown are results of significant cross-classification and classification accuracy invariance (pairwise comparisons, none significant) tests.
Fig 4.
Classification accuracy results with test data for the decoding of orientation.
Each row represents a complete analysis for a single subject. Columns represent different levels of the context dimension (spatial position) held fixed during training and the dotted line represents chance performance. Group descriptive statistics (mean and standard errors) are presented in the bottom row. Also shown are results of significant cross-classification and classification accuracy invariance (pairwise comparisons) tests.
Fig 5.
Decoding separability test results with spatial position as the target dimension.
Each row represents a complete analysis for a single subject. Columns represent different levels of the context dimension (orientation) during training. The y-axis shows the statistic and bars represent 90% bootstrap confidence intervals. The dotted line and surrounded gray area represent the expected value and 90% bootstrap confidence interval for the
statistic when no differences exist between two distributions. Group descriptive statistics (mean and standard errors) are presented in the bottom row.
Fig 6.
Decoding separability test results with orientation as the target dimension.
Each row represents a complete analysis for a single subject. Columns represent different levels of the context dimension (spatial position) during training. Values in the x-axis represent levels of the context dimension (spatial position) during testing (TR: top-right; BR: bottom-right; TL: top-left; BL: bottom-left). The y-axis shows the statistic and bars represent 90% bootstrap confidence intervals. The dotted line and surrounded gray area represent the expected value and 90% bootstrap confidence interval for the
statistic when no differences exist between two distributions. Group descriptive statistics (mean and standard errors) are presented in the bottom row.
Fig 7.
Description and results of simulation 1.
A: Encoding models used in simulation 1. B: Steps taken in each repetition of simulation 1. See main text for details. C: Classifier accuracy scores for model-generated data from both levels of the context dimension. The y-axis represents accuracy scores, the x-axis represents level of measurement noise (in units of standard deviation), the dotted line represents chance performance. D: Proportion of positive tests of each type. The y-axis represents proportion of positives, the x-axis represents measurement noise, the dotted line represents the accepted false discovery rate of 5%. Panels E-F show the proportion of each type of conclusion in Table 1 (specificity/sensitivity in red, invariance/tolerance in blue, and no conclusion in green) reached from jointly testing against specificity and invariance. In both cases, the cross-classification test is used against specificity. E: Conclusions reached by using the classification accuracy invariance test against invariance. F: Conclusions reached by using the decoding separability test against invariance. This figure includes public domain clipart and all other parts are original: https://freesvg.org/binary-file-vector-graphics.
Fig 8.
Pervasiveness of the problem of false positive invariance for the extreme case of context-specificity studied in simulation 1.
Proportion nullity represents the proportion of all dimensions in the measurement space that would produce false positive invariance, and therefore the size of the false positive invariance problem. The values reported were always the same for a given combination of number of neural channels and number of stimuli, across 200 randomly sampled encoding models.
Fig 9.
Description and results of simulation 2.
A: Encoding model for simulation 2. See main text for details. Panels B-E show the decoding results from simulation 2. B: Classifier accuracy scores for model-generated data from both levels of the context dimension. The y-axis represents accuracy scores, the x-axis represents magnitude of noise added to measurement weights for the second level model, the dotted line is chance performance. Proportion of positive tests of each type. The y-axis represents proportion of positives, the x-axis represents measurement noise, the dotted line represents the accepted false discovery rate of 5%. Panels C-D show the proportion of each type of conclusion in Table 1 (specificity/sensitivity in red, invariance/tolerance in blue, and no conclusion in green) reached from jointly testing against specificity and invariance. In both cases, the cross-classification test is used against specificity. D: Conclusions reached by using the classification accuracy invariance test against invariance. E: Conclusions reached by using the decoding separability test against invariance.
Fig 10.
Description and results of simulation 3.
A: Illustration of the L1 (blue) and (red) distances between two distributions. The green dotted line represents the optimal classification bound (i.e., the value with equal likelihood to belong to either distribution). The proposed invariance coefficient represents the proportion of overlap between the two distributions (red area over the sum of red and blue areas). B: Correlation between the continuous level of change implemented in the encoding model and
. C-D: Construct validity of the two versions of
(i.e., their correlation with the true value
): one version computed by decoding stimulus values (panel C) and another by decoding level of context (panel D).
Fig 11.
Decoding distributions cannot be used to obtain a valid test of no overlap between neural representations across two contexts.
The main axes represent measurements at the voxel level, and each ellipse represents the distribution of neural activity (after transformation by the measurement model) for a target stimulus property presented in two different contexts. The two distributions are completely non-overlapping at the level of the multivariate voxel patterns. However, when the two distributions are projected onto the decoded variable, they show a non-zero overlap represented by the yellow area.
Fig 12.
Model used in our simulations.
A: Population encoding model, consisting of a set of channels that are tuned to specific stimulus values along a given dimension (e.g., maleness). When a stimulus with a particular value on the maleness dimension is presented, the channels respond according to their stimulus preferences. The channel responses are then perturbed by random channel noise. The final output represents a vector of noisy firing rates in response to a particular stimulus. B: Linear measurement model. The measurement model provides a link between neural encoding channels and voxel-wise activity measures. Activity in each voxel (represented by cubes) is a linear combination of neural channel responses. This figure includes public domain clipart and all other parts are original: https://creazilla.com/nodes/29498-white-dice-with-black-spots-clipart.