Stimuli and design.

Stimuli consisted of the full-length feature movie—“Raiders of the Lost Ark”—divided into eight parts of approximately 14 to 15 min duration. Video was projected onto a rear projection screen with an LCD projector which the subject viewed through a mirror on the head coil. The video image subtended a visual angle of approximately 22.7° horizontally and 17° vertically. Audio was presented through MR Confon’s MRI-compatible headphones. Participants were instructed to pay attention to the movie and enjoy. See [16] for details.

fMRI protocol.

Participants were scanned in a Philips Intera Achieva 3T scanner with an 8 channel head coil at the Dartmouth Brain Imaging Center. T1-weighted anatomical scans were acquired at the end of each session (MPRAGE, TR = 9.85 s, TE = 4.53 s, flip angle = 8°, 256 × 256 matrix, FOV = 240 mm, 160 1 mm thick sagittal slices). The voxel resolution was 0.9375 mm × 0.9375 mm × 1.0 mm. Functional scans of the whole brain were acquired with an echo planar imaging sequence (TR = 2.5 s, TE = 35 ms, flip angle = 90°, 80 × 80 matrix, FOV = 240 mm × 240 mm) every 2.5 s with whole brain coverage (41 3 mm thick interleaved axial slices, giving isotropic 3 mm × 3 mm × 3 mm voxels). We acquired a total of 2718 functional scans with 1350 TRs in four runs during the first session and 1368 TRs in four runs during the second session.

fMRI data preprocessing.

fMRI movie data were preprocessed using AFNI software [42](http://afni.nimh.nih.gov). Functional data were first corrected for the order of slice acquisition and head motion by aligning to the last volume of the last functional run. Any spikes in the data were removed using 3dDespike in AFNI. Data were then filtered using 3dBandpass in AFNI to remove any temporal signal variation slower than 0.00667 Hz, faster than 0.1 Hz or that correlated with the whole brain average signal or the head movement parameters. Each subject’s anatomical volume was first aligned to the motion corrected average EPI volume and then to the MNI 152 brain template in AFNI. Functional EPI BOLD data were then aligned to the MNI 152 brain template using nearest neighbor resampling by applying the transformation derived from the alignment of the anatomical volume to the template. Data acquired during the overlapping movie segments were discarded resulting in a total of 2662 TRs with 1326 TRs in the first session and 1336 TRs in the second session.

Definition of masks and searchlights for movie data.

We derived a gray matter mask by segmenting the MNI_avg152T1 brain provided in AFNI and removing any voxel that was outside the cortical surface by more than twice the thickness of the gray matter at each surface node. It included 54,034 3 mm isotropic voxels across both hemispheres. We used this mask for all subsequent analyses of all subjects.

Hyperalignment of movie data started with hyperalignment of data in 20,484 overlapping searchlights of 20 mm radius centered on cortical nodes with 2.9 mm average spacing between the nodes. Cortical nodes were defined in a standard cortical surface from FreeSurfer (fsaverage)(https://surfer.nmr.mgh.harvard.edu) and resampled into a regular grid using AFNI’s MapIcosahedron [42,43] with 10,242 nodes in each hemisphere. We defined the surface searchlights [44] in PyMVPA [45](http://www.pymvpa.org) as cortical disks. The thickness of disks was extended beyond the gray matter, as defined in FreeSurfer, 1.5 times inside the white-matter gray-matter boundary and 1.0 times outside the gray-matter pial surface boundary to accommodate any misalignment of gray matter as computed from the anatomical scan and the gray matter voxels in the EPI scan. To reduce the contribution from noisy or non-gray matter voxels that were included due to this dilation, we used a between-subject correlation measure on training data [13] to select 70% of the voxels in each searchlight [16]. The mean number of selected voxels in movie data searchlights was 235.

Searchlights for defining connectivity targets were defined using a coarse surface grid corresponding to the ico8 surface in SUMA [43] with 1284 nodes (10.7 mm spacing between nodes). We used surface disk searchlights [44] centered on these nodes as the movie data connectivity target searchlights. These searchlights had a radius of 13 mm, as did those used for the HCP data, producing complete coverage of the cortex with overlapping searchlights. Cortical disks centered on these voxels were dilated using the same procedure as for hyperalignment of cortical surface searchlights. Movie connectivity target searchlights had a mean of 99 voxels.

Definition of masks and searchlights for HCP data.

We masked the data to include only the left and right cortices (Cortex_Left and Cortex_Right), removing all the non-zero nodes that correspond to the medial subcortical regions, resulting in 59,412 nodes across both hemispheres. These nodes also defined the centers of 59,412 surface searchlights [44] with 20 mm radii that were used for hyperalignment. All nodes in these searchlights were included. The mean number of surface nodes in the HCP searchlights was 337.

We defined connectivity target searchlights using a coarser surface grid corresponding to the ico8 surface in SUMA [43] with 1284 nodes (10.7 mm spacing between nodes). We found the closest matching nodes on the 32K surface to the nodes on the ico8 surface, and used those as centers for connectivity target searchlights. These searchlights had a radius of 13 mm, producing complete coverage of the cortex with overlapping searchlights. HCP connectivity target searchlights had a mean of 142 loci. See further details below for how time-series were extracted from these searchlights.

For validation of task fMRI, we used all of the maps provided by the HCP after removing redundancies (such as FACE-AVG and AVG-FACE), which resulted in 46 maps (S2 Table).

The basic equation for hyperalignment (both CHA and RHA).

B_ij are the original matrices of data for cortical fields, j, in individual brains, i, which have m_ij columns of cortical loci and n rows of data vectors. Hyperalignment derives a transformation matrix for each cortical field in each individual, R_ij, and a matrix for each cortical field, M_j, that is the mean of transformed individual brain matrices, B_ijR_ij, minimizing the Frobenius norm of differences between transformed individual brain matrices and the model space matrix. For each cortical field j: (1)

For whole cortex hyperalignment we define the cortical fields, j, as searchlights. Thus, we estimate a transformation matrix, R_ij, for each of N_sl searchlights in each subject i. We then aggregate these searchlight transformation matrices into a whole cortex transformation matrix, R_iA (details below): (2)

The whole cortex common model data matrix, M, is created by transforming individual whole cortex data matrices, B_iA, into common model space coordinates and calculating the mean: (3)

Conversely, other subjects’ data in the common model space can be mapped into any subject’s individual anatomical space using the transpose of that subject’s whole cortex transformation matrix, , producing a data matrix, M_i, in which the columns are that subject’s cortical loci, making it possible to analyze and visualize transformed group data in any subject’s anatomical space: (4)

In our implementations of hyperalignment, we have used a variant of Generalized Procrustes Analysis [46,47](described in detail below) to derive orthogonal transformation matrices for the improper rotations of a brain data matrix from a cortical field (region of interest or searchlight) to the mean of others’ matrices for the same region to minimize interindividual differences between the transformed individual and mean data matrices. Aggregation of searchlight transformation matrices, R_ij, produces a whole cortex transformation matrix, R_iA. Because R_iA is derived from searchlight transformation matrices, R_ij, it imposes a locality constraint that limits remapping of brain data to nearby cortical loci (see details below), making the whole cortex transformation matrix nonorthogonal by design. We also have tested other hyperalignment algorithms that use alternatives for calculating the transformation matrices, such as regularized canonical correlation and probabilistic estimation [48,49]. These alternatives are effective but have not yet been extended to aggregate local transformation matrices for cortical fields into a whole cortex transformation matrix.

The dimensionality of the brain and model data matrices is n × m, in which m equals the number of cortical nodes or dimensions in brain and model data matrices—B_ij, B_iA, M_j, and M—and n equals the number of data vectors across these dimensions. The number of data vectors, n, is set and determined by the number of connectivity targets for defining connectivity pattern vectors (see details below). For RHA, n is set by the number of response pattern vectors in an experimental dataset. The number of cortical loci in a cortical field or searchlight, m_ij, can vary across subjects. If the number of cortical loci or dimensions differs between subjects or between an individual subject and the model space, the new subject’s data are transformed into a space with the same dimensionality as the first subject’s or the model’s space. The number of cortical loci in the whole cortex model is set at m = 59,412 for HCP data and m = 54,034 for movie data. We also have shown that the dimensionality of a model for region of interest or searchlight (m_j) can be reduced substantially relative to the dimensionality of individual brain spaces in imaging datasets (m_ij), m_Mj << m_ij [13,16,48]. In the current version of CHA, as in whole cortex RHA, however, we do not reduce the dimensionality of the model space because these reduced dimensionality local models are difficult to aggregate into a whole cortex model.

Note that the common model data matrix has two distinct components. The columns define a common model space, whereas the rows are defined by the experimental data—either patterns of connectivity to targets elsewhere in the brain for CHA, or patterns of response for RHA. The space can be illustrated as an anatomical space insofar as it can be rotated into any individual’s cortical loci (Eq 4), but there is no “canonical” anatomical space, rather the individuality of each individual brain is preserved. We illustrate results in the anatomical space of one subject, the “reference subject”, but we also could illustrate the results in other subjects’ anatomical spaces. The special nature of the common space derives from the alignment of functional indices—connectivities and responses—to minimize interindividual differences and, thereby, discover shared basis functions for the individually variable functional architecture. These basis functions are the response and connectivity profiles for model dimensions that model the response and connectivity profiles of cortical loci in individual brains as linear weighted sums. In other words, Eq 4 models single columns in B_i as weighted sums of columns in M_i.

Transformation matrices consist only of weights for the projection of individual brain spaces for cortical fields, B_ij, or the whole cortex, B_iA, into model spaces (M_j, M) and contain no connectivity or response data. Thus, a transformation matrix can be applied to any matrix of data vectors in an individual brain space. Similarly, the transposes of transformation matrices, , can be applied to any data vector in the model space to project that vector into the cortical topographies of individual brains. For all applications of the common model, including the validation tests presented here, the transformation matrices are applied to independent data that played no role in derivation of the model space and the individual transformation matrix parameters. This is necessary to avoid overfitting [50]. Transformation matrices derived from connectivity data also can be applied to response data and vice versa. In other words, RHA and CHA are complementary methods for deriving a common model of information spaces in cortex, and RHA-derived and CHA-derived transformation matrices are alternative projections for mapping individual brain data into the same common model space. Note that each column of the transformation matrix R_iA contains weights for cortical loci in subject i’s brain. These columns of weights are basis functions for modeling functional topographies in individual brains as linear weighted sums of topographies associated with model dimension functional profiles.

Derivation of transformation matrices for regions of interest and searchlights.

The derivation of individual transformation matrices that map individual brain spaces into the common model space is a three-level iterative process. We present the iterative algorithm for deriving transformation matrices and the common model space in greater detail here to help readers understand better its structure.

In the first step of the first level, the data matrix for a cortical field in one subject, B_2j, is transformed to be in optimal alignment with the same cortical field in another subject’s brain, B_1j, referred to here as the reference subject: (5)

We use the Procrustes transformation to find the orthogonal matrix that affords the optimal improper rotation to achieve this minimization [46]. Note that this “rotation” is a rotation of data in the high-dimensional feature space, not a rotation in a two or three dimensional anatomical space. Elsewhere we have shown that other algorithms can be used to achieve this minimization [48,49].

In the following steps of the first level, the brain data matrices for the third and subsequent subjects are transformed to be in optimal alignment with the matrix defined by the mean of the previous subject’s matrix and the previous mean: (6) where (7)

M_{i-1,j(level1)} is the target data used to hyperalign the current subject’s data, B_ij, and M_{i-2,j(level1)} is the target data used to hyperalign the previous subject’s data, B_i-1,j. Target data is updated with previous subjects’ aligned data in this first level. In the subsequent two levels each subject’s data matrix is hyperaligned to the simple, unweighted mean of all other subjects’ matrices.

At the end of the first level, level one transformation matrices have been derived for all cortical fields in all subjects, R_i,j(level1), which are used to project each subjects’ brain data into the provisional common spaces that evolved over level one iterations M_i,j(level1). Each subject is then re-hyperaligned to the mean data matrix for all other subjects’ transformed data from level one to derive new individual transformation matrices, R_ij(level2). Note that the new transformation matrices are derived using each subject’s original brain data, B_ij. Note also that the mean matrices in provisional common spaces, M_{¬i,j(level1)}, exclude data from the subject being hyperaligned: (8) where M_{¬i,j(level1)} is the equally-weighted mean of level one transformed data for all subjects but subject i: (9)

After the level two transformation matrices, R_ij(level2), are calculated for each subject, the level one transformation matrices are discarded, and the group mean of transformed individual brain data matrices is recalculated, using these new transformation matrices, producing the model matrix, M: (10)

In level 3, the last level, the final searchlight transformation matrices, R_ij, are recalculated for each subject (see Eq 1 above).

Derivation of whole cortex transformation matrices.

Orthogonal transformation matrices for hyperaligning a cortical field can map information from a cortical locus into model dimensions anywhere else in that cortical field. To constrain the remapping of information to nearby locations in the reference subject’s cortical anatomy, we developed a searchlight-based approach [16]. We hyperalign the data in N_sl overlapping searchlights, where N_sl is the number of searchlights (59,412 for HCP data, 20,484 for movie data). The number of model dimensions in each searchlight transformation matrix is determined in the movie data by cortical location and the number of selected features in the reference subject (mean = 235) and in HCP data by the cortical location of the searchlight (mean = 337). The transformation for each searchlight, R_ij, has dimensionality that corresponds to the number of nodes in an individual’s searchlight (m_ij rows) and the number model dimensions in that searchlight, derived from the reference subject’s anatomy (m_1j rows). Thus, each transformation matrix has on the order of 28K and 57K free parameters for movie data and HCP data, respectively. Because the searchlights are overlapping, there are multiple estimates of weights for mapping each cortical locus to each model dimension. As described in Guntupalli et al. [16] these weights are aggregated across searchlights by adding all weights for each cortical-locus-to-model-dimension mapping. In essence, this is equivalent to creating a whole cortex transformation matrix, R_iA, of dimensionality m × m, by padding each searchlight transformation matrix, R_ij, with zeroes in all rows and columns for cortical loci and model dimensions that are not in the individual or model searchlight cortical field to give them the same dimensionality to produce R_ij(padded), and then summing these padded transformation matrices. Thus, for each subject, i: (11) where N_sl is the number of searchlights and R_ij(padded) is the padded transformation matrix for subject i in searchlight j with dimensionality m × m. As noted above, m is the number of cortical loci—59,412 surface nodes for HCP data and 54,034 voxels in the gray matter mask for movie data—and the number of whole cortex model dimensions. Because the searchlight approach constrains cortical-locus-to-model-dimension mapping to nearby cortical locations, the whole cortex transformation matrix, R_iA, is sparse with zero weights for all mappings of cortical loci to model dimensions that are separated by more than 2x the searchlight diameter (~4 cm in this implementation). The whole cortex transformation matrices, R_iA, are large (m × m) but sparse. 98.7% of the entries are zeros, and roughly 20 million entries have nonzero values in each of these matrices. The additive aggregation of mapping parameters weights nearby cortical location pairs, which co-occur in more searchlights than distant pairs, more strongly than distant pairs, adding a further locality constraint. Note that the searchlight transformation matrices, R_ij, are orthogonal but the whole cortex transformation matrices, R_iA, are not by design, to introduce the locality constraint. The whole cortex transformation matrices, R_iA, are used in all validation tests to hyperalign independent new data matrices after normalizing the data in each cortical node or voxel. In other words, all validation tests are performed on independent data that played no role in deriving the transformation matrix parameters or the common model connectome, providing cross-validated generalization testing. CHA of movie data was based on one half of the movie data (~55 min, ~1300 TRs) and the other, independent half of the movie data was used for validation tests with two-fold cross-validation. CHA of HCP data was based on one session of resting state data (~15 min, 1200 TRs) and a second session of independent resting state data was used for validation tests, as well as independent data from task fMRI [18].

Note that transformations map the cortical loci of a subject’s data matrices (columns) into the reference subject’s cortical loci. Thus, we use the reference subject’s cortex for illustration, but note that the anatomical coordinates for model dimensions are an abstraction, as even the reference subject’s data are mapped into model space coordinates with a transformation matrix that is not the identity matrix. Data matrices in the model space also can be mapped into any subject’s cortical anatomy by using the transpose of that subject’s transformation matrix (Eq 4). Thus, the hyperaligned data in the common model space can be illustrated in any subject’s anatomical space. The anatomical space that we use for illustration, that of the reference subject, should not be considered a canonical space but, rather, simply as one of many possible physical instantiations.

Connectivity targets.

We define functional connectivities as the correlations of the response profiles—series of responses across time—of cortical loci or dimensions with the response profiles of targets (t_j) distributed across the cortex. We use two sets of connectivity targets, one reduced set to derive the transformation matrices and common model connectivity data matrix, and a more complete set to test the validity of the model. We define a reduced set of connectivity targets using surface-searchlight target ROIs to make derivation of the model more computationally tractable, as compared to using all cortical loci as individual connectivity targets. For the reduced set, we use 3852 targets (top 3 components for 1284 searchlights; note that the searchlights for connectivity targets are different from the searchlights that are hyperaligned as described above in the Resting State Data and Movie Data sections; see details for defining searchlight PC connectivity targets in the next section). For validation testing we analyze the full connectome, defining connectivity targets as all cortical loci in the brain (N_cl = 54,034 gray matter voxels in the movie data and 59,412 cortical nodes in the HCP resting state data).

Searchlight ROI connectivity targets.

Each surface-searchlight connectivity target has a radius of 13 mm and is centered on a node of a coarse surface with a total of 1284 nodes covering both hemispheres. Thus, neighboring connectivity targets searchlights are overlapping. Unlike others (e.g., [5]) we do not assume that a searchlight connectivity target has a single response profile. We find, rather, a variety of response profiles for individual cortical loci in a target searchlight that can be captured as principal components. We used the top three principal components to represent the response profiles in a target searchlight.

To insure that the top components in target searchlights capture the same connectivity patterns across subjects, we performed a singular value decomposition (SVD) on the group mean connectivity matrix for each target searchlight after a simplified hyperalignment of individual matrices. Note that using a naive PCA/SVD to derive top components in each subject’s searchlight independently will not guarantee their functional similarity. Target searchlights had a mean of 142 loci (HCP data) or 99 voxels (movie data). At this stage it was not yet possible to break the response profiles for searchlight targets into multiple components with shared connectivity profiles. Consequently, connectivity targets for the procedure to derive these components were simply the mean time-series responses for target searchlights. For each target searchlight with N_s features (surface nodes or voxels), we computed a 1284 × N_s correlation matrix (the correlations between each cortical locus in the target searchlight and the mean time series for all target searchlights) for each subject. We hyperaligned the features (cortical loci) in each target searchlight across subjects based on these matrices and calculated the mean correlation matrix after hyperalignment in each target searchlight. We then performed a singular value decomposition (SVD) of each searchlight’s group mean matrix to obtain the top three components that explained the most shared variance. Each of these components is a weighted sum of cortical loci in a target searchlight for each subject, and these weights afford calculation of a time-series response whose connectivity profile with other targets is shared across subjects. Each individual subject’s time-series responses for the top three components were then used as target response profiles for CHA. This step gave us 1284 × 3 = 3852 target response profiles in each subject’s cortex.

Functional ROIs.

In addition to analyzing the results of validation tests in each feature or searchlight across the whole cortex, we also examined the results of validation tests in functional ROIs associated with different sensory, perceptual, and cognitive functions to assess the general validity of the common model [16]. We searched for terms and cortical areas implicated in visual, auditory, cognitive, and social functions in NeuroSynth [22] and took the coordinates for the peak location associated with each of 24 terms (S1 Table). For validation testing using the movie dataset, we used volume searchlights centered around those peak loci with a radius of 3 voxels as our functional ROIs. For validation testing using the HCP dataset, we found the closest surface node corresponding to each peak locus and used a surface searchlight with a 10 mm radius around that surface node as the functional ROI. Functional ROIs that were medial and encompassing both hemispheres in the volume space were split into left and right ROIs in the surface space resulting in 26 ROIs for tests on the HCP data. For analyses of ISCs and PSFs of connectivity profiles in functional ROIs, we calculated the mean ISC or PSF across all cortical loci within the ROI searchlights (Figs 3C, 4D, 5A, and 5C).

Statistics.

We used bootstrapping to test for significance of the contrasts between alignment methods by sampling subjects 10,000 times to compute 95% CIs using BootES [51]. We did this for each ROI and for the mean of all ROIs separately. We used the same bootstrapping procedure for all validation tests unless specified otherwise.

Control for effect of filtering.

In addition to the anatomically-aligned movie data and MSM-All aligned HCP resting state fMRI data, we calculated a third dataset that controls for the effect of filtering the data through CHA transformations but aligns those filtered data across subjects based on anatomical or MSM-All alignment. To produce the filter control data, we created multiple common model connectomes using each subject as the reference. Each subject’s connectome was transformed into the common connectome whose reference subject was the next subject in our order of subjects. The last subject’s connectome was transformed into the common model connectome whose reference brain was that of the first subject. Thus, each subject’s connectome is filtered by hyperalignment, but since the common model connectome for each subject has a different reference, the correspondence across filtered connectomes is based only on anatomical alignment and preserves the anatomical variability in the movie data and HCP datasets.

Intersubject correlation (ISC) of connectivity profile vectors.

For validity testing we applied a more detailed definition of the connectome to measure fine-grained structure. The connectivity profile vector for a feature (or a cortical node or voxel) was defined as the correlation of its time-series with of all other cortical nodes or voxels. ISCs of connectivity profiles were computed between each subject’s connectivity profiles and the average connectivity profiles of all other subjects in each cortical locus.

For the movie data ISCs of connectivity profiles were computed within each movie half separately and before and after CHA based on the other half of the movie. Correlation values were Fisher transformed before averaging across both halves of the movie in each voxel. These were then averaged across all subjects and inverse Fisher transformed before mapping onto the cortical surface for visualization. ISCs of resting state connectivity profiles were computed for session REST2. Session REST1 was used for deriving the common model connectome and transformation matrices. ISCs were calculated for data mapped into the common model connectome, for movie data aligned anatomically, for HCP resting state data aligned with MSM-All, and for filter control movie and HCP data.

We also computed within-subject between-session (REST1 and REST2) correlation of resting state connectivity profiles. Within-subject between-sessions correlations were calculated on data that are transformed by CHA as used for our main analyses.

Spatial point spread function.

To investigate the spatial granularity of representation, we computed a spatial point spread function (PSF) of ISCs or WSCs of connectivity profiles. We computed the correlation of connectivity profiles in each cortical locus (surface node or voxel) with the average connectivity profiles of cortical loci at varying cortical distances in other subjects’ data. To account for the effect of filtering, we did this analyses with filter control data that were filtered with CHA but aligned based on anatomy and MSM-All and after CHA with each subject aligned to the same reference subject [16]. We computed similar PSFs for connectivity profiles within-subject between-sessions (REST1 and REST2). This was also performed after CHA to account for any filtering effects but to a single common space as used for our main analyses.

ISC of representational geometry.

ISCs of similarity structures were computed within each movie half separately using a searchlight of 3 voxel radius. The mean number of voxels in these searchlights was 102. In each searchlight, similarity structure was computed as a matrix of correlation coefficients between patterns of response for every pair of time-points from that movie half for each subject. The flattened upper triangle of this matrix excluding the diagonal was extracted as the profile of representational geometry at each searchlight for each subject. ISC of representational geometry in each searchlight was computed as the correlation between each subject’s representational geometry and the average of all other subjects’ representational geometries for that searchlight. Correlation values were Fisher transformed before averaging across both movie halves in each voxel. These were then averaged across all subject-average pairs and inverse Fisher transformed before mapping onto the cortical surface for visualization. The same steps were performed to compute inter-subject correlation of representational similarity before and after hyperalignment.

Between-subject multivariate pattern classification (bsMVPC).

bsMVPC of 15 s movie time segments (6 TRs) was computed within each movie half separately using searchlights of 3 voxel radius, as in the analysis of representational geometry. bsMVPC was performed using a one-nearest neighbor classifier based on correlation distance [12,16]. Each 15 s (6TR) sequence of brain data for an individual was compared to other subjects’ mean responses to that sequence and all other 15 s sequences in the same movie half using a sliding time window, resulting in over 1300 alternative time segments (chance classification accuracy < 0.1%). Classification accuracies in each searchlight were averaged across both halves in each subject before mapping the subject means onto searchlight center voxels on the cortical surface for visualization.

We implemented our methods and ran our analyses in PyMVPA [45](http://www.pymvpa.org) unless otherwise specified. All preprocessing and analyses were carried out on a 64-bit Debian 7.0 (wheezy) system with additional software from NeuroDebian [52](http://neuro.debian.net).