MRI data preparation.

The considered HCP dataset included complete T1-weighted and diffusion-weighted magnetic resonance imaging (dwMRI) as well as resting-state functional MRI (fMRI) data for every subject. The extraction of SC was performed with a publicly available pipeline (https://github.com/inm7/vbc_dwmri). We refer to our related works [8,44] for further details about the utilized software tools. The reconstructed, parcellation-based SC comprised two matrices: one featuring the number of streamlines between individual regions, i.e., the streamline count, henceforth referred to as SC, and another one providing the average length of streamlines between regions, termed path lengths (PL). We considered the functional Schaefer atlas [45] with = 100 cortical regions (Sch100) as well as the anatomical Harvard-Oxford atlas [46] featuring = 96 regions and a probability threshold of 0% (HO0Thr).

The calculation of eFC in turn was based on the resting-state fMRI data provided by the HCP repository [43]. Time series of the blood-oxygen-level-dependent (BOLD) fluctuations were extracted as mean signals averaged over all voxels of individual brain regions parcellated according to the considered brain atlases. A subsequent calculation of the Pearson correlation for each pair of linearly detrended and z-scored BOLD time series from the respective cortical areas then yielded the desired eFC matrix. In this setting, four fMRI sessions comprising 1200 volumes each (sampled with a repetition time of TR = 0.72 s) were available for every subject. Scans were performed twice using different phase-encoding directions on two days. A concatenated BOLD signal was generated by combining the z-scored BOLD time series from the mentioned sessions. We focused on the eFC matrix derived from this concatenation (and eventual cross-correlation of the regional signals) for the optimization of model parameters in terms of the best fit between simulated and empirical data, i.e., between sFC and eFC.

Model simulations.

To obtain sFC, we implemented the Kuramoto model of coupled phase oscillators [47–49]. In the model, each of oscillators represented one brain region defined by the underlying parcellation ( ∈ {96, 100} in this study). Their temporal dynamics served as a basis to compute simulated BOLD signals that were then cross-correlated in order to derive sFC. More precisely, the phase dynamics of the mean BOLD signal of a brain region ∈ {1, …, } can be described by the following differential equation:

(1)

For a given time , denotes the phase of the oscillator that oscillates with a natural frequency if left uncoupled ( = 0) from the other oscillators. This assumption considers that the ensemble of neurons contained within a cortical area behaves coherently, leading to a well-defined average phase, whose dynamics is described by the natural frequency [50,51]. For the low-dimensional cases, the frequency parameters , …, were estimated from the maximal spectral peaks (restricted to the frequency range from 0.01 Hz to 0.1 Hz) of the empirical BOLD time series. In the model simulations and validations with high-dimensional parameter spaces in turn, we treated them as free parameters restricted to the same range.

Apart from intrinsic oscillations, the phase dynamics is influenced by delayed interactions with the other oscillators. The individual coupling strengths and delay values were derived from the empirical matrices and , respectively:

(2)

In this context, indicates the mean over all matrix elements, excluding the diagonals. and are global weighting factors that influence the strength and delay of the interactions between individual oscillators, respectively. These quantities were treated as free parameters in all considered scenarios. Another variable for optimization can be the intensity of the independent noise which is sampled from a uniform distribution over [-1, 1] and perturbs the individual oscillators. While and are dimensionless quantities, is measured in seconds.

We solved Eq. (1) numerically using Heun’s method for stochastic differential equations [52] with a discrete time step of = 0.06 s. Here, we simulated phase dynamics for 3500 s after skipping 500 s as a transient. The calculated phases were down-sampled to 0.72 s in order to match the repetition time of the fMRI data provided by the HCP. With the model at hand, we generated simulated BOLD time series by computing for all ∈ {1, …, } corresponding to the regions in the respective brain parcellation. As mentioned above, sFC was then derived from the Pearson correlation coefficients across all pairs of simulated time series. We assessed the similarity between simulated and empirical data by correlating the matrices of sFC and eFC, where the former depended on the model parameters , , and .

Parameter optimization.

As mentioned above, the model was validated by optimization of the model parameters for the best correspondence between sFC and eFC. The case, where the two model parameters of global coupling and delay were optimized, corresponds to the model validation in a two-dimensional (2D) parameter space, see also [49,53]. Besides, a three-dimensional (3D) parameter space was considered, where also the noise intensity was treated as a free parameter, see also [3]. These two scenarios can be described as the low-dimensional cases. In our preceding work [42], we performed a systematic comparison of optimization methods applied to individualized whole-brain models in such low-dimensional parameter spaces (for fewer subjects than here and only one brain atlas). After investigating the stability and computational expenses of several approaches, we identified the best performing algorithms (BO, CMAES) for further usage. Here, we additionally worked with two high-dimensional parameter spaces for two brain atlases, in which we adapted the idea of region-specific model parameters that had already been motivated in [27], for example. We therefore applied our previously tested methods, circumvented the unsurmountable resource requirements of a potential grid search and optimized the frequency parameters for all brain regions. This extends previous attempts to introduce regional heterogeneity, in which the natural frequencies in the Kuramoto model were based on anatomical node strengths, i.e., row sums in the SC matrix [54–56]. Taking into account the three previously mentioned parameters of global coupling, delay and noise intensity, this ultimately led to 99-dimensional (99D) and 103-dimensional (103D) parameter spaces for the Harvard-Oxford atlas and the Schaefer atlas, respectively. We note here that the consideration of local model parameters for all brain regions appeared as the most logical step for introducing regional heterogeneity in the considered phase oscillator model.

Applying two mathematical optimization methods instead of one appeared reasonable since we could not always use a grid search as a baseline for result verification. Our confidence in both methods was based on previous successful applications and analyses [28,42,57–59]. In the current study, however, we did not intend to compare the methods as such very extensively. We treated them more as a mutual reassurance to minimize the risk of biased result interpretations due to algorithm peculiarities. Instead, we inspected the differences between models validated in low- and high-dimensional parameter spaces. We therefore investigated the approximated optimal parameters and obtained fitting quality together with the sFC and computational requirements in all tested dimensions.

We repeat at this point that we worked with several model parameter spaces. In the simplest case (2D), we fixed the noise intensity at = 0.3 and extracted the frequency parameters , …, from the empirical BOLD signals. The free parameters were then the global coupling strength ∈ [0, 1] and the global delay ∈ [0, 100]. In the 3D scenario, we increased the dimension of the parameter space by adding ∈ [0, 2] as a free variable. The motivation of the considered parameter ranges and the efficacy of the considered optimization algorithms on such problems are detailed in our preceding work [42]. For the high-dimensional simulations, which constitute the essence of this study, we further included all remaining free parameters of the model, namely the natural frequencies , …, ∈ [0.01, 0.1] as parameters for simultaneous optimization. This resulted in the 99D and 103D parameter spaces for HO0Thr and Sch100, respectively. The range for was increased to [0, 2] here in order to enable a more global search in the high-dimensional cases.

All model fitting simulations for this study were performed on the CPU partition of the high-performance supercomputing cluster JURECA-DC at Forschungszentrum Jülich. Every computation node featured two AMD EPYC Rome 7742 processors with 64 cores each, and each core supported two hardware threads, so that one computation node ultimately offered 2 × 2 × 64 = 256 threads for simultaneous computations, see [60] for further details about the configuration of JURECA-DC.

Optimization algorithms.

The BO method is a sequential design strategy suited for optimization problems with an unknown goal function that is expensive or time-consuming in its evaluation. After computing the goal function values for a random sample of ∈ candidate solutions in the parameter space, the obtained information is used to construct a probabilistic surrogate model for [61]. This approximation of can be updated based on the knowledge obtained from further function evaluations. Ideally, the surrogate model will eventually deliver a close approximation of the goal function, thereby enabling its efficient optimization. Detailed descriptions of all relevant steps can be found in algorithm-related papers [42,62,63]. We used the C++ software package BayesOpt [64] for our simulations. After analyzing the gradual development of the correlation between sFC and eFC for a saturation (depending on the number of performed iterations), we applied a stopping criterion in the form of a maximal number of iterations. We worked with 80 and 230 iteration steps in the low- and high-dimensional cases, respectively, see also Supplementary Materials. For every subject and parameter space, we performed = 30 executions of BO with random initial data. Some results of the convergence tests are shown in Supplementary Fig 1 in S1 Appendix .

The CMAES constitutes a global, population-based search procedure. In every iteration, a sample of candidate solutions is drawn from a multivariate normal distribution. We refer to it as the search distribution. CMAES works with a weighted mean of solutions which yield the lowest values of the goal function (in a minimization problem). Afterwards, a new generation of candidates is obtained by taking new samples from the search distribution centered around the weighted mean of these most promising points. That is how the search distribution is adapted iteratively towards a concentration around the optimal solutions, see [42,65,66] for more details. In accordance with the setting for BO, we also executed CMAES = 30 times for every subject and parameter space, see also Supplementary Materials. The initial search distribution mean was selected uniformly randomly within the considered parameter ranges. The correlation of sFC and eFC showed a saturation after 80 and 150 iteration steps in the low- and high-dimensional parameter spaces, respectively. Some examples of the performed convergence tests are shown in Supplementary Fig 1 in S1 Appendix.

Intra-subject variability of approximated optimal frequency parameters.

We stress that we had added the frequencies , …, to the list of parameters for optimization in the high-dimensional cases (99D, 103D). Due to such an increase in the dimension of the parameter space and the related complexity of the corresponding model validation procedure, we had to elaborate more on the properties of the optimized model parameter values. Besides the optimized triple (, , ), we obtained optimized frequency values after every algorithm execution with random initial conditions for a given subject. This means that in addition to the three global parameters of delay, coupling and noise intensity, the simulations yielded specific local parameters related to every brain region. An analysis of the distributions and structure of the latter parameters was performed to inspect the variability within subjects and check for potential subject-specific characteristics in the optimized values. In search of such common patterns across any two executions

∈ {1, …, }, , we correlated all pairs of vectorized frequency sets and . For every subject ∈ {1, …, }, we then saved the obtained correlation values to a matrix ∈ and applied the -means clustering method [68] to the rows of . Additionally, we computed the sum of all squared distances between the cluster elements and their cluster centroids. The final number of clusters was then selected by the “elbow method” as the one for which the sum of distances showed the strongest decrease as compared to the previous case 1, and where the larger number of clusters 1 did not reduce it notably. As a quantification of the cluster size, we considered the number of elements in the largest cluster. The number and size of clusters were however set to zero if the absolute mean of the off-diagonal elements in the correlation matrix was below 0.25.

As an extension to the clustering procedure, we also investigated whether a principal component analysis (PCA) of the obtained model parameters can provide additional insights about the structure of the solutions.

Reliability of modeling outcomes.

We analyzed the test-retest reliability of the simulation results with the intraclass correlation coefficient (ICC) [67,69–71]. In this approach, we considered the optimized model parameters and the obtained GoF values as well as the elements/edges of the sFC matrices. The ICC quantifies the between-subject variance of the mentioned values relative to the total variance, i.e., between and within subjects. As stated above, we treated repeated executions of the optimization algorithms as proxies for different scanning sessions. Higher ICC scores imply more reliable results.

There exist many different definitions of the ICC in the literature, all of them being suitable for particular problems [67]. We worked with the ICC that is also known as ICC(1) or ICC(1,1) and can be described as the original ICC without bias or systematic error [67]. It is defined as follows:

(3)

where denotes the variance of the considered quantity that is related to the variance among subjects and represents the remaining residual variance that may be induced by different measurements of a given subject, for instance. It has been proposed to use this definition in cases where the sources of fluctuations within subjects are unspecified and no structured variability can be assumed [70,71]. This was adequate for our setting since we only changed the random initial data of every algorithm execution for a given subject. To compute the ICC, we used the approximative sample ICC recommended in [67]. It allowed us to inspect the ratio of the fluctuations of the simulation outcomes between subjects relative to the fluctuations between algorithm executions across and within subjects. We adapted the quality gradation suggested in [72,73], see also Supplementary Materials.

As an extension of the analyses pertaining to the fluctuations within and between subjects, we also computed the subject specificity [74–76] derived from the differences between intra- and inter-subject (i) FC correlations and (ii) FC matrix norm distances. The details are provided in the Supplementary Materials. GoF-based specificity measures were computed as well, see also Supplementary Materials.

Statistical tests and effect sizes.

In this work, we utilized two non-parametric tests to assess statistical significance. When dealing with two related samples (e.g., the GoF for the same subjects, atlas and algorithm, but for a different dimension of the parameter space), we applied the Wilcoxon signed-rank test. In case of two independent samples (e.g., the GoF for males and females in the same parameter space), we used the Wilcoxon rank-sum test. For the latter method, we further assessed the ES of statistical differences with Rosenthal’s formula [80]:

(4)

where denotes the z-score of the test statistic and represents the total sample size (number of males + number of females, for example). We used this formula in accordance with related works [22,81] and adapted the quality gradation from very low to very strong according to [82], see also Supplementary Materials.

GoF, computation time and optimized model parameters.

The quality of the model validation (GoF values) is influenced by the number of free model parameters (dimension of the model parameter space) being simultaneously optimized for the best correspondence between simulated and empirical data. We compared the detected GoF values across all subjects and observed that for both tested atlases, more free parameters led to a higher fitting quality [Fig 1, Supplementary Table 2 in S1 Appendix]. Indeed, increasing the dimension from 2D to 3D resulted in a moderate median improvement of 11% – 15% [Fig 1] and a low ES of 0.3–0.4 for the GoF differences [Supplementary Table 2 in S1 Appendix]. However, the transition to the high-dimensional cases caused a remarkable jump, where the median GoF could double [Fig 1A, CMAES for the Schaefer atlas (Sch100)]. It demonstrated a strong effect of 0.7–0.8 for the GoF differences between the 2D and high-dimensional cases for both considered atlases [Supplementary Table 2 in S1 Appendix].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 1. Distributions of the goodness-of-fit (GoF) values for all tested approaches and parameter spaces.

The considered optimization methods and the respective dimensions of the parameter spaces are indicated on the horizontal axes (BO 2D, CMAES 2D, BO 3D and CMAES 3D in the low-dimensional cases and BO 103D, CMAES 103D, BO 99D and CMAES 99D in the high-dimensional ones) along with the detected GoF values on the vertical axes. Violins show the distributions of the median GoF values obtained for all subjects across 30 algorithm executions with random initial data (option 2, see Methods). The medians (across subjects) of the relative increase between the results obtained in different parameter spaces for a given algorithm are indicated in the plots together with -values of the Wilcoxon signed-rank test and the considered atlas: (A) the Schaefer atlas (Sch100) and (B) the Harvard-Oxford atlas (HO0Thr). Statistically significant differences are marked with an asterisk. The significance level of 5%, < 0.05, has been Bonferroni-corrected for multiple comparisons.

https://doi.org/10.1371/journal.pone.0322983.g001

Similarly, a respective increase of required computational resources can be observed when comparing the median consumption of low- and high-dimensional optimizations [Supplementary Fig 3 in S1 Appendix]. We however note here that more iteration steps were performed in the high-dimensional cases (see Methods and Supplementary Materials), meaning that also additional time-consuming evaluations of the goal function were made.

When inspecting the model parameters optimized in the high-dimensional space, we found large fluctuations across different algorithm executions even for the same subject [Fig 2, Supplementary Fig 4 in S1 Appendix]. Especially the optimized frequency parameters seem to cover the entire plausible range of values in [0.01, 0.1] Hz for the vast majority of brain regions [Fig 2E,F, Supplementary Fig 4 in S1 Appendix]. Compared with this variance, the spread of the GoF values obtained in 30 algorithm executions appears to be low [Fig 2D, Supplementary Fig 4 in S1 Appendix]. Indeed, for the considered example of the 103D parameter space with CMAES and the Schaefer atlas [Fig 2, Supplementary Fig 4 in S1 Appendix], we found that the GoF values are narrowly distributed around the median 0.59 with interquartile range (IQR) 0.03, i.e., about 5% of the relative variability as given by the fraction IQR/median [Fig 2D]. On the other side, such a variability reaches on average 90% for the optimized frequencies [Fig 2E,2F], as well as 43% and 41% for the optimized parameters of coupling and noise intensity , respectively [Fig 2B,2C]. Probably the largest variability is manifested by the optimized parameter of delay with IQR around 57.60 at zero median [Fig 2A]. The median value of zero however indicates that more than half of the performed algorithm executions for this subject yielded an optimal delay of = 0.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 2. Example of the results of the model fitting in the 103-dimensional parameter space.

Boxplots illustrate the distributions of the values for the optimal (A) delay , (B) coupling , (C) noise intensity , (D) goodness-of-fit (GoF) and (E,F) frequency parameters found in 30 executions (with random initial data) of the CMAES algorithm for one subject. The considered modeling quantities are indicated in the titles of each plot, while their considered ranges are given on the vertical axes. Plot (E) shows the first half of the frequency parameters, belonging to the brain regions in the left hemisphere in the Schaefer atlas, and (F) shows the second half, which represents the right hemisphere. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g002

We verified the above findings for the entire subject cohort. Therefore, we evaluated the variability (IQR) of the approximated optimal model parameters and GoF values across algorithm executions for all subjects, see Supplementary Materials. We observed a persistently high variability of the model parameters for both atlases and optimization algorithms [Supplementary Fig 5 in S1 Appendix]. Compared with the regional model parameters, i.e., frequencies , the global parameters of delay , coupling and noise intensity may show a slightly lower variability, but still a much higher one than that of the GoF values, see Supplementary Materials.

Correlations of optimized frequency distributions.

We found that, despite the fluctuations, the values of the optimized frequency parameters are by no means entirely random across different algorithm executions. To illustrate this, we calculated the correlations between the values of the model parameters obtained for different optimization runs (with different initial conditions) for the same subject. More precisely, the -dimensional vector of optimal frequencies obtained for one optimization run may show a pattern that tends to be strongly (anti)correlated with the optimal frequencies calculated for another optimization run for the same subject [Supplementary Fig 4 in S1 Appendix, Fig 3A]. An example that visualizes such an (anti)correlation based on the optimized frequency parameter values is provided in Supplementary Fig 4 in S1 Appendix. We observed (anti)correlated clusters of frequency patterns obtained for different optimization runs for individual subjects. All ( = 30) runs can be split into two groups (clusters), where the vectors of the optimized frequencies strongly correlate with one another within the same cluster, but negatively correlate with those from the other cluster [Fig 3A]. The clusters may however vary in size for different subjects [Fig 3B-3I]. This means that one frequency pattern may be dominant for a given subject, and there is just one cluster of optimized frequency parameters, where all obtained frequency realizations strongly correlate with one another [Supplementary Fig 6 in S1 Appendix]. For another subject, two anticorrelated frequency patterns (and respective clusters) can occur with almost equal or different probabilities [Fig 3A]. It may also happen that there are no clusters, and different optimization trials lead to weakly correlated frequency constellations [Supplementary Fig 6 in S1 Appendix]. A PCA of the frequency patterns revealed that strong anticorrelations might be reflected by clear differences (separation) regarding the scores of the first principal component in the PC space [Supplementary Fig 7 in S1 Appendix]. Weak correlations or the presence of just one dominating pattern in turn may align with increased variation in the second principal component but seem harder to be described by the PC scores [Supplementary Fig 7 in S1 Appendix]. The fraction of explained variation by a given number of principal components differs considerably across subjects, while 13–16 components are on average needed to explain 80% of variation [Supplementary Fig 7 in S1 Appendix]. This indicates that the space dimension of the optimized frequency parameters may be around 15 or higher. In general, the tendency to obtain two anticorrelated frequency patterns is more pronounced for the Schaefer atlas than for the Harvard-Oxford atlas, as more subjects exhibit two-cluster frequency realizations for the former parcellation than for the latter [Fig 3B and 3I]. Coming back to the analysis of the (GoF and parameter) variability across repeated algorithm executions for same subjects, we want to highlight that the IQR of GoF values was relatively low [Supplementary Fig 5 in S1 Appendix] even when almost no clusters of frequency patterns could be observed across subjects [Fig 3E].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 3. Structure of optimized model parameters in the high-dimensional cases.

(A) Example of the correlations between optimized frequency parameter sets obtained at the model validation in the 103-dimensional parameter space via CMAES for the Schaefer atlas (Sch100). For a given subject, the correlations between any two sets of optimized frequency parameters obtained in different algorithm executions (Ex.) were calculated and then indicated in the table cells also highlighted in color. The algorithm executions were sorted, so that the cluster structure becomes observable. (B-I) Number of clusters and cluster size for all subjects and optimization approaches. (B-E) Subject-dependent number of the frequency clusters (# Clusters) for the atlases (Sch100 and HO0Thr) and methods (BO and CMAES) indicated in the titles. In each plot, the subjects were sorted according to the number of elements in the largest cluster as illustrated in the bottom plots. (F-I) Cluster size for all subjects, i.e., the number of algorithm executions in the largest frequency cluster. The number and size of the clusters were determined with help of the -means clustering method. Both were set to zero if the absolute mean value of the off-diagonal elements in the correlation matrix (cf. (A)) was below 0.25 (corresponding to weak correlations between different frequency sets). The subjects considered in (A) and Supplementary Fig 6 in S1 Appendix are highlighted in the plots (F) and (G). This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g003

Nevertheless, the presence of anticorrelated frequency patterns, which reflect the discussed variability of the optimized model parameters, together with relatively small GoF changes can still be observed in cases where the high-dimensional optimization was informed by the results of the low-dimensional one [Supplementary Fig 8 in S1 Appendix]. We confirmed the reported conclusions by taking into account the optimal parameter values approximated during the low-dimensional model fitting as starting points for the high-dimensional optimization problems (instead of random initial conditions that were considered above) [Supplementary Fig 8 in S1 Appendix]. We also note here that the model includes a random noise term that can be an additional source of the observed variability at the optimization in the high-dimensional spaces.

Based on these findings, we suspect that the optimal solutions for a model validation in high-dimensional parameter spaces are located on a certain subspace of the domain of the goal function. We may refer to it as a characteristic manifold of model fitting, whose properties (such as a degeneracy) require a deeper investigation that builds on the methodological scope presented here.

Reliability of model parameters.

At first, we considered the optimized parameters of global delay , coupling and noise intensity , and found that their reliability is influenced by the employed brain atlas and optimization method [Fig 4]. The reliability of parameter decreases for an increasing number of free model parameters. While it varies between fair (BO) and good (CMAES) for both atlases in 2D, it drops to a poor reliability for all other conditions that we tested [Fig 4A,4D]. For the coupling parameter , we also observed the highest values in 2D (BO: fair, CMAES: excellent), and subsequent drops for the other dimensions of the parameter space [Fig 4B,4E]. The only exception is given for BO and the Schaefer atlas, where the ICC reliability of is smallest in the 3D case [Fig 4B]. The reliability of the noise intensity remains nearly unchanged for the transition from 3D to the high-dimensional cases for BO, staying within the fair (Schaefer atlas) or within the good range (Harvard-Oxford atlas) [Fig 4C,4F], which is in contrast with that of the delay and coupling parameters (but see Fig 4B). However, for CMAES, the reliability of the parameter clearly drops from the excellent to the fair range during the same transition to the high-dimensional spaces [Fig 4C,4F].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 4. Reliability of the optimized model parameters of delay , coupling and noise intensity .

Colored bars illustrate the reliability of the modeling results as measured by the intraclass correlation coefficient (ICC). The considered optimization algorithms and parameter spaces are indicated on the horizontal axes along with the ICC scores on the vertical axes. Results are shown for (A-C) the Schaefer atlas (Sch100) and (D-F) the Harvard-Oxford atlas (HO0Thr). Dashed horizontal lines indicate the levels of poor, fair, good and excellent reliability in terms of the ICC as suggested in [72,73]. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g004

In the latter cases, we also evaluated the reliability of the frequency parameters . We found that their ICC reliability hardly ever exceeds the threshold of 0.4 and can therefore be considered poor for all methods and parcellations [Fig 5A,5B,5D,5E, gray histograms].

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 5. Reliability of the optimized frequency parameters and sFC matrix edges in the high-dimensional cases as well as of the GoF.

(A-B,D-E) Colored, horizontal histograms visualize the distributions of the intraclass correlation coefficients (ICC) for the model parameters (gray) as well as for the above-diagonal edges of the matrices of sFC (orange and blue for BO and CMAES, respectively). The considered atlases, i.e., (A-B) the Schaefer atlas (Sch100) and (D-E) the Harvard-Oxford atlas (HO0Thr), are provided in the titles together with the utilized optimization algorithms and parameter spaces. (C,F) Colored bars illustrate the ICC scores of the GoF for all considered optimization algorithms and parameter spaces, which are indicated on the horizontal axes. The names of the atlases are again provided in the titles. Dashed horizontal lines indicate the levels of poor, fair, good and excellent ICC reliability as suggested in [72,73]. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g005

In summary, the reliability of the three model parameters of delay , coupling and noise intensity shows a rather complex behavior as measured by the ICC. We can, however, conclude that the reliability of the parameters , and becomes lower when the number of free model parameters increases for CMAES. For BO, this is also the case, except for (Schaefer atlas) and (both atlases). In the high-dimensional cases, the optimal delay, coupling and frequency parameters mainly demonstrate a poor reliability. Nevertheless, the optimal noise intensity can still be obtained with fair or even good (BO, Harvard-Oxford atlas) reliability, which indicates that its variability within subjects can be lower than between subjects for multiple executions of the optimization methods.

Reliability of sFC and GoF.

In this step of our analyses, we investigated the ICC reliability of the optimal sFC matrices generated in each algorithm execution. Here, the reliability of every individual matrix element above the diagonal was calculated across subjects (see Methods), and the distributions of the obtained values were inspected. We started with the high-dimensional cases, where it turned out that the sFC matrices feature reliable edges ranging mainly between a fair and good quality [Fig 5A,5B,5D,5E, orange and blue histograms]. These matrices can nevertheless be outperformed by their equivalents from the 2D and 3D cases which, for their part, may even reach an excellent reliability, except for BO with the Schaefer atlas [Supplementary Fig 9 in S1 Appendix].

We further evaluated the ICC reliability of the GoF values obtained for repeated runs of the optimization procedures. Here, we observed a decreasing trend for both algorithms and the Schaefer atlas, with the highest reliability in the case of the two-dimensional parameter space (excellent), followed by 3D (excellent) and 103D (good) [Fig 5C]. For the Harvard-Oxford parcellation, the GoF values reached an excellent reliability in all considered cases, including the high-dimensional ones [Fig 5F].

In summary, we observed an sFC reliability of good quality together with high ICC values for the GoF (good and excellent reliability) even for the high-dimensional model fitting. These findings are in alignment with the low GoF variability for single subjects described above but needed to be verified given the remarkable fact that the optimized model parameters, which generate such sFCs and GoF values, tend towards a poor reliability [Figs 4,5].

To deeper investigate the two modeling results (sFC, GoF) that turned out as the most reliable ones across optimization runs, we used them to compute the subject specificity, see Supplementary Materials. The specificity characterizes the similarity/discrepancy between intra- and inter-subject properties like average FC correlations within subjects vs. average FC correlations between subjects, for instance. We tested ES- and difference-based approaches to compute the specificity and analyzed the impact of the dimension of the parameter space.

We found that the specificity indices frequently remained within narrow ranges of values, and we did not find a clear trend that would indicate an enhancement or reduction of the specificity depending on the dimension of the model parameter space considered for the model validation [Supplementary Figs 10,11,12,13 in S1 Appendix].

At this point, we emphasize that the model validation in high-dimensional parameter spaces much better approximates the measured brain activity as the simulated data (sFC) closely replicate the empirical ones (eFC). A comparably low correspondence between empirical and simulated data in the low-dimensional parameter spaces can be balanced by an enhanced reliability of the model parameters and connectivity patterns. While the reliability of sFC edges and GoF values can reach the fair, good and excellent ranges also for the model validation in the high-dimensional parameter spaces, there remains the high variability of the approximated optimal model parameters. This may play a role in further applications, where it may often be desirable to extract one final outcome per subject. However, as we will see in the next part, sampling from parameter distributions or selecting one solution based on given characteristics may open up new possibilities for further successful usage of the simulated data. Our results for the high-dimensional cases do bear a potential for a better detection of inter-individual differences despite the observed variability of model parameters.

Sex differences in GoF distributions.

We observed significantly higher GoF values for males during the model fitting procedure, with a regular exception given by the 3D cases with the Schaefer atlas [Fig 6A,6B]. Among the optimization algorithms, we obtained mostly consistent results, where the sex-related GoF differences are significantly stronger for the high-dimensional model fitting than for the low-dimensional cases (except for BO with the Schaefer atlas) [Fig 6C,6D]. To be more precise, the median ES of the differences between males and females for the Schaefer atlas ranges around 0.21 in the 2D case and climbs to roughly 0.3 for CMAES (ES = 0.86 between 2D and 103D) in the high-dimensional model fitting [Fig 6D], but not for BO, where it remains at the approximate level it had in 2D [Fig 6C]. The situation is different for the Harvard-Oxford atlas, where the ES of the sex differences are in general higher (around 0.33 in both 2D and 3D [Fig 6C,6D]) than their counterparts for the Schaefer atlas. An obvious increase of the GoF differences between males and females was observed in the high-dimensional case [Fig 6C,6D]. It approaches a moderate ES around 0.4 for both algorithms (ES = 0.87 between 2D and 99D). So, except for BO with the Schaefer atlas, the high-dimensional model validations seem to strongly enhance the observed differences between males and females in GoF values (with very strong ES of the differences between the low- and high-dimensional cases [Fig 6C,6D]).

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 6. GoF values for males and females together with the corresponding effect sizes (ES) of the observed differences. (A-B) Colored boxplots show the distributions of the median model fits obtained for males (brown) and females (pink) in 30 algorithm executions with random initial data (option 2, see Methods). The considered dimensions of the parameter spaces combined with the group abbreviations of males (M) or females (F) are indicated on the horizontal axes along with the detected GoF values on the vertical axes. Solid vertical lines separate the results for the Schaefer atlas (Sch100) and the Harvard-Oxford atlas (HO0Thr). The names of the utilized optimization algorithms are given in the titles. Statistically significant sex differences as assessed by the Wilcoxon rank-sum test are marked with an asterisk. The significance level of 5%, < 0.05, has been Bonferroni-corrected for multiple comparisons. The corrected -values of the sex differences are indicated in the plots. (C-D) Boxplots illustrate the distributions of the ES for the sex differences observed during a random selection of one of the 30 available algorithm executions for every subject and a subsequent comparison of the corresponding GoF values across both groups (1000 repetitions). The ES of the changes in ES from the low- (2D) to the high-dimensional (99D, 103D) cases are indicated in the plots. The Wilcoxon rank-sum test was applied. Statistically significant differences are marked with an asterisk. The significance level of 5%, < 0.05, has been Bonferroni-corrected for multiple comparisons. (E-F) Same as (C-D), but the ES of the sex differences were computed for the GoF residuals obtained by regressing out the intracranial volume of the individual subjects from the GoF values. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g006

Subsequently, we investigated whether the sex differences in GoF could be related to the differences in intracranial brain volume between males and females. We therefore applied linear regression to regress out the intracranial volume from the GoF values and repeated the analysis from above for the respective GoF residuals (see Methods). Here, we indeed observed a reduction of the differences between males and females, which is indicative of an impact of the intracranial brain volume on sex differences in GoF values [compare Fig 6C,6D with Fig 6E,6F]. Nevertheless, we also confirmed that the sex differences in the modeling results are more pronounced when the model validation is performed in high-dimensional parameter spaces [Fig 6E,6F]. Even for BO with the Schaefer atlas, which previously did not confirm the trend, we observed a very strong increase of the differences between males and females when comparing the low- and high-dimensional cases (ES = 0.85 between 2D and 103D) [Fig 6E].

Sex differences in empirical data, model parameters and FC strength.

To assess the actual benefit of the data generated by whole-brain model simulations, we also searched for similar patterns in empirical data. We therefore considered the structure-function relationship as given by the correlation between SC and eFC. It appeared as the most straightforward empirical alternative to the GoF of the modeling results. Here, we found larger correlation values for the females than for the males for both atlases [Supplementary Fig 14 in S1 Appendix], in contrast to the GoF values, see also [81]. However, the absolute ES of the sex differences did not exceed 0.2 (-0.13 for the Schaefer atlas and -0.19 for the Harvard-Oxford atlas), which was regularly surpassed in the modeling outcomes [Fig 6C,6D]. This result provides further evidence of the practical utility of whole-brain model simulations, which can, as in the considered case, reveal differences between males and females that are hardly observable in empirical brain imaging data.

We note that our data do not yet show equipollent differences between males and females when the analysis was based on the optimized values of the model parameters instead of the GoF values. The ES remained very low in most cases. Significant and relatively strong increases in the (still low) ES could only be observed during the transition from low- to high-dimensional parameter spaces for the coupling parameter for both atlases and optimization methods and slight increases for the noise intensity for the Schaefer atlas and both methods [Supplementary Fig 15 in S1 Appendix]. For the other cases and model parameters, the sex differences can even decrease for the high-dimensional model fitting [Supplementary Figs 15,16 in S1 Appendix]. Especially the frequency parameters did not seem to contain information suitable for differentiating between subjects [Supplementary Fig 16 in S1 Appendix]. By contrast, the sex differences in the GoF values can frequently and strongly increase for the high-dimensional model fitting, where the ES may reach up to 0.4 in the median [Fig 6].

Nevertheless, as the approximated optimal parameters constitute one major source of new, model-based information regarding the brain dynamics of a given subject, we investigated the practical utility of the model parameters further. We attempted to suggest a justified strategy of selecting one of the 30 fluctuating solutions per subject and find relations to other properties of the neuroimaging and behavioral data. In this endeavor, we could repeatedly detect slightly higher coupling values for males in the high-dimensional cases [Fig 7A,7C] when selecting one of the available 30 solutions per subject based on the following three options: Choose the algorithm solution from the execution yielding (1) the highest GoF, (2) the median GoF or (3) the minimal , see also [Supplementary Figs 17,18 in S1 Appendix]. We controlled our results for GoF outliers.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 7. Distributions of optimized coupling parameters and median FC strengths for males and females.

Colored boxplots together with gray circles in the background show the distributions of (A,C) the values of approximated optimal coupling parameters and (B,D) the median connectivity strength in the FC matrices for males (brown) and females (pink). Empirical (eFC, Emp) as well as simulated (sFC) data were considered. For every subject, the eFC from the concatenated sessions as well as the sFC matrix and coupling parameters yielding the highest GoF across algorithm executions were selected (option 1, see Methods). We refrained from including the 3D cases in the figures as we found no remarkable differences to the 2D cases. The names of the utilized optimization algorithms and considered dimensions of the parameter spaces combined with the group abbreviations of males (M) or females (F) are indicated on the horizontal axes along with the median FC and optimal C values on the vertical axes. The names of the considered atlases are given in the titles, i.e., (A,B) the Schaefer atlas (Sch100) and (C,D) the Harvard-Oxford atlas (HO0Thr). Statistically significant sex differences as assessed by the Wilcoxon rank-sum test are marked with an asterisk. The significance level of 5%, < 0.05, has been Bonferroni-corrected for multiple comparisons. The effect sizes (ES) and corrected -values of the sex differences are indicated in the plots. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g007

Trying to relate the pattern of higher coupling parameter values for males to the practical role of the coupling parameter in the considered whole-brain model, we inspected whether there are also sex differences in the FC strength. We therefore compared the respective median FC values when we selected one FC matrix per subject from the three mentioned options. Here, we indeed found that the connectivity seems to be significantly stronger for males than females despite a wide overlap [Fig 7BD,]. For the eFC, we observed a significant ES of 0.34 and 0.33 for the Schaefer and Harvard-Oxford atlas, respectively. While the ES of sFC in the low-dimensional cases are clearly lower (0.1 to 0.24 for option 1) across all tested scenarios, they approach or even exceed (ES = 0.36 for CMAES with HO0Thr, option 1) the empirical values in the high-dimensional cases. Very similar trends can be identified for option 2 [Supplementary Fig 17 in S1 Appendix] and 3 [Supplementary Fig 18 in S1 Appendix]. So, higher values for the optimized model parameter of coupling in the high-dimensional cases seem to align with higher values of the median connectivity strength in the FC matrices, which may explain the observed stronger FC in males also in empirical data.

Model-based sex classification.

Finally, we inspected the applicability of the modeling outcomes to a sex classification task as an example of the investigation of inter-individual variability. We trained the machine learning classifier, a logistic regressor, separately on the GoF values as well as on the optimized model parameter values and inspected the performance changes between low- (2D) and high-dimensional (99D, 103D) parameter spaces. (The 3D cases did not show outstanding differences to the 2D cases.) It turned out that the GoF and optimized coupling parameter values are useful features for classification in the high-dimensional cases, while the remaining parameters are not. We found balanced prediction accuracies centered mostly around the chance level of 50% for the model parameters of delay , noise intensity and frequencies (not shown), which resonated with a poor practical utility of these quantities. For the features of GoF and coupling , we however observed median accuracies reaching up to 62% and 57% in the high-dimensional cases, respectively, when the GoF or coupling parameter value obtained from the algorithm execution yielding the highest GoF (option 1) were selected [Fig 8]. The ES between low- and high-dimensional scenarios can reach 0.53 and 0.78, respectively [Fig 8]. We also tested the classification performances for the options 2 and 3 of solution selections [Supplementary Figs 19,20 in S1 Appendix] and found that option 1 might exhibit the most consistent and pronounced classification improvements for the high-dimensional cases, especially for the coupling parameter . Option 1 therefore appears to be most suited for the applications presented here.

Download:

• PPT
  PowerPoint slide
• PNG
  larger image
• TIFF
  original image

Fig 8. Model-based sex classification accuracies.

(A-D) Boxplots together with gray circles in the background visualize the balanced accuracies of a logistic regressor trained for sex classification based on (A-B) the GoF or (C-D) the optimized values of the coupling parameter . For every subject, the GoF and parameter values derived from the algorithm execution yielding the highest GoF were selected (option 1, see Methods). We refrained from including the 3D cases in the figures as we found no remarkable differences to the 2D cases. The names of the utilized parameter optimization algorithms and parameter spaces are provided on the horizontal axes along with the balanced accuracy values on the vertical axes. Dashed horizontal lines indicate the chance level of 50%. Indications of which property (GoF or coupling parameter) was used for classification (classific.) are provided in the titles. The names of the considered atlases (Sch100 for the Schaefer atlas and HO0Thr for the Harvard-Oxford atlas) are given in the plots. Statistically significant performance gains between low- and high-dimensional parameter spaces as assessed by the Wilcoxon rank-sum test are marked with an asterisk. The significance level of 5%, < 0.05, has been Bonferroni-corrected for multiple comparisons. The effect sizes (ES) and corrected -values of the increases are indicated in the plots. This figure was created with MATLAB R2021a (www.mathworks.com).

https://doi.org/10.1371/journal.pone.0322983.g008

We conclude that the validation of the whole-brain models in high-dimensional parameter spaces is beneficial for studying differences between males and females in simulated resting-state data. For selected modeling quantities, the extent of observable sex differences exceeded those that are present in purely empirical data. We found in particular that the gap in the respective GoF value distributions between male and female subject groups can be expanded with a very strong ES of about 0.87 when the model validation is performed in high-dimensional parameter spaces. This conclusion still holds after regressing out the intracranial brain volume of the individual subjects. Additionally, we found significantly higher classification accuracies for a sex classifier trained on the modeling output from the high-dimensional cases, where one out of 30 available algorithm solutions was selected as the final output for a given subject. Our results indicate beneficial effects of the model validation in high-dimensional parameter spaces and a general importance of involving modeling results in the analyses and interpretations of brain and behavior.