Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

Novel subgroups of attention-deficit/hyperactivity disorder identified by topological data analysis and their functional network modular organizations

  • Sunghyon Kyeong,

    Roles Conceptualization, Formal analysis, Investigation, Methodology, Software, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation Severance Biomedical Science Institute, Yonsei University College of Medicine, Seoul, Republic of Korea

  • Jae-Jin Kim,

    Roles Writing – review & editing

    Affiliations Severance Biomedical Science Institute, Yonsei University College of Medicine, Seoul, Republic of Korea, Department of Psychiatry and Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea

  • Eunjoo Kim

    Roles Conceptualization, Funding acquisition, Supervision, Writing – original draft, Writing – review & editing

    ejkim96@yuhs.ac

    Affiliation Department of Psychiatry and Institute of Behavioral Science in Medicine, Yonsei University College of Medicine, Seoul, Republic of Korea

Novel subgroups of attention-deficit/hyperactivity disorder identified by topological data analysis and their functional network modular organizations

  • Sunghyon Kyeong, 
  • Jae-Jin Kim, 
  • Eunjoo Kim
PLOS
x

Abstract

Attention-deficit/hyperactivity disorder (ADHD) is a clinically heterogeneous condition and identification of clinically meaningful subgroups would open up a new window for personalized medicine. Thus, we aimed to identify new clinical phenotypes in children and adolescents with ADHD and to investigate whether neuroimaging findings validate the identified phenotypes. Neuroimaging and clinical data from 67 children with ADHD and 62 typically developing controls (TDCs) from the ADHD-200 database were selected. Clinical measures of ADHD symptoms and intelligence quotient (IQ) were used as input features into a topological data analysis (TDA) to identify ADHD subgroups within our sample. As external validators, graph theoretical measures obtained from the functional connectome were compared to address the biological meaningfulness of the identified subtypes. The TDA identified two unique subgroups of ADHD, labelled as mild symptom ADHD (mADHD) and severe symptom ADHD (sADHD). The output topology shape was repeatedly observed in the independent validation dataset. The graph theoretical analysis showed a decrease in the degree centrality and PageRank in the bilateral posterior cingulate cortex in the sADHD group compared with the TDC group. The mADHD group showed similar patterns of intra- and inter-module connectivity to the sADHD group. Relative to the TDC group, the inter-module connectivity between the default mode network and executive control network were significantly increased in the sADHD group but not in the mADHD group. Taken together, our results show that the data-driven TDA is potentially useful in identifying objective and biologically relevant disease phenotypes in children and adolescents with ADHD.

Introduction

Attention-deficit/hyperactivity disorder (ADHD) is the most common neurodevelopmental disorder of childhood, affecting 5% of school-age children worldwide [1]. ADHD is a clinically heterogeneous condition, with deficits in multiple neuropsychological processes and related brain systems [2]. Thus, different individuals with ADHD display diverse profiles in cognitive, emotional and motivational domains [3]. Therefore, there may be different disease phenotypes under the diagnosis of ADHD, and the identification of subgroups with distinct patterns of clinical manifestation may lead to the development of more effective and targeted intervention for personalized treatment and improved patient care [4].

Recent efforts in data sharing, such as the ADHD-200 Consortium have made it possible to develop a large dataset representative of the whole spectrum of the disorder. The ADHD-200 dataset provides information on age, sex, intellectual ability, symptom severity, and neuroimaging data from hundreds of individuals with ADHD and TDC [5]. To identify clinically useful new subgroups from large datasets, the choice of an appropriate clustering method and input features are important because varying results are expected depending on the machine learning algorithm and input features used [4]. Regarding the clustering method, the classical unsupervised machine learning method, e.g., k-means clustering, has been widely used for this purpose. However, a major criticism of this method is that identified subgroups are locally optimized, given that the initial clusters are heuristically assigned. Furthermore, the k-means clustering would fail to detect subgroups if the shape of the data becomes complex [6] and thus has limitations in the ability to extract meaningful subgroups from the complex and multidimensional neuropsychological data.

Because of the above-mentioned limitations of the traditional clustering algorithm, Topological Data Analysis (TDA) is receiving increasing attention for its ability to define the shape of the data and produce outcomes in the form of an easily recognizable graph, as graphically illustrated in Fig 1. TDA, or “partial clustering”, which has been widely used for obtaining new insights from the heterogeneous high dimensional behavioral, clinical and biological datasets [712]. As an unsupervised machine learning approach, TDA has the attractive features of being data driven, without first having to formulate a pre-defined hypothesis. TDA can more finely stratify patients than standard clustering methods and can identify new as well as interesting patient sub-groups [10].

thumbnail
Fig 1. K-means clustering analysis with two (k = 2) and three (k = 3) centroids (a) and topological data analysis as an example of shape extraction (b).

The classical k-means clustering fails to detect subgroups with complex shape of data.

https://doi.org/10.1371/journal.pone.0182603.g001

With regard to input features, several studies have attempted to identify ADHD subgroups by clustering them according to symptom severity [13, 14], reading disabilities [15], personality traits [16], anxiety/mood symptoms [17], and neuropsychological [18] and neuroimaging data [19] using data-driven machine learning algorithms. In our previous study, we have attempted to use a functional connectome-based topological data analysis (TDA) method to unveil hidden subgroups of ADHD, but failed to detect ADHD subgroups as a branch in the topology from neuroimaging data [19]. Instead, subgroup-like patterns have been observed when we mapped IQ and ADHD symptom severity on the resulting topology. In addition, previous studies demonstrated that general cognitive ability (intelligence quotient; IQ) and the severity of ADHD symptoms are two of the most important factors related to the prognosis of ADHD [20]. Specifically, individuals with high levels of ADHD symptoms are at an increased risk of continuing problems associated with ADHD during adolescence [21, 22]. It is also reported that IQ, but no other cognitive measures, affects ADHD outcome and that lower IQ is associated with comorbid conduct disorder, poor response to stimulant treatment, and poor prognosis [23, 24]. In fact, commonly measured basic neurocognitive and clinical characteristics, such as IQ and symptom severity, have their own advantages in that they can be more easily translated into clinical application [23] than that of neuroimaging data. Besides, these measures are best represented as dimensions and have well-theorized relationships to biological systems from previous studies [23, 25]. Therefore, IQ and symptom severity measures were selected to identify hidden subgroups using TDA in this study.

After identifying subgroups within large ADHD datasets, the investigation of the neurobiological validity of the newly defined subgroups is another important issue. In prior studies, neuroimaging methods, such as resting state functional magnetic resonance imaging (rs-fMRI) connectivity strength, were used to investigate the neurobiological substrates of human characteristics, such as personality types [4, 26]. A benefit of rs-fMRI is that it is more developmentally and contextually variable than brain structural measures, making it a highly attractive place to start working towards the goal of identifying subgroups of individuals with ADHD based on similar brain processes [27]. The presence of the subgroups that correspond to well-known brain functional systems identified by MRI implies that those subgroups may model some features of brain organization, and the absence of such subgroups may suggest that a subgroup may not be well-defined biologically [28].

On this background, the two main objectives of this study were (1) to determine whether we can identify new clinical phenotypes based on symptom severity and IQ measurements in children and adolescents with ADHD, and (2) to investigate whether neuroimaging findings validate the identified phenotypes. In this study, we used two brain-based measures to investigate the characteristics of the brains in each subgroup identified by TDA: 1) centrality measures, and 2) modularity analysis, exploring resting-state functional connectivity in ADHD in five pre-defined neural networks, searching for the neurobiological significance of the subgroups identified by TDA.

Materials and methods

Principal and validation datasets

We obtained the preprocessed rs-fMRI data from the Neuro Bureau ADHD-200 Preprocessed repository [29]. In the ADHD-200 dataset, the largest dataset from Peking University (PKU), containing both typically developing controls (TDCs) and children with ADHD, was chosen for this study. Among 176 male subjects, we selected 62 TDCs and 67 children with ADHD, choosing only those with complete fMRI, IQ, and clinical datasets without missing variables. The principal dataset contained the neuroimaging network data as well as the clinical measures, such as ADHD symptom severity and IQ scores. For the validation of the proposed topology-based methods, we chose the second largest dataset. From the 173 male subjects included in the New York University (NYU) dataset, we selected 43 TDCs and 91 children with ADHD, choosing only those with complete Q and symptom severity of ADHD datasets without missing variables. To control the potential confounding effects of sex and IQ, we excluded girls and subjects with IQs lower than 80. The ADHD Rating Scale IV [30] and the Conners’ Parent Rating Scale-Revised [31] were used to assess symptom severity in the principal and validation datasets, respectively. The Wechsler Intelligence Scale for Chinese Children-Revised [32] and Wechsler Abbreviated Scale of Intelligence [33] were used to assess intelligence ability in the principal and validation datasets, respectively. Participant demographic and clinical information are shown in Table 1 and S1 Table online. All experimental protocols were in compliance with the policies of site-specific institutional review boards (IRBs). The ADHD-200 datasets are anonymized, with no protected health information included. Therefore, this study met the requirements for exemption from our local IRB review.

thumbnail
Table 1. Demographic and clinical characteristics of the principal dataset.

https://doi.org/10.1371/journal.pone.0182603.t001

Topological data analysis (TDA)

We performed TDA to identify hidden subgroups of ADHD. We prepared a dataset as a matrix form using three clinical variables for symptom severity and three variables for intelligence. Therefore, the matrix (Mij) indicates a point of data for a subject i and a variable j, where characteristic variables contained three symptom variables, such as ADHD index, inattentive score and hyperactive/impulsivity score, and three intelligence scores, such as full-scale IQ, verbal IQ and performance IQ. Fig 2 shows the preprocessing and analysis steps of TDA. At first, the means and standard deviations of the TDCs were computed as following: (1) where NTDC is the number of subjects in TDCs; XTDC is a subset of subjects in TDC. Finally, the variance normalized data matrix () was computed as (2)

thumbnail
Fig 2. Preprocesses and analysis steps of topological data analysis.

Abbreviations: ADHD, attention-deficit/hyperactivity disorder; TDC, typically developing controls; Sx, symptom severity scores; IQ, intelligence quotient.

https://doi.org/10.1371/journal.pone.0182603.g002

Now, the distance metrics between α- and β-th subjects were computed by Euclidean distance as following: (3)

Then, we applied L-infinity centrality to compute filter metric as following: (4)

This computes the maximal distance to any other data points in the dataset X for every subject’ data point. Finally, Mapper, a tool for TDA, was used to find hidden subgroups in the patient-patient networks with the distance and filter metrics as input variables. After extracting the shape of the data, we mapped variables of interest onto the resulting topology, or patient-patient network, to unveil hidden neuropsychological characteristics of the output topology.

Functional network matrix

The datasets that we downloaded from the Neuro Bureau ADHD-200 repository were the filtered time courses files, ADHD 200_AAL_TCs_filtfix.tar.gz. A detailed description of the preprocessing steps can be found elsewhere [19, 29]. In brief, slice timing correction, coregistration, normalization, smoothing, regressions to remove physiological noises and head motion artifacts, and band-pass (0.009–0.08 Hz) filtering were applied. Using the filtered time-course data, we scrubbed time points for huge head motions by removing the fMRI scans using a framewise displacement > 1 mm [34]. Then, the functional network of each subject (Rij) was computed by Pearson’s correlation coefficients of the scrubbed time-courses between the i-th and j-th regions of interest. These functional connectivity matrices were used for further investigation of functional network properties.

Graph theoretical analysis

For the modularity optimization analysis and computation of centrality measures, we only used positive edge weights [26, 35]. The degree centrality of a node was computed by summation of all connection weights from its neighboring nodes [36]. The betweenness centrality and PageRank of a node were also computed following the definition described in elsewhere [36, 37]. The betweenness centrality of a node captures the global structure of the network. The PageRank evaluates the importance of nodes in network systems [38]. We set α to 0.85 for the evaluation of the PageRank.

To evaluate modular organization of each group, the group representing connection matrices were computed by averaging the functional network within each group. There are numerous approaches to compute the modular architectures in a network [3942]. Among them, we adopted the Louvain community detection method that uses a heuristic algorithm to compute the modularity. (5) where Aij is the connection weight of the edge between i and j; ; si = ∑j Aij is the degree centrality; Ci is the community structure to which node i is assigned; the delta function δ(x, y) is 1 if x = y and 0 otherwise. The heuristic modularity optimization algorithm produced slightly variable modular partitions and modularity (Q) run by run. Therefore, we performed 1000 independent optimization processes and found the best module partition that had the highest average value of normalized mutual information over all other optimization results [26].

Statistical analysis

The statistical comparisons of demographic variables, symptom severity, and IQ scores among subgroups identified by TDA were conducted by analysis of variance (ANOVA) methods. In addition, group level differences of graph theoretical measures and the intra- and inter-modules functional connectivity density (FCD) were investigated by analysis of covariance (ANCOVA). In addition, the statistically meaningful FCDs and centrality measures among the three groups were identified using a statistical significance of corrected P < 0.05, correcting for multiple comparisons using the Benjamini-Hochberg procedure [43]. Further investigations of the meaningful FCDs and centrality measures (P < 0.05, Bonferroni-corrected) were conducted using a post-hoc analysis to identify differences between any combinations of the groups.

Results

Participant characteristics

In the principal and validation datasets, no significant differences were found in age and handedness between the ADHD and TDC groups (see S1 Table online). However, significantly higher IQ scores were observed for both datasets (P < 0.05) in TDC subjects compared to children with ADHD.

Topological data analysis

Fig 3 presents the results of the TDA (or patient-patient network), showing a three branch-shaped graph for group separation in terms of the distance between the participants. TDA identified three subgroups of children: one TDC and two ADHD subgroups. The average values of the ADHD index and full-scale IQ within each node of the output topology are shown in Fig 3a and 3b. According to the variable mapping onto the patient-patient network, one branch in the patient-patient network of ADHD had milder inattentive and impulsivity/hyperactivity symptoms, labelled as mild ADHD (mADHD), and the other branch had more severe symptoms, labelled as severe ADHD (sADHD). Furthermore, for the quantitative analysis of the intelligence and ADHD symptoms among the three branches, we selected the 15 children lying on the most extreme peripheral position of each branch in the patient-patient networks. The demographic and clinical profiles for each subgroup are described in Table 1. Specifically, symptom severity of mADHD subgroup was significantly lower than that of sADHD subgroup (P < 0.05), whereas no significant differences in intelligence scores were observed between two ADHD subgroups (P > 0.05). The validation analysis confirmed that similar patterns in the patient-patient network were observed in the validation dataset. The shape of the data in the intelligence and symptom severity space obtained from the validation dataset was the same as the shape of the principal dataset, a three branch-shaped graph including severe and mild symptom ADHD subgroups (see S1 Fig online).

thumbnail
Fig 3. Output of topological data analysis of the principal dataset.

The patient-patient network discriminates children with ADHD into two distinct groups in terms of symptom severity. The average values of various clinical and neuroimaging variables within each node were plotted. Abbreviations: ADHD, attention-deficit/hyperactivity disorder; BGN, basal ganglia network; CAU, caudate; DC, degree centrality; DMN, default mode network; FCD, functional connectivity density; IQ, intelligence quotient; L, left; PCC, posterior cingulate cortex; R, right; PR, PageRank; SN, salience network; VN, visual network.

https://doi.org/10.1371/journal.pone.0182603.g003

Centrality measures among the three subgroups

The ANOVA test showed that the degree centrality and PageRank in the bilateral posterior cingulate cortex (PCC) were significantly different among the three groups (Corrected P < 0.05). The PageRank in the bilateral caudate was significantly different among the three groups (Corrected P < 0.05). The statistical comparisons of all nodal areas are summarized in supplementary materials (see S2 and S4 Tables online). Specifically, the degree centrality and PageRank in the bilateral PCC was significantly decreased in the sADHD group relative to that of the mADHD and TDC groups (Bonferroni-corrected P < 0.05). Interestingly, the PageRank in the bilateral caudate was significantly increased in both the mADHD and sADHD groups relative to the TDC group (Bonferroni-corrected P < 0.05). There were no significant differences in the betweenness centrality among three subgroups (S3 Table online). Furthermore, we mapped the degree centrality in the bilateral PCC (Fig 3c and 3d) and the PageRank in the bilateral PCC (Fig 3e and 3f) and bilateral caudate (Fig 3g and 3h) onto the patient-patient network to visualize the neural characteristics of each subgroup.

Modularity optimizations and functional connectivity density (FCD)

Fig 4a shows the patterns of the functional connectivity and modular organization for the three subgroups of ADHD. In brief, the default mode network (DMN), visual network (VN), salience network (SN), executive control network (ECN), and basal ganglia network (BGN) were identified in all subgroups, though each group had slightly different community membership. These memberships are described in S5 Table online. The functional communities obtained from the mADHD group were selected for statistical comparisons of FCD among the three groups. As shown in Table 2, the ANOVA test revealed significant group differences in the intra-module FCDs in the DMN, VN, and BGN (Corrected P < 0.05). Meanwhile, the inter-module FCDs in the DMN—ECN, DMN—SN, DMN—VN, SN—BGN, and VN—BGN were also significantly different among the three subgroups (Corrected P < 0.05).

thumbnail
Fig 4. Functional network modular organizations for typically developing controls (TDC), children with mild symptom ADHD (mADHD), and children with severe symptom ADHD (sADHD) (a), and statistical comparisons of the functional connectivity density among the three subgroups (b).

Modular partitions were obtained from the best community structure of the mADHD group and significant intra- and inter-module connectivities were plotted as bar graphs. *P < 0.05 indicates a statistical significance from the post-hoc analysis after Bonferroni correction. Abbreviations: ADHD, attention-deficit/hyperactivity disorder; BGN, basal ganglia network; DMN, default mode network; ECN, executive control network; SN, salience network; VN, visual network.

https://doi.org/10.1371/journal.pone.0182603.g004

thumbnail
Table 2. Analysis of covariance of the functional connectivity density (FCD) among three groups.

The statistical significance of the intra-module and inter-module FCD were summarized.

https://doi.org/10.1371/journal.pone.0182603.t002

Fig 4b shows the significant intra- and inter-module FCDs among the three groups and presents the significance from the post-hoc analyses. Relative to the TDCs, the intra-module FCDs within the VN and BGN, and inter-module FCD in the DMN—SN were significantly increased in the mADHD as well as the sADHD groups (Corrected P < 0.05). Meanwhile, the intra-module FCD within the DMN was significantly decreased in the sADHD group compared to the TDC groups (Corrected P < 0.05). In contrast, the inter-module FCD in the DMN—VN was significantly decreased in the mADHD and sADHD groups relative to the TDCs (Corrected P < 0.05). In addition, the inter-module FCD in the SN—BGN was significantly decreased in the sADHD relative to the TDC group (Corrected P < 0.05), and the inter-module FCD in the VN—BGN was significantly decreased in the mADHD relative to the TDC group (Corrected P < 0.05). Finally, the inter-module FCD of the DMN—ECN in the sADHD group was significantly increased relative to that of the TDC and mADHD groups (Corrected P < 0.05). These significant FCD values were mapped onto the patient-patient network, as shown in Fig 3i–3p. Visualization of the variables on the topology provided a complementary information to the F-test among the three groups.

Discussion

In the present study, we demonstrated that the use of common clinical phenotypes (symptom severity and IQ) and an innovative unsupervised data-driven machine learning algorithm such as TDA is an informative approach for understanding the heterogeneity of ADHD. We also examined whole-brain intrinsic functional connectivity and identified the specific brain network that shows significant differences between the mADHD group and the sADHD or the TDC groups. This result suggests that the identification of the mADHD and sADHD subgroups may be biologically meaningful when the functional connectome is examined using graph theoretical approaches.

Subgroup identification of ADHD by TDA

Our results showed that TDA produced an easily recognizable branch-shaped graph with three progressive arms and that individuals with ADHD in the same arm shared similar characteristics to form specific subgroups of the disorder. Similar results were also replicated in the independent validation dataset. The most compelling finding of this study is the identification of two unique phenotypes (mADHD and sADHD), which overlaps with current ADHD categories identified in the Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5) [44], i.e., combined, inattentive, and hyperactive/impulsive presentation. As shown in Table 1 (see also S2 Fig online), most of the sADHD (80%) and mADHD (73%) overlapped with the combined subtype and the inattentive subtype, respectively. Given that the wording ‘subtype’ that was used in the DSM-IV to categorize the children with ADHD has been changed as ‘presentation’ in the DSM-5 [44], the mADHD and sADHD identified by TDA might be called as predominately inattentive presentation and predominantly combined presentation, respectively. In fact, the combined subtype was clinically more severe than the inattentive subtype (see S6 Table online). However, these two subtypes showed no significant differences in the functional network measures (see S7S10 Tables online). Taken together, although mADHD and sADHD showed overlaps in the DSM-IV subgroups or DSM-5 presentation, they are ‘novel subgroups’ regards to showing significant differences in the functional network measures as well as symptom severity.

In our previous study, we have failed to detect subgroups of ADHD using the functional connectome data as the input features [19]. Healthy state modeling, which introduced in the first TDA paper that identified a subgroup of breast cancers [11], was used to extract disease components from the functional imaging data. Then, the filter metric was computed as the size of disease component. The filter metric might have captured the overall characteristic of ADHD, but failed to capture additional information to classify subgroups of ADHD. However, the current study used easily obtainable clinical measures, such as ADHD symptom severity and IQ scores, as the input features to the TDA. We used L-infinity centrality as the filter metric, which computes for every data point the maximal distance to any other data point in the set, and thus the extreme (highest or lowest) filter metric corresponds to flares (or branches) in the patient-patient network of the TDA output. Therefore, the current approach has an advantage in being readily translated into clinical practice. Notably, these two subgroups displayed both similar and distinct functional connectivity findings in whole-brain and subnetwork connectivity, suggesting the biological validity of our subgroups. It is worth noting that the advantage of TDA is the ease with which it can be used to visualize similarities and differences between the mADHD and sADHD groups by mapping variables of interest, as shown in Fig 3.

Functional network modular organization

In the analysis of functional network modular organization for the mADHD, sADHD, and TDC groups, we observed altered functional connectivity within several large-scale brain networks in children with ADHD compared with TDCs. Specifically, relative to the TDC group, the sADHD group displayed increased intra-module connectivity in the VN and BGN, and inter-module connectivity in the DMN—ECN, DMN—SN, DMN—VN, but decreased intra-module connectivity in the DMN and inter-module connectivity in the SN—BGN. Our results also revealed that the mADHD group generally showed similar patterns of modular architectures to that of the sADHD group. For example, intra-module connectivity in the VN and BGN, and inter-module connectivity in the DMN—SN and DMN—VN were not significantly different between the mADHD and sADHD groups, but the values were significantly different between mADHD and TDC groups. Taken together, our results suggest that both intra-module and inter-module functional connectivity are critical components of the underlying neurobiology of ADHD. In general, abnormal intra- and inter-module connectivity may limit dynamic interactions among networks, which are necessary for successful processing of complex stimuli in the real world [45].

Regarding the hypoconnectivity within the DMN and the decreased degree centrality and PageRank in the bilateral PCC regions, our results support the idea that ADHD might be considered as a default network disorder [4650]. The altered inter-module connectivity between the DMN and the task-positive regions found in our ADHD group may be associated with a dysfunction in the switching of the brain state from a resting to functioning state [50], thus producing periodic lapses in on-task performance, which is a hallmark of ADHD [51].

Focusing on the subcortical system of ADHD, previous study associated the malfunction of the cognitive control in ADHD with abnormality in the striatal circuits [52]. The mADHD and sADHD showed an increase in PageRank in the bilateral caudate (S4 Table online) and FCD within the BGN (Fig 2b), which includes the striatum and thalamus, relative to the TDC group. The striatum has multiple roles relevant to ADHD such as executive function [53] and reward processing [54]. Considering that the PageRank measures an importance of a node in the network system [37], the increase in PageRank relative to the normal condition might be interpreted as information overload and might be associated with dysfunction in the striatal circuits.

Instead of the traditional simple models of ADHD, mainly involving DMN—ECN FC or within-DMN FC, our findings with regard to the VN and SN are consistent with the recent neurobiological formulations of ADHD [49, 5557], implicating atypical functional connectivity within and between networks affecting sensory and salience processing. This theory suggests that enhanced bottom-up processing of external stimuli, as well as top-down processing of executive control, can contribute to increased distractibility in ADHD patients [57, 58]. Specifically, the visual cortex interacts with the dorsal attention network [59, 60] to maintain attention and suppress attention to irrelevant stimuli [55, 61]. Therefore, the abnormal intra- and inter-module VN connectivity found in our mADHD and sADHD groups may affect poor inhibition of sensory perception in ADHD patients [6266]. The salience network integrates external sensory stimuli with internal states, and it is critical for attention allocation to stimuli that are salient to the individual [67] and units conflict monitoring, interoceptive-autonomic, and reward processing centers [68]. In addition, diminished DMN connections with the VN and SN may suggest that there is reduced information transfer from the sensory or salience network to the DMN [69]. The role of BGN has been less studied compared to that of other networks, but it has been implicated in working memory, motivation and reward-related processing in prior studies [70]. Taken together, our results are consistent with the current multi-network models in ADHD, which suggest that many psychiatric conditions, including ADHD, are characterized by inappropriate engagement of the SN with the ECN, DMN, VN, and BGN [55, 68, 71].

Neural characteristics of our mADHD and sADHD subgroups

Our study results revealed that aberrant connectivity in the DMN, VN, BGN, ECN, or SN mainly concerned our sADHD group. The mADHD group shared neural signatures with the TDC group as well as the sADHD group. First, the degree centrality and PageRank in the bilateral PCC and intra-module connectivity in the DMN of the mADHD group were found to be intermediate between the sADHD and TDC groups. In contrast, relative to the TDC group, inter-module hypoconnectivity in the DMN—VN, inter-module hyperconnectivity in the DMN—SN, and intra-module hyperconnectivity in the VN and BGN were observed in both the mADHD and sADHD groups. However, inter-module hyperconnectivity in the DMN—ECN was observed in the sADHD group, but not in the mADHD group compared to TDC group. These findings show that mADHD and sADHD are somewhat different in terms of their DMN—ECN connectivity, suggesting neurobiological differences between these two subgroups. The DMN and ECN are known as the task negative and task positive regions, respectively, indicating that they should be anti-correlated in the normal brain [72]. Therefore, ADHD subjects with mild symptoms might be different from those with severe symptoms and more similar to TDC in terms of their DMN—ECN network.

Despite the important findings, several limitations should be considered for the proper interpretation of our results. Our findings cannot be extended to the entire ADHD population because we only used samples from boys to avoid possible bias from sex differences [73]. Furthermore, in this analysis of archival data, we were limited by the content of the datasets available. In the ADHD-200 datasets, medication status was missing for a large number of subjects, including the doses and types of medications and whether the subjects were under the effects of the medication during the fMRI scan. A history of treatment with stimulant medication may be relevant in interpreting our results. In addition, significantly higher IQ scores in TDC subjects than children with ADHD were observed in the principal and validation datasets. As individual differences in intrinsic functional connectivity are systematically related to measures of IQ [74] and behavioral variability [75], it is possible that the identified differences may be due to IQ differences rather than differences in symptom severity. The selection of the 15 subjects at the extreme of each subgroup was somewhat arbitrary. However, this was a post-hoc comparison and selection of 15 subjects is enough to produce statistically significant results. Our findings are somewhat dissimilar to the results from prior studies reporting reduced connectivity between the DMN and other networks of brain, such as the ECN [4649]. These discrepancies are explained by the developmental perspective that suggests that DMN connections known to increase with development are weaker in participants with ADHD, and conversely, that a DMN connection known to decrease with development is abnormally increased in the children with ADHD [47, 49]. Therefore, further work with larger samples, tighter age ranges, and more standardized protocols for diagnosis is needed and the effects of comorbidities should be considered in future studies.

Conclusion

Despite the limitations mentioned above, we showed two notable findings in this study. First, TDA is potentially useful in defining less heterogeneous clinical subgroups in ADHD. Second, neuroimaging measurements are potential validators of phenotypic groups identified using behavioral data. Taken together, data-driven and biologically relevant disease phenotypes identified using TDA will give researchers a unique opportunity to identify objective and biologically relevant disease categories. The identification of subgroups with distinct patterns of clinical manifestation may lead to development of more effective and better-targeted intervention for individualized treatment and improved patient care.

Supporting information

S1 Fig. Output of topological data analysis (TDA) of the validation dataset.

The patient-patient network discriminates children with ADHD into two distinct groups in terms of symptom severity. The average values of full-scale IQ and ADHD index within each node were plotted. Abbreviations: ADHD, attention-deficit/hyperactivity disorder; IQ, intelligence quotient; NYU, New York University.

https://doi.org/10.1371/journal.pone.0182603.s001

(DOCX)

S2 Fig. Mapping clinical subtypes on the patient-patient network produced by topological data analysis (TDA) of the principal dataset.

The patient-patient network discriminates children with ADHD into two distinct groups regarding symptom severity. Most severe symptom ADHD (sADHD) were the clinically combined subtypes, whereas most mild symptom ADHD (mADHD) were the clinically inattentive subtype. Each node indicates a group of subjects with similar characteristics in dimensions of symptom severity and intelligence score. Node color indicates the ratio of subjects with a clinical subtype. Abbreviations: ADHD, attention-deficit/hyperactivity disorder; IQ, intelligence quotient; PKU, Peking University.

https://doi.org/10.1371/journal.pone.0182603.s002

(DOCX)

S1 Table. Demographic and clinical characteristics of the principal and validation datasets.

https://doi.org/10.1371/journal.pone.0182603.s003

(DOCX)

S2 Table. Mean values of degree centrality for each mADHD and sADHD subgroup and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s004

(DOCX)

S3 Table. Mean values of betweenness centrality for each mADHD and sADHD subgroup and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s005

(DOCX)

S4 Table. Mean values of PageRank for each mADHD and sADHD subgroup and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s006

(DOCX)

S5 Table. Functional modular organization for each mADHD and sADHD subgroup.

Modularity optimization analyses were conducted for the principal data set.

https://doi.org/10.1371/journal.pone.0182603.s007

(DOCX)

S6 Table. Demographic and clinical characteristics of the inattentive and combined subtypes.

https://doi.org/10.1371/journal.pone.0182603.s008

(DOCX)

S7 Table. Two-sample t-test of the functional connectivity density (FCD) between the inattentive and combined subtypes.

The statistical significance of the intra-module and inter-module FCD were summarized.

https://doi.org/10.1371/journal.pone.0182603.s009

(DOCX)

S8 Table. Mean values of degree centrality for each inattentive and combined subtype and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s010

(DOCX)

S9 Table. Mean values of betweenness centrality for each inattentive and combined subtype and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s011

(DOCX)

S10 Table. Mean values of PageRank for each inattentive and combined subtype and its statistical comparison using analysis of variance.

https://doi.org/10.1371/journal.pone.0182603.s012

(DOCX)

Acknowledgments

We thank the ADHD-200 Consortium for sharing the preprocessed version of the rs-fMRI data and phenotypic information.

References

  1. 1. Faraone SV, Biederman J. What Is the Prevalence of Adult ADHD? Results of a Population Screen of 966 Adults. J Atten Disord. 2005;9(2):384–91. pmid:16371661.
  2. 2. Sonuga-Barke EJ, Coghill D. The foundations of next generation attention-deficit/hyperactivity disorder neuropsychology: building on progress during the last 30 years. J Child Psychol Psychiatry. 2014;55(12):e1–5. pmid:25399637.
  3. 3. Sjöwall D, Roth L, Lindqvist S, Thorell LB. Multiple deficits in ADHD: executive dysfunction, delay aversion, reaction time variability, and emotional deficits. J Child Psychol Psychiatry. 2013;54(6):619–27. pmid:23061803
  4. 4. Karalunas SL, Fair D, Musser ED, Aykes K, Iyer SP, Nigg JT. Subtyping attention-deficit/hyperactivity disorder using temperament dimensions: toward biologically based nosologic criteria. JAMA Psychiatry. 2014;71(9):1015–24. pmid:25006969.
  5. 5. The ADHD-200 Consortium. The ADHD-200 Consortium: A Model to Advance the Translational Potential of Neuroimaging in Clinical Neuroscience. Front Syst Neurosci. 2012;6:62. pmid:22973200.
  6. 6. Wang X, Qian B, Davidson I. On constrained spectral clustering and its applications. Data Min Knowl Discov. 2014;28(1):1–30.
  7. 7. Shin JE, Kyeong S, Lee JS, Park JY, Lee WS, Kim JJ, et al. A personality trait contributes to the occurrence of postoperative delirium: a prospective study. BMC Psychiatry. 2016;16(1):371. pmid:27809817.
  8. 8. Li L, Cheng WY, Glicksberg BS, Gottesman O, Tamler R, Chen R, et al. Identification of type 2 diabetes subgroups through topological analysis of patient similarity. Sci Transl Med. 2015;7(311):311ra174. pmid:26511511.
  9. 9. Nielson JL, Paquette J, Liu AW, Guandique CF, Tovar CA, Inoue T, et al. Topological data analysis for discovery in preclinical spinal cord injury and traumatic brain injury. Nat Commun. 2015;6:8581. pmid:26466022.
  10. 10. Lum PY, Singh G, Lehman A, Ishkanov T, Vejdemo-Johansson M, Alagappan M, et al. Extracting insights from the shape of complex data using topology. Sci Rep. 2013;3:1236. pmid:23393618.
  11. 11. Nicolau M, Levine AJ, Carlsson G. Topology based data analysis identifies a subgroup of breast cancers with a unique mutational profile and excellent survival. Proc Natl Acad Sci U S A. 2011;108(17):7265–70. pmid:21482760.
  12. 12. Yao Y, Sun J, Huang X, Bowman GR, Singh G, Lesnick M, et al. Topological methods for exploring low-density states in biomolecular folding pathways. J Chem Phys. 2009;130(14):144115. pmid:19368437.
  13. 13. Marsh PJ, Williams LM. An investigation of individual typologies of attention-deficit hyperactivity disorder using cluster analysis of DSM—IV criteria. Pers Individ Dif. 2004;36(5):1187–95.
  14. 14. Pauli-Pott U, Dalir S, Mingebach T, Roller A, Becker K. Attention deficit/hyperactivity and comorbid symptoms in preschoolers: Differences between subgroups in neuropsychological basic deficits. Child Neuropsychol. 2014;20(2):230–44. pmid:23557149.
  15. 15. Bonafina MA, Newcorn JH, McKay KE, Koda VH, Halperin JM. ADHD and reading disabilities: a cluster analytic approach for distinguishing subgroups. J Learn Disabil. 2000;33(3):297–307. pmid:15505967.
  16. 16. Robin AL, Tzelepis A, Bedway M. A cluster analysis of personality style in adults with ADHD. J Atten Disord. 2008;12(3):254–63. pmid:18460638.
  17. 17. Roberts BA, Martel MM, Nigg JT. Are There Executive Dysfunction Subtypes Within ADHD? J Atten Disord. 2013:1087054713510349. pmid:24214969.
  18. 18. Fair DA, Bathula D, Nikolas MA, Nigg JT. Distinct neuropsychological subgroups in typically developing youth inform heterogeneity in children with ADHD. Proc Natl Acad Sci U S A. 2012;109(17):6769–74. pmid:22474392.
  19. 19. Kyeong S, Park S, Cheon KA, Kim JJ, Song DH, Kim E. A New Approach to Investigate the Association between Brain Functional Connectivity and Disease Characteristics of Attention-Deficit/Hyperactivity Disorder: Topological Neuroimaging Data Analysis. PLoS One. 2015;10(9):e0137296. pmid:26352147.
  20. 20. Cheung CH, Rijdijk F, McLoughlin G, Faraone SV, Asherson P, Kuntsi J. Childhood predictors of adolescent and young adult outcome in ADHD. J Psychiatr Res. 2015;62:92–100. pmid:25680235.
  21. 21. Rasmussen P, Gillberg C. Natural outcome of ADHD with developmental coordination disorder at age 22 years: a controlled, longitudinal, community-based study. J Am Acad Child Adolesc Psychiatry. 2000;39(11):1424–31. pmid:11068898.
  22. 22. Biederman J, Petty CR, Clarke A, Lomedico A, Faraone SV. Predictors of persistent ADHD: an 11-year follow-up study. J Psychiatr Res. 2011;45(2):150–5. pmid:20656298.
  23. 23. Rommel AS, Rijsdijk F, Greven CU, Asherson P, Kuntsi J. A Longitudinal Twin Study of the Direction of Effects between ADHD Symptoms and IQ. PLoS One. 2015;10(4):e0124357. pmid:25875897.
  24. 24. Mill J, Caspi A, Williams BS, Craig I, Taylor A, Polo-Tomas M, et al. Prediction of heterogeneity in intelligence and adult prognosis by genetic polymorphisms in the dopamine system among children with attention-deficit/hyperactivity disorder: evidence from 2 birth cohorts. Arch Gen Psychiatry. 2006;63(4):462–9. pmid:16585476.
  25. 25. de Zeeuw P, Schnack HG, van Belle J, Weusten J, van Dijk S, Langen M, et al. Differential brain development with low and high IQ in attention-deficit/hyperactivity disorder. PLoS One. 2012;7(4):e35770. pmid:22536435.
  26. 26. Kyeong S, Kim E, Park HJ, Hwang DU. Functional network organizations of two contrasting temperament groups in dimensions of novelty seeking and harm avoidance. Brain Res. 2014;1575:33–44. pmid:24881884.
  27. 27. Gates KM, Molenaar PC, Iyer SP, Nigg JT, Fair DA. Organizing heterogeneous samples using community detection of GIMME-derived resting state functional networks. PLoS One. 2014;9(3):e91322. pmid:24642753.
  28. 28. Cai W, Chen T, Szegletes L, Supekar K, Menon V. Aberrant Cross-Brain Network Interaction in Children With Attention-Deficit/Hyperactivity Disorder and Its Relation to Attention Deficits: A Multisite and Cross-Site Replication Study. Biol Psychiatry. 2015. pmid:26805582.
  29. 29. Bellec P, Chu C, Chouinard-Decorte F, Benhajali Y, Margulies DS, Craddock RC. The Neuro Bureau ADHD-200 Preprocessed repository. NeuroImage. 2017;144(Pt B):275–86. pmid:27423255.
  30. 30. DuPaul GJ, Power TJ, Anastopoulos AD, Reid R. ADHD Rating Scale IV: Checklists, Norms, and Clinical Interpretation. New York: Guilford; 1998.
  31. 31. Conners CK, Sitarenios G, Parker JD, Epstein JN. The revised Conners' Parent Rating Scale (CPRS-R): factor structure, reliability, and criterion validity. J Abnorm Child Psychol. 1998;26(4):257–68. pmid:9700518.
  32. 32. Li D, Jin Y, Vandenberg SG, Zhu YM, Tang CH. Report on Shanghai norms for the Chinese translation of the Wechsler Intelligence Scale for Children-Revised. Psychol Rep. 1990;67(2):531–41. pmid:2263706.
  33. 33. Wechsler D, Hsiao-pin C. WASI-II: Wechsler abbreviated scale of intelligence. San Antonio: TX: Psychological Corporation; 2011.
  34. 34. Power JD, Barnes KA, Snyder AZ, Schlaggar BL, Petersen SE. Steps toward optimizing motion artifact removal in functional connectivity MRI; a reply to Carp. NeuroImage. 2013;76:439–41. pmid:22440651.
  35. 35. Shin DJ, Jung WH, He Y, Wang J, Shim G, Byun MS, et al. The effects of pharmacological treatment on functional brain connectome in obsessive-compulsive disorder. Biol Psychiatry. 2014;75(8):606–14. pmid:24099506.
  36. 36. Rubinov M, Sporns O. Complex network measures of brain connectivity: uses and interpretations. NeuroImage. 2010;52(3):1059–69. pmid:19819337.
  37. 37. Morrison JL, Breitling R, Higham DJ, Gilbert DR. GeneRank: Using search engine technology for the analysis of microarray experiments. BMC Bioinformatics. 2005;6(1):233. pmid:16176585.
  38. 38. Sen P, Hernán AM. Spreading dynamics in complex networks. J Stat Mech-Theory E. 2013;2013(12):P12002.
  39. 39. Newman MEJ. Modularity and community structure in networks. Proc Natl Acad Sci U S A. 2006;103(23):8577–82. pmid:16723398.
  40. 40. Newman MEJ. Communities, modules and large-scale structure in networks. Nat Phys. 2012;8(1):25–31.
  41. 41. Sun Y, Danila B, Josić K, Bassler KE. Improved community structure detection using a modified fine-tuning strategy. Europhys Lett. 2009;86(2):28004.
  42. 42. Vincent DB, Jean-Loup G, Renaud L, Etienne L. Fast unfolding of communities in large networks. J Stat Mech-Theory E. 2008;2008(10):P10008.
  43. 43. Benjamini Y, Hochberg Y. Controlling the False Discovery Rate: A Practical and Powerful Approach to Multiple Testing. J R Statist Soc B. 1995;57(1):289–300.
  44. 44. American Psychiatric Association. Diagnostic and Statistical Manual of Mental Disorders, Fifth Edition (DSM-5). Washington, DC: American Psychiatric Association; 2013.
  45. 45. Uddin LQ, Supekar K, Lynch CJ, Khouzam A, Phillips J, Feinstein C, et al. Salience network-based classification and prediction of symptom severity in children with autism. JAMA Psychiatry. 2013;70(8):869–79. pmid:23803651.
  46. 46. Castellanos FX, Margulies DS, Kelly AMC, Uddin LQ, Ghaffari M, Kirsch A, et al. Cingulate—Precuneus Interactions: A New Locus of Dysfunction in Adult Attention-Deficit/Hyperactivity Disorder. Biol Psychiatry. 2008;63(3):332–7. pmid:17888409.
  47. 47. Fair DA, Posner J, Nagel BJ, Bathula D, Dias TG, Mills KL, et al. Atypical default network connectivity in youth with attention-deficit/hyperactivity disorder. Biol Psychiatry. 2010;68(12):1084–91. pmid:20728873.
  48. 48. Qiu MG, Ye Z, Li QY, Liu GJ, Xie B, Wang J. Changes of brain structure and function in ADHD children. Brain Topogr. 2011;24(3–4):243–52. pmid:21191807.
  49. 49. Posner J, Park C, Wang Z. Connecting the dots: a review of resting connectivity MRI studies in attention-deficit/hyperactivity disorder. Neuropsychol Rev. 2014;24(1):3–15. pmid:24496902.
  50. 50. Liddle EB, Hollis C, Batty MJ, Groom MJ, Totman JJ, Liotti M, et al. Task-related default mode network modulation and inhibitory control in ADHD: effects of motivation and methylphenidate. J Child Psychol Psychiatry. 2011;52(7):761–71. pmid:21073458.
  51. 51. Castellanos FX, Sonuga-Barke EJ, Scheres A, Di Martino A, Hyde C, Walters JR. Varieties of attention-deficit/hyperactivity disorder-related intra-individual variability. Biol Psychiatry. 2005;57(11):1416–23. pmid:15950016.
  52. 52. Durston S, Tottenham NT, Thomas KM, Davidson MC, Eigsti IM, Yang Y, et al. Differential patterns of striatal activation in young children with and without ADHD. Biol Psychiatry. 2003;53(10):871–8. pmid:12742674.
  53. 53. Castellanos FX, Sonuga-Barke EJ, Milham MP, Tannock R. Characterizing cognition in ADHD: beyond executive dysfunction. Trends Cogn Sci. 2006;10(3):117–23. pmid:16460990.
  54. 54. Schultz W. Behavioral theories and the neurophysiology of reward. Annu Rev Psychol. 2006;57:87–115. pmid:16318590.
  55. 55. Castellanos FX, Proal E. Large-scale brain systems in ADHD: beyond the prefrontal-striatal model. Trends Cogn Sci. 2012;16(1):17–26. pmid:22169776.
  56. 56. Carmona S, Hoekzema E, Castellanos FX, Garcia-Garcia D, Lage-Castellanos A, Van Dijk KR, et al. Sensation-to-cognition cortical streams in attention-deficit/hyperactivity disorder. Hum Brain Mapp. 2015;36(7):2544–57. pmid:25821110.
  57. 57. Tegelbeckers J, Bunzeck N, Duzel E, Bonath B, Flechtner HH, Krauel K. Altered salience processing in attention deficit hyperactivity disorder. Hum Brain Mapp. 2015;36(6):2049–60. pmid:25648705.
  58. 58. Schroger E, Marzecova A, SanMiguel I. Attention and prediction in human audition: a lesson from cognitive psychophysiology. Eur J Neurosci. 2015;41(5):641–64. pmid:25728182.
  59. 59. Shulman GL, Astafiev SV, Franke D, Pope DL, Snyder AZ, McAvoy MP, et al. Interaction of stimulus-driven reorienting and expectation in ventral and dorsal frontoparietal and basal ganglia-cortical networks. J Neurosci. 2009;29(14):4392–407. pmid:19357267.
  60. 60. Capotosto P, Babiloni C, Romani GL, Corbetta M. Frontoparietal cortex controls spatial attention through modulation of anticipatory alpha rhythms. J Neurosci. 2009;29(18):5863–72. pmid:19420253.
  61. 61. Yeo BT, Krienen FM, Sepulcre J, Sabuncu MR, Lashkari D, Hollinshead M, et al. The organization of the human cerebral cortex estimated by intrinsic functional connectivity. J Neurophysiol. 2011;106(3):1125–65. pmid:21653723.
  62. 62. McCarthy H, Skokauskas N, Mulligan A, Donohoe G, Mullins D, Kelly J, et al. Attention network hypoconnectivity with default and affective network hyperconnectivity in adults diagnosed with attention-deficit/hyperactivity disorder in childhood. JAMA Psychiatry. 2013;70(12):1329–37. pmid:24132732.
  63. 63. Cao X, Cao Q, Long X, Sun L, Sui M, Zhu C, et al. Abnormal resting-state functional connectivity patterns of the putamen in medication-naive children with attention deficit hyperactivity disorder. Brain Res. 2009;1303:195–206. pmid:19699190.
  64. 64. Dillo W, Goke A, Prox-Vagedes V, Szycik GR, Roy M, Donnerstag F, et al. Neuronal correlates of ADHD in adults with evidence for compensation strategies—a functional MRI study with a Go/No-Go paradigm. Ger Med Sci. 2010;8:Doc09. pmid:20421953.
  65. 65. Schneider MF, Krick CM, Retz W, Hengesch G, Retz-Junginger P, Reith W, et al. Impairment of fronto-striatal and parietal cerebral networks correlates with attention deficit hyperactivity disorder (ADHD) psychopathology in adults—a functional magnetic resonance imaging (fMRI) study. Psychiatry Res. 2010;183(1):75–84. pmid:20558047.
  66. 66. Hale TS, Bookheimer S, McGough JJ, Phillips JM, McCracken JT. Atypical brain activation during simple & complex levels of processing in adult ADHD: an fMRI study. J Atten Disord. 2007;11(2):125–40. pmid:17494829.
  67. 67. Menon V, Uddin LQ. Saliency, switching, attention and control: a network model of insula function. Brain Struct Funct. 2010;214(5–6):655–67. pmid:20512370.
  68. 68. Seeley WW, Menon V, Schatzberg AF, Keller J, Glover GH, Kenna H, et al. Dissociable intrinsic connectivity networks for salience processing and executive control. J Neurosci. 2007;27(9):2349–56. pmid:17329432.
  69. 69. Kessler D, Angstadt M, Welsh RC, Sripada C. Modality-spanning deficits in attention-deficit/hyperactivity disorder in functional networks, gray matter, and white matter. J Neurosci. 2014;34(50):16555–66. pmid:25505309.
  70. 70. Shaw P, De Rossi P, Watson B, Wharton A, Greenstein D, Raznahan A, et al. Mapping the development of the basal ganglia in children with attention-deficit/hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. 2014;53(7):780–9 e11. pmid:24954827.
  71. 71. Castellanos FX, Aoki Y. Intrinsic Functional Connectivity in Attention-Deficit/Hyperactivity Disorder: A Science in Development. Biol Psychiatry Cogn Neurosci Neuroimaging. 2016;1(3):253–61. pmid:27713929.
  72. 72. Fox MD, Snyder AZ, Vincent JL, Corbetta M, Van Essen DC, Raichle ME. The human brain is intrinsically organized into dynamic, anticorrelated functional networks. Proc Natl Acad Sci U S A. 2005;102(27):9673–8. pmid:15976020.
  73. 73. Tomasi D, Volkow ND. Abnormal functional connectivity in children with attention-deficit/hyperactivity disorder. Biol Psychiatry. 2012;71(5):443–50. pmid:22153589.
  74. 74. van den Heuvel MP, Stam CJ, Kahn RS, Hulshoff Pol HE. Efficiency of functional brain networks and intellectual performance. J Neurosci. 2009;29(23):7619–24. pmid:19515930.
  75. 75. Kelly AMC, Uddin LQ, Biswal BB, Castellanos FX, Milham MP. Competition between functional brain networks mediates behavioral variability. NeuroImage. 2008;39(1):527–37. pmid:17919929.