Advertisement
  • Loading metrics

One for all and all for One: Improving replication of genetic studies through network diffusion

  • Daniel Lancour,

    Roles Conceptualization, Formal analysis, Methodology, Software, Writing – original draft

    Affiliations Bioinformatics Graduate Program, Boston University, Boston, Massachusetts, United States of America, Department of Medicine (Biomedical Genetics), Boston University School of Medicine, Boston, Massachusetts, United States of America

  • Adam Naj,

    Roles Data curation, Formal analysis, Writing – review & editing

    Affiliation Department of Pathology and Laboratory Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America

  • Richard Mayeux,

    Roles Data curation, Funding acquisition, Writing – review & editing

    Affiliation Department of Neurology and Sergievsky Center, Columbia University, New York, New York, United States of America

  • Jonathan L. Haines,

    Roles Funding acquisition, Writing – review & editing

    Affiliation Department of Epidemiology and Biostatistics, Case Western Reserve University, Cleveland, Ohio, United States of America

  • Margaret A. Pericak-Vance,

    Roles Data curation, Funding acquisition, Writing – review & editing

    Affiliation Hussman Institute for Human Genomics, University of Miami Miller School of Medicine, Miami, Florida, United States of America

  • Gerard D. Schellenberg,

    Roles Data curation, Funding acquisition, Writing – review & editing

    Affiliation Department of Pathology and Laboratory Medicine, University of Pennsylvania, Philadelphia, Pennsylvania, United States of America

  • Mark Crovella ,

    Roles Conceptualization, Methodology, Supervision, Writing – review & editing

    ‡ These authors are joint senior authors on this work.

    Affiliations Bioinformatics Graduate Program, Boston University, Boston, Massachusetts, United States of America, Department of Computer Science, Boston University, Boston, Massachusetts, United States of America

  • Lindsay A. Farrer ,

    Roles Conceptualization, Data curation, Funding acquisition, Investigation, Project administration, Resources, Supervision, Writing – original draft, Writing – review & editing

    farrer@bu.edu

    ‡ These authors are joint senior authors on this work.

    Affiliations Bioinformatics Graduate Program, Boston University, Boston, Massachusetts, United States of America, Department of Medicine (Biomedical Genetics), Boston University School of Medicine, Boston, Massachusetts, United States of America, Department of Neurology, Boston University School of Medicine, Boston, Massachusetts, United States of America, Department of Ophthalmology, Boston University School of Medicine, Boston, Massachusetts, United States of America, Department of Biostatistics, Boston University School of Public Health, Boston, Massachusetts, United States of America, Department of Epidemiology, Boston University School of Public Health, Boston, Massachusetts, United States of America

  • Simon Kasif

    Roles Conceptualization, Methodology, Project administration, Supervision, Writing – review & editing

    ‡ These authors are joint senior authors on this work.

    Affiliations Bioinformatics Graduate Program, Boston University, Boston, Massachusetts, United States of America, Department of Biomedical Engineering, Boston University, Boston, Massachusetts, United States of America

One for all and all for One: Improving replication of genetic studies through network diffusion

  • Daniel Lancour, 
  • Adam Naj, 
  • Richard Mayeux, 
  • Jonathan L. Haines, 
  • Margaret A. Pericak-Vance, 
  • Gerard D. Schellenberg, 
  • Mark Crovella, 
  • Lindsay A. Farrer, 
  • Simon Kasif
PLOS
x

Abstract

Improving accuracy in genetic studies would greatly accelerate understanding the genetic basis of complex diseases. One approach to achieve such an improvement for risk variants identified by the genome wide association study (GWAS) approach is to incorporate previously known biology when screening variants across the genome. We developed a simple approach for improving the prioritization of candidate disease genes that incorporates a network diffusion of scores from known disease genes using a protein network and a novel integration with GWAS risk scores, and tested this approach on a large Alzheimer disease (AD) GWAS dataset. Using a statistical bootstrap approach, we cross-validated the method and for the first time showed that a network approach improves the expected replication rates in GWAS studies. Several novel AD genes were predicted including CR2, SHARPIN, and PTPN2. Our re-prioritized results are enriched for established known AD-associated biological pathways including inflammation, immune response, and metabolism, whereas standard non-prioritized results were not. Our findings support a strategy of considering network information when investigating genetic risk factors.

Author summary

Integrating multiple types of -omics data is a rapidly growing research area due in part to the increasing amount of diverse and publicly accessible data. In this study, we demonstrated that integration of genetic association and protein interaction data using a network diffusion approach measurably improves reproducibility of top candidate genes. Application of this approach to Alzheimer disease (AD) using a large dataset assembled by the Alzheimer’s Disease Genetics Consortium identified several novel candidate AD genes that are supported by pre-existing knowledge of AD pathobiology. Our findings support a strategy of considering network information when investigating genetic risk factors. Finally, we developed a transparent and easy-to-use R package that can facilitate the extension of our methodology to other phenotypes for which genetic data are available.

Introduction

The discovery of disease-associated genomic variation has numerous clinical and scientific applications, including earlier disease prognosis, improved understanding of disease pathophysiology, and development of personalized treatment therapies [1]. A commonly used technique for identifying these mutations is the genome wide association study (GWAS) approach [2]. Typically, a large sample of affected and unaffected individuals are genotyped for many single nucleotide polymorphisms (SNPs) using a high-density microarray chip and then test statistically if the allele frequency of each variant is associated with disease status [2]. Significant associations in this first step (“discovery phase”) are deemed to be robust if they replicate in an independent cohort (“replication phase”). In this study, we focused on improving the replicability of GWAS results for Alzheimer disease (AD), although our methodology is applicable to genetic data for other diseases and traits. AD is a neurodegenerative disease resulting in irreversible dementia and memory loss with elevated prevalence in older populations [3]. Recent estimates suggest that approximately 5.4 million Americans have AD, and the number of cases of AD is expected to increase dramatically in future years if medical advances continue to improve life expectancy, thereby allowing more individuals to reach ages where AD is on the rise [3].

Genetic studies of AD have led to identifying numerous AD associated genes such as APP [4], PSEN1 [5], and PSEN2 [6] for early onset AD (EOAD), as well as APOE [7, 8] and SORL1 [8, 9] for late onset AD (LOAD). Common variants in more than 20 other genes have been robustly associated with AD risk [8]. However, not all AD associated genes will reach genome wide significance in current datasets of sample sizes below 100,000 individuals. It is well recognized that incorporating other forms of biological data improves confidence in genetic findings [1012].

Our computational framework is based on the following biological hypothesis. If a known AD variant is associated with a gene that is involved in a particular biological process (BP) (e.g. inflammation), we assume as a probabilistic prior that other AD variants might be associated with proteins involved in this BP or proteins that physically interact with this BP. This hypothesis can be tested computationally using a protein interaction network [1315] by extending the “guilt by association” principle via propagation of probabilistic evidence in a network [16, 17]. This general idea has similarity to the Google ranking algorithm of web pages, in which a web page that has a short link distance to many “important” pages will itself be considered “important”.

In the case of protein interactions, guilt by association-based inference is typically performed by inspecting the function of direct neighbors of a predicted disease gene in a protein-interaction network. This approach has been incorporated in multiple interpretation systems as well as commercially such as Ingenuity Pathway Analysis (IPA). However, it has been shown that network propagation, diffusion or other related methods that go beyond simple neighbor-based analysis can carry functional or disease associations further in the network with improved predictive accuracies [10, 11]. This idea extends to predicting both gene function and disease phenotypes associated with genes [11, 1822].

We hypothesize that this general framework, and network diffusion in particular, can be extended to aid prioritization of AD genes. Although the underlying biology of AD may be far more diverse than a single function, there are several biological pathways that are aberrantly activated in AD brains, and not surprisingly, most of the genes identified by AD GWAS contribute to these pathways [23]. For example, a primary indicator of AD is the accumulation of amyloid beta plaques in the brain, resulting from mis-processing of APP protein [23].

We developed a novel re-prioritization approach that can be integrated easily into the current genetic analysis design (Fig 1). First, we curated the AD literature to produce a set of approximately 60 robust AD (RAD) genes that includes those that have been associated with AD at the genome-wide significance level or that contain variants shown to affect AD-related processes directly (Table 1). We then constructed a network of protein-protein interactions and applied network diffusion to score and rank genes based on their proximity to the RAD genes. Network diffusion allows modeling of indirect interactions, modules and protein complexes that are not modeled if only the direct interactions of proteins are considered. Next, we combined our genetic association results with the network diffusion scores to produce a newly re-prioritized ranking of genes. Finally, we validated our methodology using a novel approach involving bootstrap aggregation on one of the largest assembled genetic datasets of AD. Network-augmented genetic results have measurably improved replication rates in this validation approach. We also show that our main results and key predictions were essentially unchanged after restricting the RAD set to 19 genes which have had been functionally validated as well as replicated in independent datasets.

thumbnail
Fig 1. Summary of analysis steps.

A set of AD genes that are reproducible (RAD genes) across different genetic studies was assembled through literature curation. The RAD genes were assigned a high initial risk score, and graph theoretical diffusion was employed to derive network diffusion scores for the rest of the genes in the network. Scores obtained from genetic screens and network diffusion were integrated to derive a new prioritization.

https://doi.org/10.1371/journal.pgen.1007306.g001

Results

RAD genes are proximal in a PPI network

We assembled a PPI network using interactions pooled from multiple PPI databases (ConsensusPathDB [13], iRefIndex [14], and Human Interactome Y2H [15]) inspired by recent work [21]. Pooling interactions from these three databases resulted in a connected network that includes a large percentage of the genes in our GWAS dataset. We then determined if the RAD genes are proximal within this network. The first proximity measure tested was the average shortest path (ASP) distance [50]. The ASP distance between RAD genes, determined by cross-validation (See Methods), is much smaller than would be expected by random chance (Table 2). One problem is that ASP distance between RAD genes and genes with many interactions (the number of interactions a gene has corresponds to its “degree” and high degree genes are considered to be hubs) tends to be small (Table 3). In this situation, all hub genes will be falsely predicted to be AD-related. Thus, we incorporated instead the Regularized Laplacian diffusion kernel [51] which penalizes paths going through hubs. The diffusion distance between RAD genes is smaller than would be expected by chance (p = 0.00054) (Table 2). Simultaneously, the problematic hub genes in the network have discounted scores as demonstrated by the notable drop in ranking of the 10 genes with the highest number of overall interactions (Table 3).

thumbnail
Table 2. Proximity between RAD genes in PPI network.

Each RAD gene was ranked (in comparison to the other 19,972 genes in the network) based upon its degree (number of interactions in network), its ASP distance to the RAD genes, and total diffusion distance from the RAD genes. The average ranking of the RAD genes was 7,949 using ASP (60th percentile, t-test p = 0.015) and 6,959 for diffusion (65th percentile, t-test p = 0.00054).

https://doi.org/10.1371/journal.pgen.1007306.t002

Filtering by network diffusion score improves replication rate

We next tested if genes with high diffusion scores replicate more frequently in order to demonstrate that diffusion scores are informative when used in conjunction with genetic data. Bootstrap aggregation [52] was applied to our genetic dataset to produce a large number of pairs of discovery and replication datasets (See Methods). In each discovery + replication pair, we conducted a standard genetic workflow, beginning with a screen in the discovery dataset followed by validating top findings in the replication dataset. For each pair, a replication rate was calculated by determining the percentage of genes that surpass a given significance threshold also replicated. To test if network diffusion scores improved replication, we altered the standard discover + replication approach. We ranked genes by their network diffusion score and then iteratively dropped genes that had ranking diffusion scores below a given stringency threshold. At first we retained only genes in the 50th percentile of network scores, then gradually increased the threshold to only include genes in the 60th, 70th, 80th, and 90th percentiles. For each threshold, we computed the replication rate and compared to the baseline. As shown in Fig 2, filtering based upon network score percentile noticeably increased replication rate. Genes with a–log(p-value) of > 6 replicated at a rate of approximately 16% in simulations (farthest right purple point), while additional strict network filtering improved the replication rate to nearly 34% (farthest right red point).

thumbnail
Fig 2. Filtering on network score improves replication rate.

The replication rate was computed for all genes surpassing the significance threshold for each GWAS. This procedure was repeated in each bootstrapped dataset and the average replication rate was determined (purple). This process was repeated using increasingly strict filters on the network diffusion scores. The baseline replication rate without utilizing network scores (naïve method) is represented by the purple points. The strictest network filter (red) has a consistently higher replication rate than the naïve method.

https://doi.org/10.1371/journal.pgen.1007306.g002

Combined Z-scores predict novel AD genes

Since filtering on network diffusion score improved replication rate, we next sought to integrate the network diffusion scores and genetic results into a single score. First, we converted the p-value of each gene from genetic analysis into a Z-score (“GWAS Z-Scores”) and then converted the network diffusion percentile of each gene into a Z-score (“Network Z-scores”). Linear regression analysis showed that the Network and GWAS Z-scores are independent (Fig 3A). Next, we assigned each gene a replication rate based upon how frequently the gene replicated in our bootstrapped validation datasets (See Methods). We observed that replication rates were higher for genes with higher network Z-scores compared to genes with lower network Z-scores (Fig 3B).

thumbnail
Fig 3. Comparison of GWAS and network Z-scores.

A. Transformed Z-scores are uncorrelated. B. Genes with high network scores had higher replication rates compared to those with low network scores, as further visualized and confirmed statistically as shown in Fig 4. Reprate = replication rate.

https://doi.org/10.1371/journal.pgen.1007306.g003

To combine the Network and GWAS Z-scores, we developed an approach that uses a linear support vector machine (SVM) [53] to determine how heavily each type of score should be weighted in order to maximize replication rate (See Methods). These weights were then used in conjunction with the meta-analysis method for combining summary results implemented in METAL [54]. The weights predicted by the SVM (Fig 4) were 0.703 (GWAS) and 0.297 (Network). As further confirmation, we conducted binomial (logit family) logistic regression using network and GWAS Z-scores as predictors and the replication class (high/low) as the outcome. Both network and GWAS score were significant, (GWAS: coefficient = -0.659, p <2.0×10−16) (Network: coefficient = -0.229, p = 0.0016). The coefficients derived from logistic regression are very similar to the SVM-derived weights (GWAS weight = 0.742, Network weight = 0.258).

thumbnail
Fig 4. Support vector machine training to predict GWAS and network Z-score weights.

Selection of genes with a high replication rate (> 0.7, blue points) and low replication rate (<0.1, red points) yielded a balanced number of genes in each replication class (high/low). A linear SVM model was trained to predict replication class using the GWAS and network Z-scores of each gene. Genes represented as X's were used as support vectors for the training of the SVM, whereas genes represented as O's were not. Both network and GWAS Z-scores contributed to the decision boundary, as demonstrated by the significance of their predicted coefficients using logistic regression (GWAS: p <2.0×10−16, Network: p = 0.0016).

https://doi.org/10.1371/journal.pgen.1007306.g004

Next, we applied our combined approach genome-wide, excluding the RAD genes and genes containing significantly associated variants (p <1.0x10-7) to focus on novel candidates. Among the genes with largest combined Z-scores (Table 4, S1 Table), several have important roles in inflammation. CR2 (p = 5.95×10−7) is a receptor protein involved in immune response (genecards.com [55]). SHARPIN (p = 1.43×10−5) is a component of the LUBAC complex that plays a regulatory role in inflammation [55]. PTPN2 (p = 3.21×10−5) is a phosphatase that also serves an important role in regulation of inflammation and glucose homeostasis [55]. The Bonferroni-corrected significance threshold when considering only genes in the 75th percentile of network scores is p = 1.46 x 10−5, although this is likely to be overly strict since proximally located genes are not inherited independently.

We performed pathway analysis using Gene Set Enrichment Analysis (GSEA) [56] to determine if AD-related pathways are more enriched when genes are ranked by their combined Z-scores versus GWAS-only Z-scores (See Methods). Notably, ranking genes based upon combined Z-scores resulted in several significantly enriched AD-related pathways including immune response, FOX03 targeting (indicates enrichment for aging), and hippocampal development (Table 5). By comparison, ranking genes based only upon their GWAS Z-scores resulted in virtually no significant pathways entirely (Table 6).

Discussion

GWAS of AD and AD-related endophenotypes have discovered and replicated associations with more than 60 genes (Table 1), many of which have roles in AD-related pathways (amyloid β aggregation, inflammation, cholesterol transport, immune response, etc.). To identify additional AD-related genes, we hypothesized that genes having suggestive evidence for association from a genome-wide screen and protein-level interactions (both direct and indirect) are more likely to replicate. This idea has been referred to as functional linkage [57]. To test this hypothesis, we developed a novel approach for improving the prioritization of candidate disease genes that incorporates a network diffusion of scores from known disease genes using a protein network and integration with GWAS risk scores. We tested this approach on a large AD GWAS dataset and validated the performance of the methodology using bootstrap aggregation. Several novel AD genes were predicted including CR2, SHARPIN, and PTPN2.

Part of the motivation for our approach was to identify genes that are more obviously biologically relevant to AD. This is exemplified by SHARPIN, whose principal known function is to form the LUBAC complex and prevent inflammation, a major process through which amyloid aggregation and AD are thought to develop [23]. Similarly, CR2, a homolog of CR1 which is a well-established AD gene [8], is involved in immune response. Many immune response genes are differentially expressed between healthy and AD brains, and investigations into the connection between expression in cell types and the presence of AD has led to growing interest in the role microglial cells (a first responder in the immune response pathway) [58]. Finally, PTPN2 is involved in multiple AD-related pathways; it has roles in negatively regulating inflammation and de-phosphorylation of key glucose metabolism kinases including INSR and EGFR [59]. The AD-related roles of each of our novel AD gene predictions, in combination with their strong network and genetic scores, make them highly promising candidates.

One biological form of functional linkage that does not require direct physical interaction is membership in the same signaling pathway or protein complex. For example, our study identified interaction between FOXO and INSR that is consistent with evidence of a multi-link signaling pathway comprised of direct physical interactions in the insulin-signaling pathway [60]. By comparison, neighborhood enrichment approaches (i.e., testing a gene’s direct interactions) cannot detect indirect interactions. Furthermore, neighborhood enrichment approaches are unreasonable for AD because some RAD genes are network hubs (e.g., APP has more than 2000 interactions) which would result in an unreasonably high number of genes having AD-enriched neighborhoods.

Some distance metrics capture indirect interactions by calculating the proximity between a pair of genes based upon short paths between them in the network. However, after testing a simple distance metric known as average shortest path (ASP), we observed that hub genes were still the top-ranked predicted genes. Since hub genes have many interactions, they tend to have short overall paths to any genes in a network, although their functions are highly generic and unlikely tied to a particular disease. Ubiquitin C (UBC), for example, has nearly 9,000 interactions; however, this is simply because protein degradation is essential for regulating the vast majority of proteins. Therefore, a more nuanced network propagation approach can aid in making disease specific inferences.

Network diffusion is a widely used class of spectral graph clustering methods that have been applied to many computational disciplines [51]. We used this approach to propagate evidence in the form of AD scores throughout the network. A protein in the network that has a short “diffusion distance” to one or more well-established AD genes will receive a high network risk score. Notably, we observed that network diffusion down-weights hubs while simultaneously outperforming ASP distance when applying leave-one-out cross-validation to the RAD genes. Many diffusion kernels have been proposed in graph theory, however the Regularized Laplacian [51] approach used in this study has the highly desirable properties of requiring very little parameterization (in fact, only a single parameter is required to be set) and also more computationally efficient than other diffusion kernels. Network diffusion methods have been applied in other genetics research contexts such as labeling somatic network mutations in cancer [61], characterizing gene sets [62], and predicting risk genes for amyotrophic lateral sclerosis [21].

We also observed that genes with high diffusion scores tended to replicate more frequently in our 125 pairs of bootstrapped discovery and replication datasets. However, network Z-scores and GWAS Z-scores in the full dataset were not strongly correlated. Taken together, these observations indicate the importance of considering jointly protein interaction data and genetic results even though they are independent because the integration of both types of information will likely yield noticeable improvement in replicability of findings. Since our bootstrapping procedure required splitting the original dataset, the simulations were conducted using datasets that contained only one-half of the total sample. This suggests that our network scores aided in determining which genetic associations were real in datasets with reduced power. We note that our bootstrapping approach was performed on the same data from which we derived the GWAS Z-scores used to train the SVM. Therefore, the selection of combination weights may have been biased in favor of GWAS Z-scores. Furthermore, it is unclear whether the weight combination used in this study (0.297/0.703) would be appropriate for combining genetic and network data for other disorders or traits.

The GWAS approach has a very limited capability to identify the entire set of genes which contribute to the risk of a complex disease like AD, even in datasets containing up to 100,000 individuals, because some genes do not contain variants that are sufficiently frequent and/or exert a large enough effect to yield a statistically significant association. To overcome this limitation, we developed a novel SVM approach to integrate the genetic and network scores by propagating GWAS Z-scores in a PPI network. In the AD example presented here, we initialized the RAD genes to have an identical high score in the network, thereby allowing re-prioritization of genes in any AD dataset regardless of the internal Z-scores of the RAD genes.

We acknowledge that our initial choice to treat each RAD gene equally may be controversial. Arguably, we could have seeded our analyses with GWAS Z-scores for each RAD gene from the original studies. However, our approach permits unbiased exploration of interactions of all plausible AD genes and does not require adjustment to these Z-scores for sample size or allele frequencies. Moreover, results derived from weighted RAD genes would be dominated by interactions with APOE for which the significance level exceeded a–log(p-value) of more than 100 in several datasets (compared to < 10 for most other RAD genes in the total group of datasets). Also, several key AD-related genes (e.g., APP, PSEN1 and PSEN2) which show little evidence for association with individual SNP or gene-based tests for AD would be undervalued in analyses using weighted Z-scores. In order to make our software maximally flexible and support weights derived from confidence in the seed genes, we implemented an option for users to specify unequal weights on the seed genes at their own discretion.

A potential concern about our results is the strategy for selecting RAD genes because many significant GWAS findings include variants located in intergenic regions. The most parsimonious explanation is that the variant responsible for the association peak influences the nearest gene, but there is abundant evidence suggesting this assumption is often incorrect. To address this issue, we repeated our analyses using a more restricted set of RAD genes that included only those supported by genome-wide significant evidence of association with AD risk and replication in independent datasets or by other genetic evidence plus experiments linking them to AD-related pathophysiology. Our leave-one-out cross validation approach demonstrated that the genes in the restricted RAD set had closer network proximity to each other than would be expected by chance (p = 5.93x10-5, S2 Table). The statistical support for the novel genes CR2 (p = 4.09x10-7), SHARPIN (p = 1.10x10-5), and PTPN2 (p = 2.41x10-5) remained the same (S3 Table). Finally, combined Z-scores that were derived using diffusion from the more conservative RAD gene set yielded similar AD-related pathways such as Fx03 targets (FWER p = 0.064), antigen processing (FWER p = 0.02), and hippocampal development (FWER p = 0.065) (S4 Table). These results confirm that the genes with a clear functional role in AD produce network diffusion-based predictions that are consistent with the results presented here. Curiously, the inclusion or exclusion of the portion of RAD genes that have an ambiguous or non- validated functional role in AD did not affect our results.

We also acknowledge that several of the novel putative AD genes may have been erroneously prioritized because they are in the same locus with RAD genes. This concern is unlikely noting that there are several instances where a genetic association peak includes multiple genes that may have a possible functional role in AD (e.g., the MS4A gene cluster [8]). Although one of our novel AD genes, CR2, is located close to CR1, which is an unambiguous RAD gene given its robust replication in GWAS and effect on deposition of neuritic amyloid plaque [63], CR2 is also an intriguing AD candidate gene because it has been shown to regulate hippocampal neurogenesis [63]. Thus, our findings suggest that our approach will aid in predicting truly multiple AD-related genes at a locus, however additional biological evidence may be required in some instances to make this distinction.

Previous AD studies have implicated inflammation and immune response genes, but we did not observe enrichment for these pathways when incorporating only GWAS scores in the analysis. However, these and other recognized AD-related pathways emerged after applying our network re-prioritization method (Table 6) suggesting that incorporation of network data can help minimize discrepancies in predictions across different genetic datasets. On the other hand, other well-established AD-related pathways, including cholesterol metabolism and endocytosis, were not detected by our approach. Further inspection of the results revealed, for example, that enrichment for the cholesterol homeostasis pathway is not significant when applying GSEA to the genetic data only (FWER p = 1). This pathway as defined in the Molecular Signatures Database (MSigDB) is very broad and contains many genes that are weakly associated with AD which consequently diminish the enrichment of the set. The evidence for this pathway is greater in the analysis using only network scores (FWER p = 0.18), which indicates our method still improves the detection of cholesterol homeostasis. Even pathways such as HDL-mediated lipid transport that were enriched in analyses considering only genetic data (largely due to the strong signal from APOE) were not ranked highly by our network diffusion algorithm because RAD genes such as APOE are ignored to minimize bias.

Although merging of multiple databases to obtain a very highly connected network is a requirement for the diffusion algorithm to work properly, our approach offers several advantages in comparison to other network-based approaches including biological transparency, ease of integration with a variety of GWAS methods, and the ability to balance data-driven statistics and biological prior probabilities. The extensive simulations we conducted provide a general basis for further establishing the practicality of genetic and network-based integration. Our network methodology was developed with the goal of accommodating known complications of genetic analysis.

The software developed for this study is open source, accessible to most users (incorporated in an R package), and applicable to any set of variant- or gene-level disease association results. Importantly, it requires only a set of GWAS results and a list of previously known disease genes and, therefore, does not necessitate changes to previously established genetic analysis pipelines. Although we used an SVM procedure to determine the weights for the score combination, a user can specify any weights or simply use our defaults that are based on the 0.297/0.703 ratio determined by SVM. Our package is accessible through GitHub (https://github.com/lancour/ignition).

Methods

Assembling an AD gene list

A set of genes ascribed to AD with a high degree of certainty was assembled through curation of published findings ascertained through PubMed searches that emerged from studies using a variety of approaches including GWAS of AD risk and AD-related endophenotypes, family-based linkage analysis, positional cloning, whole exome sequencing (WES), and candidate gene testing (CGS) (Table 1). Criteria for inclusion in this set included (1) genome-wide significance for GWAS and WES studies (p < 5x10-8) and LOD score > 3 for linkage studies and (2) replication of association signals in independent datasets; or (3) biological evidence that demonstrate functional relevance to AD of associated variants or the encoded protein.

Harmonizing protein-protein interaction databases

A set of interacting gene-gene pairs (in HGNC symbol format) is required as input for this software. To compile this set, three databases (RefIndex v14 [14], ConsensusPathDB v31 [13], and Human Interactome Y2H DB vHI-II-14 [15]) were selected based on their demonstrated utility in recent work [21]. iREFINDEX and ConsensusPathDB interactions were filtered to remove self and complex (more than two proteins) interactions. The ConsensusPathDB interactions are given in uniProt ID format, which were converted to HGNC symbols using the official website (http://www.genenames.org). iREFINDEX provides a HGNC symbol for each interactor of an interaction when possible, and so only interactions which had a HGNC for both interactors were kept. The Human Interactome DB already provides a set of binary gene-gene interactions in HGNC format, so no processing was required. The union of the processed sets from each database was used as the final interaction set. The unified set contains 19,972 unique gene symbols and 236,642 interactions. These databases are curated collections of experimentally determined interactions (typically binding or affinity) reported in the literature, such as from co-immunoprecipitation, as well as predicted interactions in a small number of databases.

Assigning network scores to genes through diffusion

Network diffusion is a very well-studied spectral approach to graph clustering and annotation [17, 51, 64, 65]. It attempts to mimic node-to-node distance in the graph that in turn aims to capture functional relevance. The first step of the diffusion method is to model the protein interactions as a network. A network is comprised of a set of nodes, V, and a set of edges between nodes, E. For this work, nodes represent genes, and edges represent an interaction present in the unified set. Although we use unweighted edges in this work, our network methods and software are able to receive weighted input as well, such as protein interactions with confidence measures taken from STRING [66]. The construction of diffusion kernels using weighted edges has been well studied and is equally valid [51]. n is the number of nodes in the network, which is 19,972 (yielding 236,642 edges). All network methods were implemented in R. The regularized Laplacian kernel [51] is constructed by: (1) where K is the resulting kernel, I is the identity matrix, L is the graph Laplacian, and alpha is a constant (see S1 Text and [51] for additional details). For this study, an alpha value of 0.1 was used, consistent with other work in this field [17]. Next, a network diffusion score was computed for each gene. To do this, the diffusion score vector, y, was initialized to be a length n vector that contains 1’s in the indices of the RAD genes, and 0’s otherwise. Risk scores for all genes in the graph were then derived by multiplication of K by the diffusion score vector y: ỹ = Ky.

Validation of diffusion approach using leave-one-out cross validation

To test if RAD genes had closer than random diffusion proximity to other RAD genes in a network, leave-one-out cross validation [67] was applied to the RAD gene set. First, a single RAD gene from the RAD set was set to 0 in the initial diffusion score vector, y. Then, diffusion scores were computed based upon this new initialization of y. The diffusion scores were sorted and the sorted rank of the removed RAD gene’s diffusion score was determined in comparison to all other non-RAD genes. This process was repeated for each gene in the RAD set, resulting in a list of ranks. If diffusion proximity is informative and potentially predictive, the average rank of the RAD genes should be significantly lower than the average rank of all genes, (n+1) / 2, which was verified using a one-tailed t-test.

ADGC GWAS dataset

The Alzheimer’s Disease Genetics Consortium (ADGC) is an NIA-funded project whose goal is to identify genes associated with an increased risk of developing late-onset Alzheimer disease (LOAD) by assembling and analyzing genetic and phenotypic data from large cohorts containing rigorously evaluated AD cases and cognitively normal controls of various ethnic ancestries. Details of ascertainment, collection, quality control (QC), and analysis of genotype and phenotype data in the individual datasets of the ADGC are provided elsewhere [8, 68]. Here we examined genotype data that were generated using high-density SNP microarrays from 32 prospective, case-control, and family-based studies of LOAD comprising 16,175 case and 17,176 controls of European ancestry. After QC steps to filter low-quality SNPs and individuals with low genotype call rates, principal components (PCs) of ancestry were computed within each dataset using EIGENSTRAT [69] and a set of 21,109 SNPs common to all genotyping platforms and datasets in order to account for population substructure in genetic association analysis. Samples with outlier PC values >six standard deviations from the mean were excluded from subsequent analyses. Genotypes for a much larger set of SNPs were imputed using the Haplotype Reference Consortium panel release 1.1 [70, 71], which includes 64,976 haplotypes derived from 39,235,157 SNPs, and the Michigan Imputation Server (https://imputationserver.sph.umich.edu/) running MiniMac3 [72, 73].

Genome-wide association analysis

Association of AD with the imputed dosage of the minor allele for each SNP (a quantitative estimate between 0 and 2) genome-wide was conducted using logistic regression models implemented in PLINK [74] that included covariates for age-at-onset/age-at-exam, sex, the first three PCs, and an indicator variable for each dataset. Joint analysis was chosen in favor of meta-analysis to avoid problems that could be introduced if bootstrap aggregation under-sampled small cohorts, resulting in unreliable association estimates for those cohorts. To account for relatedness in family datasets, subsets of maximally-unrelated affected and unaffected individuals were sampled from each pedigree. Each variant was annotated to a gene region according to RefSeq release 69 [75] using the program ANNOVAR [76]. Then, each gene was assigned the minimum p-value of all variants annotated to it, after applying the following formula: (2) where N is the number of variants analyzed that were annotated to the gene. Previously, this correction [77] has been shown to perform comparably to more complex adjustments based upon gene length, recombination hotspots, and similar gene features [78].

Validation of genetic re-prioritization through bootstrap aggregation

Since the availability of large AD genetic datasets is limited, bootstrap aggregation [52] was used to generate a high number of datasets for method validation. First, the full ADGC dataset was equally separated into discovery and replication halves. Then, 25 iterations of bootstrap aggregation were applied to the discovery half and then the replication half. The resultant 25 discovery and 25 replication datasets were then matched (D1 and R1, D2 and R2….D25 and R25). To further ensure robustness, the splitting procedure was repeated a total of 5 times, with 25 iterations of bootstrap aggregation applied each time, resulting in 125 total pairings (D1 and R1, D2 and R2. …D125 and R125). Each pairing represents a discovery dataset as well as an independent replication dataset.

For each pairing, the previously described genetic analysis was conducted on the discovery half. Then all genes that passed a designated significance threshold (the number of passing genes is denoted as r) were selected to be tested again in the replication half using a significance threshold of (0.05 / r). The replication rate was computed by determining the percentage of passing genes in the discovery half that also passed in the replication half. A replication rate was estimated for each pairing, and the mean replication rate was then determined. Next, the replication rate was re-determined for each pairing, with the added criterion that selected genes must also have a top percentile network diffusion score (top 10th, 20th, 30th, 40th, and 50th were tested). The average replication rate for each filtering threshold was compared to the average replication rate without filtering.

Integrating GWAS and network diffusion scores

The p-values from genetic analysis of the ADGC dataset were converted to Z-scores using the qnorm function in R. Then, the network diffusion scores were converted into percentiles. The percentiles are transformed into Z-scores using the qnorm function, with the additional specification of lower.tail = F. The weighting scheme from METAL was applied to combine the GWAS and network Z-scores: (3)

Although any weight selection can be used, the weights were “learned” using an SVM [53] due to the observation that the GWAS and network scores did not contribute equally to predicting replication rate. First, a replication rate was determined for each gene. If a gene had a p-value of <0.05 in d discovery datasets and a replication p-value of <0.05 in r of the paired replication datasets, it was assigned a replication rate of r/d. To reduce model overfitting, create sufficient separation between the classes, and achieve a balance of high and low replicating genes, only high replication genes (≥0.7, n = 676) and low replication genes (<0.1, n = 475) representing approximately 8.4% of the total genes with both a network and GWAS scores were extracted. By comparison, using a threshold of 0.8 or 0.9 would result in an imbalanced training set with very few high replication genes because highly replicating genes are uncommon. A linear SVM [53] was trained using the network Z-scores and the genetic association Z-scores as features, and “high” and “low” as the classes. The resulting slope of decision boundary was then used to determine appropriate weights (w1 = 0.703, w2 = 0.297).

Pathway analysis using the re-prioritized ordering of genes

Pathway enrichment was performed using the Gene Set Enrichment Analysis (GSEA) software [56]. GSEA’s pre-ranked analysis tool requires that the user provide a numeric measure for ordering genes. To establish a baseline, enrichment was done using our internal GWAS Z-scores to order genes. Then, enrichment was done using the alternative ordering genes based upon their combined Z-scores (see above for combination method). The gene sets tested for enrichment were the GSEA C2 pathways in MSigDb, which are the “curated gene sets” compiled from multiple sources including KEGG [60], Reactome [79], and domain experts. The significance threshold was set at FDR < 0.25, as suggested previously for this hypothesis generating approach [56].

Ethics statement

The use of de-identified human subject information for this study was approved by the Boston University Institutional Review Board.

Supporting information

S1 Table. Top predicted AD genes using combination approach.

https://doi.org/10.1371/journal.pgen.1007306.s001

(XLSX)

S2 Table. Leave-one-out cross validation rankings using the conservative RAD set.

https://doi.org/10.1371/journal.pgen.1007306.s002

(XLSX)

S3 Table. Top combined predictions using the conservative RAD gene list.

https://doi.org/10.1371/journal.pgen.1007306.s003

(XLSX)

S4 Table. GSEA results based on conservative RAD gene set after ranking genes by combined Z-scores.

https://doi.org/10.1371/journal.pgen.1007306.s004

(XLSX)

S1 Text. Regularized Laplacian construction.

https://doi.org/10.1371/journal.pgen.1007306.s005

(DOCX)

References

  1. 1. Lander ES. Initial impact of the sequencing of the human genome. Nature. 2011; 470:187–197. pmid:21307931
  2. 2. Hayes B. Overview of Statistical Methods for Genome-Wide Association Studies (GWAS). Methods Mol Biol. 2013; 1019:149–169. pmid:23756890
  3. 3. Alzheimer's Association. 2011 Alzheimer's disease facts and figures. Alzheimers Dement. 2011; 7:208–844. pmid:21414557
  4. 4. Goate A, Chartier-Harlin MC, Mullan M, Brown J, Crawford F, Fidani L, et al. Segregation of a missense mutation in the amyloid precursor protein gene with familial Alzheimer's disease. Nature. 1991; 349:704–706. pmid:1671712
  5. 5. Sherrington R, Rogaev EI, Liang Y, Rogaeva EA, Levesque G, Ikeda M, et al. Cloning of a gene bearing missense mutations in early-onset familial Alzheimer's disease. Nature. 1995; 375:754–60. pmid:7596406
  6. 6. Levy-Lahad E, Wasco W, Poorkaj P, Romano DM, Oshima J, Pettingell WH, et al. Candidate gene for the chromosome 1 familial Alzheimer's disease locus. Science. 1995; 269:973–977. pmid:7638622
  7. 7. Saunders AM, Strittmatter WJ, Schmechel D, George-Hyslop PH, Pericak-Vance MA, Joo SH, et al. Association of apolipoprotein E allele epsilon 4 with late-onset familial and sporadic Alzheimer's disease. Neurology. 1993; 43:1467–1472. pmid:8350998
  8. 8. Lambert JC, Ibrahim-Verbaas CA, Harold D, Naj AC, Sims R, Bellenguez C, et al. Meta-analysis of 74,046 individuals identifies 11 new susceptibility loci for Alzheimer's disease. Nat Genet. 2013; 45:1452–1458. pmid:24162737
  9. 9. Rogaeva E, Meng Y, Lee JH, Gu Y, Kawarai T, Zou F, et al. The neuronal sortilin-related receptor SORL1 is genetically associated with Alzheimer disease. Nat Genet. 2007; 39:168–177. pmid:17220890
  10. 10. Jia P, Zhao Z. Network.assisted analysis to prioritize GWAS results: principles, methods and perspectives. Hum Genet. 2014; 133:125–138. pmid:24122152
  11. 11. Lee I, Blom UM, Wang PI, Shim JE, Marcotte EM. Prioritizing candidate disease genes by network-based boosting of genome-wide association data. Genome Res. 2011; 21:1109–1121. pmid:21536720
  12. 12. Beyer A, Bandyopadhyay S, Ideker T. Integrating physical and genetic maps: from genomes to interaction networks. Nat Rev Genet. 2007; 8:699–710. pmid:17703239
  13. 13. Kamburov A, Pentchev K, Galicka H, Wierling C, Lehrach H, Herwig R. ConsensusPathDB: toward a more complete picture of cell biology. Nucleic Acids Res. 2011; 39(Database issue):D712–7. pmid:21071422
  14. 14. Razick S, Magklaras G, Donaldson IM. iRefIndex: a consolidated protein interaction database with provenance. BMC Bioinformatics. 2008; 9:405. pmid:18823568
  15. 15. Rolland T, Tasan M, Charloteaux B, Pevzner SJ, Zhong Q, Sahni N, et al. A proteome-scale map of the human interactome network. Cell. 2014; 159:1212–1226. pmid:25416956
  16. 16. Letovsky S, Kasif S. Predicting protein function from protein/protein interaction data: a probabilistic approach. Bioinformatics. 2003; 19(Suppl 1):i197–i204.
  17. 17. Kolaczyk ED. Statistical Analysis of Network Data: Methods and Models. Springer Ser Stat. 2009:1–386.
  18. 18. Liu M, Liberzon A, Kong SW, Lai WR, Park PJ, Kohane IS, et al. Network-based analysis of affected biological processes in type 2 diabetes models. Plos Genet. 2007; 3:e96. pmid:17571924
  19. 19. Chuang HY, Lee E, Liu YT, Lee D, Ideker T. Network-based classification of breast cancer metastasis. Mol Syst Biol. 2007; 3:140. pmid:17940530
  20. 20. Chen Y, Zhu J, Lum PY, Yang X, Pinto S, MacNeil DJ, et al. Variations in DNA elucidate molecular networks that cause disease. Nature. 2008; 452:429–435. pmid:18344982
  21. 21. Novarino G, Fenstermaker AG, Zaki MS, Hofree M, Silhavy JL, Heiberg AD, et al. Exome sequencing links corticospinal motor neuron disease to common neurodegenerative disorders. Science. 2014; 343:506–511. pmid:24482476
  22. 22. Greene CS, Krishnan A, Wong AK, Ricciotti E, Zelaya RA, Himmelstein DS, et al. Understanding multicellular function and disease with human tissue-specific networks. Nat Genet. 2015; 47:569–576. pmid:25915600
  23. 23. Hardy J. Amyloid, the presenilins and Alzheimer's disease. Trends Neurosci. 1997; 20:154–159. pmid:9106355
  24. 24. Jun G, Naj AC, Beecham GW, Wang LS, Buros J, Gallins PJ, et al. Meta-analysis confirms CR1, CLU, and PICALM as alzheimer disease risk loci and reveals interactions with APOE genotypes. Arch Neurol. 2010; 67:1473–1484. pmid:20697030
  25. 25. Pullabhatla V, Roberts AL, Lewis MJ, Mauro D, Morris DL, Odhams CA, et al. De novo mutations implicate novel genes in Systemic Lupus Erythematosus. Hum Mol Genet. 2018; 27:421–429. pmid:29177435
  26. 26. Scheuner D, Eckman C, Jensen M, Song X, Citron M, Suzuki N, et al. Secreted amyloid beta-protein similar to that in the senile plaques of Alzheimer's disease is increased in vivo by the presenilin 1 and 2 and APP mutations linked to familial Alzheimer's disease. Nat Med. 1996; 2:864–870. pmid:8705854
  27. 27. Hollingworth P, Harold D, Sims R, Gerrish A, Lambert JC, Carrasquillo MM, et al. Common variants at ABCA7, MS4A6A/MS4A4E, EPHA1, CD33 and CD2AP are associated with Alzheimer's disease. Nat Genet. 2011; 43:429–435. pmid:21460840
  28. 28. Mez J, Chung J, Jun G, Kriegel J, Bourlas AP, Sherva R, et al. Two novel loci, COBL and SLC10A2, for Alzheimer's disease in African Americans. Alzheimers Dement. 2017; 13:119–129. pmid:27770636
  29. 29. Jun G, Asai H, Zeldich E, Drapeau E, Chen C, Chung J, et al. PLXNA4 is associated with Alzheimer disease and modulates tau phosphorylation. Ann Neurol. 2014; 76:379–392. pmid:25043464
  30. 30. Rohn TT, Head E, Nesse WH, Cotman CW, Cribbs DH. Activation of caspase-8 in the Alzheimer's disease brain. Neurobiol Dis. 2001; 8:1006–1016. pmid:11741396
  31. 31. Beecham GW, Hamilton K, Naj AC, Martin ER, Huentelman M, Myers AJ, et al. Genome-wide association meta-analysis of neuropathologic features of Alzheimer's disease and related dementias. Plos Genet. 2014; 10(9):e1004606. pmid:25188341
  32. 32. Escott-Price V, Bellenguez C, Wang LS, Choi SH, Harold D, Jones L, et al. Gene-wide analysis detects two new susceptibility genes for Alzheimer's disease. PLoS One. 2014; 9(6):e94661. pmid:24922517
  33. 33. Otani Y, Yamaguchi Y, Sato Y, Furuichi T, Ikenaka K, Kitani H, et al. PLD$ is involved in phagocytosis of microglia: expression and localization changes of PLD4 are correlated with activation state of microglia. PLoS One. 2011; 6(11):e27544. pmid:22102906
  34. 34. Cruchaga C, Kauwe JS, Harari O, Jin SC, Cai Y, Karch CM, et al. GWAS of cerebrospinal fluid tau levels identifies risk variants for Alzheimer's disease. Neuron. 2013; 78:256–268. pmid:23562540
  35. 35. Chung J, Wang X, Maruyama T, Ma Y, Zhang X, Mez J, et al. Genome-wide association study of Alzheimer's disease endophenotypes at prediagnosis stages. Alzheimers Dement. 2017. Epub 2017/12/24. pmid:29274321
  36. 36. Jun GR, Chung J, Mez J, Barber R, Beecham GW, Bennett DA, et al. Transethnic genome-wide scan identifies novel Alzheimer's disease loci. Alzheimers Dement. 2017; 13:727–738. pmid:28183528
  37. 37. Ruiz A, Heilmann S, Becker T, Hernandez I, Wagner H, Thelen M, et al. Follow-up of loci from the International Genomics of Alzheimer's Disease Project identifies TRIP4 as a novel susceptibility gene. Transl Psychiatry. 2014; 4:e358. pmid:24495969
  38. 38. Wetzel-Smith MK, Hunkapiller J, Bhangale TR, Srinivasan K, Maloney JA, Atwal JK, et al. A rare mutation in UNC5C predisposes to late-onset Alzheimer's disease and increases neuronal cell death. Nat Med. 2014; 20:1452–1457. pmid:25419706
  39. 39. Hoekstra EJ, Mesman S, de Munnik WA, Smidt MP. LMX1B is part of a transcriptional complex with PSPC1 and PSF. PLoS One. 2013; 8(1):e53122. pmid:23308148
  40. 40. Sims R, van der Lee SJ, Naj AC, Bellenguez C, Badarinarayan N, Jakobsdottir J, et al. Rare coding variants in PLCG2, ABI3, and TREM2 implicate microglial-mediated innate immunity in Alzheimer's disease. Nat Genet. 2017; 49:1373–1384. pmid:28714976
  41. 41. Jun G, Ibrahim-Verbaas CA, Vronskaya M, Lambert JC, Chung J, Naj AC, et al. A novel Alzheimer disease locus located near the gene encoding tau protein. Mol Psychiatry. 2016; 21:108–117. pmid:25778476
  42. 42. Narain Y, Yip A, Murphy T, Brayne C, Easton D, Evans JG, et al. The ACE gene and Alzheimer's disease susceptibility. J Med Genet. 2000; 37:695–697. pmid:10978362
  43. 43. Guerreiro R, Wojtas A, Bras J, Carrasquillo M, Rogaeva E, Majounie E, et al. TREM2 variants in Alzheimer's disease. N Engl J Med. 2013; 368:117–127. pmid:23150934
  44. 44. Huang KL, Marcora E, Pimenova AA, Di Narzo AF, Kapoor M, Jin SC, et al. A common haplotype lowers PU.1 expression in myeloid cells and delays onset of Alzheimer's disease. Nat Neurosci. 2017; 20:1052–1061. pmid:28628103
  45. 45. Cruchaga C, Karch CM, Jin SC, Benitez BA, Cai Y, Guerreiro R, et al. Rare coding variants in the phospholipase D3 gene confer risk for Alzheimer's disease. Nature. 2014; 505:550–554. pmid:24336208
  46. 46. Farrer LA, Cupples LA, Haines JL, Hyman B, Kukull WA, Mayeux R, et al. Effects of age, sex, and ethnicity on the association between apolipoprotein E genotype and Alzheimer disease. A meta-analysis. APOE and Alzheimer Disease Meta Analysis Consortium. JAMA. 1997; 278:1349–1356. pmid:9343467
  47. 47. Sharma M, Maraganore DM, Ioannidis JP, Riess O, Aasly JO, Annesi G, et al. Role of sepiapterin reductase gene at the PARK3 locus in Parkinson's disease. Neurobiol Aging. 2011; 32:2108 e1-5.
  48. 48. Logue MW, Schu M, Vardarajan BN, Farrell J, Bennett DA, Buxbaum JD, et al. Two rare AKAP9 variants are associated with Alzheimer's disease in African Americans. Alzheimers Dement. 2014; 10:609–618. pmid:25172201
  49. 49. Karch CM, Ezerskiy LA, Bertelsen S, Alzheimer's Disease Genetics Consortium, Goate AM. Alzheimer's disease risk polymorphisms regulate gene expression in the ZCWPW1 and the CELF1 loci. PLoS One. 2016; 11(2):e0148717. pmid:26919393
  50. 50. Dreyfus SE. An Appraisal of Some Shortest-Path Algorithms. Oper Res. 1969; 17:395–412.
  51. 51. Smola AJ, Kondor R. Kernels and regularization on graphs. Lect Notes Artif Int. 2003; 2777:144–158.
  52. 52. Efron B, Tibshirani R. Improvements on cross-validation: The 632+ bootstrap method. J Am Stat Assoc. 1997; 92:548–560.
  53. 53. Cortes C, Vapnik V. Support-Vector Networks. Mach Learn. 1995; 20:273–297.
  54. 54. Willer CJ, Li Y, Abecasis GR. METAL: fast and efficient meta-analysis of genomewide association scans. Bioinformatics. 2010; 26:2190–2191. pmid:20616382
  55. 55. Rebhan M, Chalifa-Caspi V, Prilusky J, Lancet D. GeneCards: integrating information about genes, proteins and diseases. Trends Genet. 1997; 13:163. pmid:9097728
  56. 56. Subramanian A, Tamayo P, Mootha VK, Mukherjee S, Ebert BL, Gillette MA, et al. Gene set enrichment analysis: A knowledge-based approach for interpreting genome-wide expression profiles. Proc Natl Acad Sci USA. 2005; 102:15545–15550. pmid:16199517
  57. 57. Karaoz U, Murali TM, Letovsky S, Zheng Y, Ding CM, Cantor CR, et al. Whole-genome annotation by using evidence integration in functional-linkage networks. Proc Natl Acad Sci USA. 2004; 101:2888–28893. pmid:14981259
  58. 58. Solito E, Sastre M. Microglia function in Alzheimer's disease. Front Pharmacol. 2012; 3:14. pmid:22363284
  59. 59. de la Monte SM, Wands JR. Alzheimer's disease is type 3 diabetes-evidence reviewed. J Diabetes Sci Technol. 2008; 2:1101–1113. pmid:19885299
  60. 60. Kanehisa M, Goto S. KEGG: Kyoto Encyclopedia of Genes and Genomes. Nucleic Acids Research. 2000; 28:27–30. pmid:10592173
  61. 61. Leiserson MD, Vandin F, Wu HT, Dobson JR, Eldridge JV, Thomas JL, et al. Pan-cancer network analysis identifies combinations of rare somatic mutations across pathways and protein complexes. Nat Genet. 2015; 47:106–114. pmid:25501392
  62. 62. Blatti C, Sinha S. Characterizing gene sets using discriminative random walks with restart on heterogeneous biological networks. Bioinformatics. 2016; 32:2167–2175. pmid:27153592
  63. 63. Nanji AA, Zhao S, Lamb RG, Dannenberg AJ, Sadrzadeh SM, Waxman DJ. Changes in cytochromes P-450, 2E1, 2B1, and 4A, and phospholipases A and C in the intragastric feeding rat model for alcoholic liver disease: relationship to dietary fats and pathologic liver injury. Alcohol Clin Exp Res. 1994; 18:902–908. pmid:7978103
  64. 64. Fouss F, Francoisse K, Yen L, Pirotte A, Saerens M. An experimental investigation of kernels on graphs for collaborative recommendation and semisupervised classification. Neural Netw. 2012; 31:53–72. pmid:22497802
  65. 65. Kondor R, Lafferty J. Diffusion Kernels on Graphs and Other Discrete Input Spaces. In: University CM, editor. 2002.
  66. 66. Szklarczyk D, Franceschini A, Wyder S, Forslund K, Heller D, Huerta-Cepas J, et al. STRING v10: protein-protein interaction networks, integrated over the tree of life. Nucleic Acids Res. 2015; 43(Database issue):D447–D452. pmid:25352553
  67. 67. Stone M. Cross-Validatory Choice and Assessment of Statistical Predictions. J R Stat Soc B. 1974; 36:111–147.
  68. 68. Naj AC, Jun G, Beecham GW, Wang LS, Vardarajan BN, Buros J, et al. Common variants at MS4A4/MS4A6E, CD2AP, CD33 and EPHA1 are associated with late-onset Alzheimer's disease. Nat Genetics. 2011; 43:436–441. pmid:21460841
  69. 69. Patterson N, Price AL, Reich D. Population structure and eigenanalysis. Plos Genet. 2006; 2:2074–2093.
  70. 70. Das S, Forer L, Schonherr S, Sidore C, Locke AE, Kwong A, et al. Next-generation genotype imputation service and methods. Nat Genetics. 2016; 48:1284–1287. pmid:27571263
  71. 71. McCarthy S, Das S, Kretzschmar W, Delaneau O, Wood AR, Teumer A, et al. A reference panel of 64,976 haplotypes for genotype imputation. Nat Genetics. 2016; 48:1279–1283. pmid:27548312
  72. 72. Howie B, Fuchsberger C, Stephens M, Marchini J, Abecasis GR. Fast and accurate genotype imputation in genome-wide association studies through pre-phasing. Nat Genetics. 2012; 44:955–959. pmid:22820512
  73. 73. Fuchsberger C, Abecasis GR, Hinds DA. minimac2: faster genotype imputation. Bioinformatics. 2015; 31:782–784. pmid:25338720
  74. 74. Purcell S, Neale B, Todd-Brown K, Thomas L, Ferreira MAR, Bender D, et al. PLINK: A tool set for whole-genome association and population-based linkage analyses. Am J Hum Genet. 2007; 81:559–575. pmid:17701901
  75. 75. Pruitt KD, Brown GR, Hiatt SM, Thibaud-Nissen F, Astashyn A, Ermolaeva O, et al. RefSeq: an update on mammalian reference sequences. Nucleic Acids Res. 2014; 42(D1):D756–D63.
  76. 76. Wang K, Li MY, Hakonarson H. ANNOVAR: functional annotation of genetic variants from high-throughput sequencing data. Nucleic Acids Res. 2010; 38(16):e164. pmid:20601685
  77. 77. Saccone SF, Hinrichs AL, Saccone NL, Chase GA, Konvicka K, Madden PA, et al. Cholinergic nicotinic receptor genes implicated in a nicotine dependence association study targeting 348 candidate genes with 3713 SNPs. Hum Mol Genet. 2007; 16:36–49. pmid:17135278
  78. 78. Segre AV, Groop L, Mootha VK, Daly MJ, Altshuler D, Consortium D, et al. Common Inherited Variation in Mitochondrial Genes Is Not Enriched for Associations with Type 2 Diabetes or Related Glycemic Traits. Plos Genet. 2010; 6(8).
  79. 79. Fabregat A, Sidiropoulos K, Garapati P, Gillespie M, Hausmann K, Haw R, et al. The Reactome pathway Knowledgebase. Nucleic Acids Res. 2016; 44:D481–487. pmid:26656494