Extraction of article metadata.

ENQUIRE uses the NCBI’s e-utilities to query and fetch information from the PubMed database [70]. Epost is used to request a collection of PMIDs, efetch to extract their metadata in XML format, and esearch to construct PubMed queries.

MeSH term extraction.

For each MEDLINE-indexed, input PMID, if the MeSH entity scope is selected, ENQUIRE retrieves MeSH main headings (“descriptors”) and subheadings (“qualifiers”) from their respective efetch-retrieved XML files. These MeSH terms are further selected to match biomedically relevant, non-redundant categories, by exploiting the tree-like, hierarchical structure of the MeSH vocabulary. By default, ENQUIRE only retains members downstream of the MeSH categories A (Anatomy), C (Diseases), D (Chemicals and Drugs), and G (Phenomena and Processes), except for sub-categories G01 (Physical Phenomena), G02 (Chemical Phenomena) and G17 (Mathematical Concepts).

Gene normalization from article abstracts.

For each input PMID, if the gene entity scope is selected, ENQUIRE retrieves article abstracts from their respective efetch-retrieved XML files. Additionally, the “Other Term” fields containing non-MEDLINE-indexed keywords are also efetch-retrieved and concatenated with the corresponding abstracts. Since other authors have shown that the proportion of gene mentions does not significantly differ between abstracts and full-body texts [71], we only mine the abstracts for gene mentions. In contrast to standard named entity recognition of genes (NER), whose task is to exactly match the character span of a gene mention, ENQUIRE’s text mining framework aims at detecting at least one gene alias per unique reference gene mentioned in an abstract. We therefore designed a “Swiss cheese model” for gene normalization, in which multiple methods complement each other to improve the global precision. In brief, ENQUIRE applies up to two algorithms to each unprocessed abstract: i) the Schwartz-Hearst algorithm to detect single-word abbreviations and their respective definitions [45]; ii) the optional scispaCy model (en_ner_jnlpba_md) to identify words classified as “CELL_LINE” or “CELL_TYPE” [46]. This allows ENQUIRE to construct abstract-specific lists of blocked terms that discard i) ambiguous abbreviations whose definitions are not similar to any gene alias from a pre-annotated lookup table, and ii) ambiguous or unwanted mentions to cell entities containing gene aliases, such as “CD8+ T cell”. Finally, a tokenization module generates potential gene-alias-matching tokens and redirects them to a unique, reference gene symbol using the lookup table.

Construction of the lookup table of reference gene names and respective aliases.

Similar to previous approaches [72], ENQUIRE performs NER of Homo sapiens and Mus musculus gene mentions, while also redirecting the latter to their respective human homologues using MGI’s mouse/human orthology table [73]. Each reference gene name corresponds to a HGNC-approved symbol [74]. Additional mouse and human gene aliases were pooled from HGNC (“previous symbols”, “previous names”, “alias symbols”, “alias names”), ENSEMBL (“gene stable ID”, “gene description”, “gene name”), Uniprot (“gene names”, “protein names”), and miRBase (“ID”, “alias”, “name”) [75–77]. We manually inspected sources of ambiguities and lack of spelling variants: for example, we added miRNA names without species suffixes (e.g. “miR-335” from “hsa-miR-335”), multiple spellings for lnc- and mi-RNAs (e.g. “LNC/Lnc/lnc”, “miR/mir”) and removed aliases identical to common acronyms for experimental techniques (e.g. “MRI”, “NMR”, “TEM”). We transcribed Greek letters to their names in the Latin script. We resolved ambiguities due to aliases reported under more than one reference symbol, by either assigning the alias to a single reference, or by excluding the alias.

Abstract tokenization for named-entity recognition of genes.

ENQUIRE mostly performs named-entity recognition of genes (NER) from article abstracts by exact matches between gene aliases and space- or punctuation-separated word tokens. We exclude general-purpose English words annotated in the English-words Python library to reduce the computational burden of mapping gene mentions. Like for the lookup table, we transcribe Greek letters to their names in the Latin script. Special attention is put to hyphen- and slash-containing tokens, tracing their usage as integral parts of gene aliases (e.g. “TNF-alpha”) or separators (e.g. “FcγR-TLR Cross-Talk” – PMID 31024565, “Akt/PI3K/mTOR signaling pathway” – PMID 35802302). When cases of the latter kind occur, the algorithm requires all hyphen- or slash-separated words to be gene aliases, in order to be considered individual tokens. Then, ENQUIRE tokenizes the abstract into single-word tokens and interprets unambiguous tokens as the corresponding reference gene symbol if they match an alias in the lookup table. Multiple mentions of the same gene within an abstract count as one.

Abstract-specific lists of blocked terms using cell entity mentions and abbreviation-definition pairs.

Any token that exactly matches an alias from the lookup table is redirected to the respective reference symbol, except when either the scispaCy en_ner_jnlpba_md or Schwartz-Hearst models classify that same token as part of “CELL_LINE” or “CELL_TYPE” entities, or as an abbreviation. In the former exception, the token is added to a list of blocked terms and any of its mentions within the abstract text are excluded from further gene normalization steps. In the latter exception, we evaluate the validity of an alias-matching abbreviation by means of its definition, as inferred by Schwartz-Hearst. We perform string comparison to calculate alignment scores between the definition and any recorded alias of the same reference symbol matched by the abbreviation. To this end, we implemented the Needleman-Wunsch algorithm for global alignment, with match score equal to 1, gap opening and mismatch penalties equal to −1, and gap extension penalty equal to −0.5 [78]. Next, we calibrated a threshold for either retaining or discarding an alias-matching abbreviation according to its optimal alignment score. We used a dataset of abbreviation-description pairs from more than 300 abstracts and generated a distribution of scores by aligning any description to any annotated alias. Intuitively, there could only be a handful of alignments between an actual gene description and the aliases referring to that same gene, as opposed to several alignments between that same description and unrelated aliases. Therefore, we treated the above-derived distribution as a model describing false positive alignments between descriptions and gene aliases. Finally, we identified a range between 0.1 and 0.2 that respectively correspond to 95th and 99th percentiles of the distribution of alignment scores as a sensible interval for choosing the threshold. We opted for a threshold of 0.15. Therefore, for any description whose abbreviation matches a gene alias, ENQUIRE records a gene mention only if the maximal alignment score against any alias of that same gene is higher or equal to this threshold; else, the abbreviation is added to the blocklist and all of its mentions within the text are excluded. Notice that the blocklist is independently computed for each abstract, thus making ENQUIRE’s gene normalization moderately adaptive with respect to syntactical context.

Annotation of co-occurrences.

ENQUIRE records the occurrences of MeSH and gene entities within each input article. Then, it converts the recorded co-occurrences into an undirected multi-graph, where gene or MeSH terms become nodes, and each recorded co-occurrence between two entities becomes an edge. Thus, the network has as many nodes as the number of unique MeSH and gene symbols, with as many edges between two nodes as the number of PMIDs in which they co-occur.

Reconstruction of a weighted network of significant co-occurrences.

ENQUIRE implements the soft-configuration model of Casiraghi and Nanumyan applied to undirected, unweighted edge counts to select significant co-occurrences among entities, adjusted to 1% FDR [31]. The test statistics follows a multivariate hypergeometric distribution, under the null hypothesis of observing a random graph whose expected degree sequence correspond to the observed one. This allows us to condition the testing to the sheer, per-entity occurrence, which serves as a proxy for leveraging this kind of literature bias in the corpus. It is important to note that the null model does not assume independence of individual edges, but merely their equiprobability. This selection results in an undirected, single node-to-node edge co-occurrence graph (i.e., a simple graph).

After generating the simple graph, ENQUIRE counts significant pairwise co-occurrences by enumerating the subset of PMIDs associated to both entities in each pair. For each pair of entities g_i and g_j that co-occur in at least one article, we define the weights w and distances w̃ accounting for the sheer co-occurrence X(g_i,g_j) as follows:

Where X̅ is the mean co-occurrence between any two entities in the corpus, Ψ(·,X̅) is the zero-truncated, Poisson cumulative density function with a lambda of X̅, and E^P is the set of all entities annotated within the PMID P that belongs to the submitted PMIDS corpus. This scoring system assigns higher relevance to co-occurrences that appear more often than average. Notice that w̃ shall not be interpreted as a strict mathematical distance, but rather as a measure of closeness between connected nodes. For each pair of adjacent entities g_i and g_j in the simple network, we assign the weights w_gi,gj and distances to their mutual edge. Additionally, we prune poorly connected nodes by modularity-based, w-weighted Leiden clustering [79] and removal of communities that consist of a single node. From the resulting gene/MeSH heterogeneous network, we also extract the respective gene- and MeSH-only subnetworks.

ENQUIRE-generated gene/MeSH networks can consist of multiple connected components, i.e., subgraphs. To exclude unimportant components, a subgraph S is retained for subsequent computations only if the fraction of corpus articles covered by S is higher than a threshold value, as formally defined in

where P denotes a PMID belonging to PMIDS, and E^P and E^S refer to the sets of gene or MeSH entities recorded in either P or S. Therefore, T_S reflects the representativeness of S with respect to the entirety of the submitted corpus. The value of t can be set by the user. To avoid introducing irrelevant entities, ENQUIRE stops without further network expansion if the gene/MeSH network and the respective gene- and MeSH-only subnetworks individually contain only a single, connected component with T_S ≥ t.

Finally, we compute the weight of a node g in the connected graph S utilizing the composite function W, which is the product of normalized metrics for betweenness centrality (b) and w-weighted degree strength (d):

Here, F_x denotes the empirical cumulative density function for the corresponding x parameter, calculated over S.

Construction of communities from “information-dense” cliques.

To identify the most relevant parts of the gene/MeSH network, ENQUIRE first identifies the maximal cliques of order three or more. By definition, these are graphlets whose nodes are all adjacent to each other and not a subset of a larger clique. Applying the KNet function from the SANTA R package [52] to the gene/MeSH network having distances w̃_gi,gj, we select cliques that form significant clusters of associated entities. The permutation test procedure internal to KNet allows us to consider the network topology and adjust each maximal clique’s significance, in case many other cliques of similar size exist in the network. We set the significance level for this test to 1% FDR. Subsequently, ENQUIRE generates a pruned network C containing only statistically significant cliques. Here, ENQUIRE stops if the gene/MeSH network contains less than two significant cliques according to KNet. Next, ENQUIRE identifies communities in the C network using modularity-based, w-weighted Leiden clustering. ENQUIRE stops if it detects a single community that encompasses all nodes in C.

Identification of community-connecting entities.

For any two distinct communities C_i and C_j, and for a given k, a parameter set by the user, we can define the set of community-connecting, weighted graphlets Γ_Ci,Cj(V_k, L_k-1) whose elements Γⁿ_Ci,Cj(Vⁿ_k, Lⁿ_k-1) must satisfy the following properties: i) all nodes g_i in the k-sized set Vⁿ_k belong to either C_i or C_j; ii) both intersections between Vⁿ_k and C_i or C_j are non-empty; iii) the w-weighted, k-1 edges Lⁿ_k-1 are sufficient to obtain a single connected component; iv) there is only one edge l_gi,gj that connects nodes belonging to distinct communities.

For a given graphlet Γⁿ_Ci,Cj(Vⁿ_k, Lⁿ_k-1), we can define the ranking function R:

This allows us to rank each graphlet inside the set of community-connecting, weighted graphlets Γ_Ci,Cj(V_k, L_k-1) and, consequently, also rank the set of community-connecting entities Vⁿ_k in any graphlet Γⁿ_Ci,Cj. The smaller the R, the closer two communities connected by Vⁿ_k are.

Retrieval of new PMIDs via PubMed queries based on optimal connections.

To evaluate which genes and MeSH terms are particularly suited for querying, ENQUIRE constructs a multigraph M where network communities become nodes and graphlet connections between two communities become edges, such that the value of their weights corresponds to the value of the R function applied to the corresponding graphlet Γⁿ_Ci,Cj. Edges whose weights ρ_Ci,Cj do not fulfil the triangle inequality

are excluded. Then, we solve the travelling salesman problem (TSP) utilizing Christofides’ approximate solution as implemented in the Python package Networkx [80]. Through the visited edges, this yields an optimal path across communities and a corresponding collection of Vⁿ_k entity sets. Each selected k-sized set Vⁿ_k results in a PubMed query formulated via the NCBI’s esearch utility [70]. We condition the search terms representing gene aliases and MeSH with “[Title/Abstract]” and “[MeSH Terms]”, respectively, and exclude review articles from the results. By design, the “[Title/Abstract]” field also includes the “Other Term” field when used in a PubMed query. The constructed PubMed queries require a match for all the k entities in the optimal path – e.g., “melanoma/immunology”[MeSH Terms] AND (“IL1B”[Title/Abstract] OR “interleukin 1-beta”[Title/Abstract] […]) AND […]. If all queries involving a subset of the network communities lead to empty results, we prune all previously used edges from M, compute a new TSP solution, and submit newly generated queries, provided at least one entity per query belongs to such community subset. This process is repeated A times, where A is a parameter specified by the user. If at least one new PMID matches any of the constructed queries, ENQUIRE starts a new analysis from the union of new and old PMIDs; otherwise, it stops. The rationale behind merging old and new PMIDs is to account for the original corpus when computing the statistics on new co-occurrences.

Context-aware gene sets.

To reconstruct contextual gene sets using gene/MeSH co-occurrence networks, we adapt network-based relational data to the method described by Khan and collaborators [81]. To this end, we first construct the inverse log-weighted similarity matrix between the gene/MeSH network nodes [82]. This metric prioritizes nodes sharing many lower degree neighbors rather than few higher degree ones. To transform this similarity matrix measure into a distance matrix, we multiply it by negative one and exponentiate it element-wise, hence obtaining smaller values the higher the similarity. After further applying a Z-score standardization, we use the R package DynamicTreeCut and Ward’s clustering to identify initial clusters and create an initial membership degree matrix [83,84]. Finally, we detect fuzzy clusters of genes and MeSH terms by applying Fuzzy C-means clustering to the Euclidean distance matrix, using the R package ppclust [85]. To speed up the computation, we also offer an alternative computation of fuzzy clusters using only the first N PCA-derived components explaining v or more of the overall variance, where v is a parameter chosen by the user. The resulting membership degree matrix allows annotating genes with desired cluster membership degrees and extracting the linked MeSH terms to characterize the gene set. The script implementing context-aware gene set reconstruction also offers ORA on the resulting gene clusters using Reactome pathways, restricting the universe to the complete set of ENQUIRE-derived genes.

Context-aware pathway enrichment analysis.

We designed a method to map any text-mined co-occurrence network G onto a mechanistic reference network N and infer context-specific enrichment of molecular pathways. With this strategy, we attempt to mechanistically explain the indirect relationships that constitute the co-occurrence network. To this end, we define the fitness score Q for every gene g in N with non-zero node degree d(g,N):

Here, and are the w̃-weighted and unweighted distances from g_i to g_j in the graphs G and   N, respectively. The indicator function □ implies that non-text-mined genes without at least two text-mined nodes as neighbors have Q equal to zero. We normalize all scores to decorrelate Q from the node degree d(g,N), similarly to other approaches in network propagation [86,87]. See S5 Fig for an example of Q score weighting. As a mechanistic reference network, we chose STRING’s (release 11.5) H. sapiens network of protein-coding, physically interacting genes [50]. We exclusively combined the “experimental” and “database” channels to calculate STRING’s confidence score, and then pruned all edges with score below the 90^th percentile. After removing zero-degree nodes, we obtain a reference, unweighted network of 9,482 nodes and 88,333 edges. Then, we calculate Q scores for protein-coding genes in the STRING reference network (N), using the ENQUIRE-generated gene network (G). We test for associations between predefined gene sets and high-scoring node clusters using SANTA’s KNet function [52]. KNet takes as input the STRING reference network, its nodes’ Q scores, and a gene set; it then tests if the latter is enriched, based on scores and graph distances of protein-coding genes belonging to both the network and the gene set. This way, we aim at capturing known experimentally or database-derived molecular interactions relevant to ENQUIRE’s input literature corpus, using topology-based enrichment analysis. We test for enrichment on gene sets derived from Reactome pathways, obtained via the Reactome Graph database [49]. The offered software implementation can also return the Q-score-weighted STRING reference network in GraphML format.

Assessment of ENQUIRE’s gene normalization accuracy and performance.

We evaluated ENQUIRE’s gene normalization precision and recall using abstracts from the NLM-Gene corpus mentioning at least one M. musculus or H. sapiens gene – 479 out of 550 entries [44]. We tested the four module combinations obtained by either including or excluding the cell entity recognition module en_ner_jnlpba_md and the Schwartz-Hearst abbreviation-definition algorithm [45,46]. We compared the computational performance of ENQUIRE’s gene normalization method using both en_ner_jnlpba_md and Schwartz-Hearst against GNorm2 implementation of Bioformer [47,48]. We computed wall time by accounting for both text processing and loading of required data such as gene alias lookup tables and machine learning models. RAM usage was measured using resident set size (RSS) measurements returned by the Linux built-in function ps. We ran the computations on a Linux computer with 20 CPUs (3.1 GHz) and 252 GB of RAM. Up to 8 cores were used for parallelization.

Inference of reactome gene sets from reference literature.

We extracted annotated genes and reference literature for all H. sapiens Reactome pathways from the Reactome Graph database [49]. We employed NCBI’s esearch and elink utilities to retrieve primary research articles cited by review articles [70]. After excluding pathways with less than three primary literature references or only one annotated human gene, we obtained a set of 967 pathways. For each pathway literature corpus, ENQUIRE performed one network reconstruction, set to only extract gene mentions from article abstracts. We evaluated the effects of corpus size, pathway size, and average gene-gene co-occurrence per abstract on precision and recall of ENQUIRE’s gene normalization and network reconstruction. We also evaluated the correlation between true positives and the corpus- and network-based node weight W.

Estimate of molecular interrelations.

We automatically generated a list of case studies by crossing leaf nodes downstream of Diseases and Genetic Phenomena (G05) MeSH categories. We then constructed a PubMed query from each pair by “AND” concatenation. Examples of such queries are “Stomach Neoplasm”[MeSH Terms] AND “Chromosomes, human, pair 18”[MeSH Terms], and “Acquired immunodeficiency syndrome”[MeSH Terms] AND “Polymorphism, single nucleotide”[MeSH Terms]. For each query result with a size between 50 and 500 articles, we executed one network reconstruction. If obtaining a gene-gene co-occurrence network, we investigated whether its set of genes produced a network with more functional interactions than expected by chance. To obtain background distributions of edge counts for each gene set size observed with ENQUIRE, we sampled one million random gene sets and cumulated their interconnecting edges in STRING’s v. 11.5 H. sapiens functional protein network [50]. We only included functional associations from experiments, co-expression, and third-party databases with a cumulative score higher than 0.7 between proteins. The significance of each ENQUIRE-generated gene set’s edge count was computed from the right-tailed probability of the empirical distribution.

Moreover, we compared the ENQUIRE-generated gene-gene wirings to STRING-derived associations using the DeltaCon similarity measure in a permutation test [51]. To this end, we generated 10,000 random graphs for each observed ENQUIRE network. Each random graph was obtained through 300 random edge-swapping attempts while preserving the degree sequence of the original network. To obtain sensible probability densities, we focused on ENQUIRE-generated networks with degree sequences allowing at least ten different realizations of a graph. We followed the formula Πⁿ_i d_i!, where d_i is the degree of the i-th node of a graph containing n nodes.

Assessment of context resolution by topology-based enrichment of molecular pathways.

To show that ENQUIRE preserves context-specific molecular signatures, we designed a broad panel of case studies (Table 6). Each corpus consisted of the union of references contained in three independent reviews accessible via NCBI’s elink utility [70]. We selected reviews from the results of PubMed search queries consisting of two or three MeSH terms (e.g., “Melanoma”[MeSH Terms] AND “Signal Transduction”[MeSH Terms]), favoring PubMed-ranked best matches when possible. We also included an unspecific positive control group consisting of the union of all context-specific corpora. This experimental design allowed us to construct a reference dendrogram that clusters the case studies only based on baseline biological knowledge, expecting expanded networks of a case study to cluster together with the originally reconstructed one. Then, we applied ENQUIRE with default parameters to each case study and analyzed all resulting gene-gene networks, i.e., from original and expanded corpora. We computed pairwise similarities between node and edge sets of the constructed networks using Szymkiewicz-Simpson overlap coefficient (OC):

Where X and Y are either two non-empty node sets or two non-empty edge sets. An OC of 0 indicates no overlap, while an OC of 1 indicates the smaller node or edge set is a subset of the larger one. By construction, same-case-study original and expanded networks possess OCs of 1 with each other. We applied the post hoc, context-aware pathway enrichment analysis described above to all generated networks. We tested the enrichment of Reactome pathways with sizes ranging from 3 to 100 genes, categorized as in the database’s Top-Level Pathways and disease ontologies [49]. We performed hierarchical clustering of the networks using Euclidean distance and Kendall’s correlation based on network-specific, KNet-generated p-values. We compared the resulting dendrogram to the expected one by a permutation test of Baker’s gamma correlation using one million permutations of the original dendrogram [57]. We also compared the results to two alternative statistics: i) over-representation analysis of nodes from the ENQUIRE-generated networks (the collection of all genes observed in any case study was used as the “universe”); ii) KNet statistics, using Q scores based on STRING’s high-confidence functional association network (described above) and ENQUIRE-derived gene nodes.