Theoretical background

In terms of a dynamical systems, a complex biological process evolves dynamically as high-dimensional nonlinear systems, with marked qualitative shifts interpreted as phase transitions at tipping points [33]. As illustrated in Fig 1A, the system’s dynamic progression comprises three main states: a resilient before-transition phase, an unstable critical phase marked by instability and increased sensitivity to disturbances, and a re-stabilized after-transition state phase. According to DNB theory [2,11], as the system nears a critical point, a key group of molecules called the DNBs appears, characterized by three main characteristic features (See S1 Text for details):

• The variability of each molecule within the DNBs increases sharply;
• The correlations among molecules within the DNBs significantly strengthen;
• The correlations between DNBs and those outside the group weaken.

DNB properties imply that, near the critical point, a group of fluctuation-prone and tightly correlated biomolecules exhibiting strong cooperative associations mark the forthcoming critical transition. Actually, the qualitative state transition of complex biological system can be detected through analyzing how such dominant variables in molecular associations evolve at the network level. Therefore, our proposed HNPP is designed to capture the criticality of biological systems by incorporating the structural information of higher-order (simplicial) networks into a modified personalized PageRank model (Fig 1B), which effectively quantifies quantify dynamic shifts in higher-order interactions to enable more accurate detection.

In this study, we infer higher-order interaction and reconstruct simplicial complexes based on topological data analysis (TDA) [31,32], which offers a powerful approach to uncover higher-order structural characteristics from high-dimensional single-cell datasets. Specifically, the simplicial complex inferred by TDA, widely used to describe the topological structure of biological data, is a combinatorial structure consisting of simplices such as vertices (0-simplex), edges (1-simplex), triangles (2-simplex), and higher-dimensional structures. In particular, the Rips complex is a type of simplicial complex that captures the topological relationships between nodes by connecting those within a specified distance threshold. Given a set of nodes/genes and a distance function d, the Rips complex is constructed by considering all pairs and triplets of nodes whose pairwise distances are below a specified threshold . Formally, the Rips complex is defined as follows: (i) 0-simplices (vertices): each node in P is considered a vertex; (ii) 1-simplices (edges): an edge is included if the distance between genes and is less than or equal to ; and (iii) 2-simplices (triangles): a triangle is included if the pairwise distances , , and are all less than or equal to . Thus, the Rips complex is formed by:

(1)

where denotes the k-simplex formed by nodes in P, and its diameter is defined as the maximum pairwise distance among all vertices in the simplex:

(2)

Especially, the 2-simplex complex can be expressed by:

(3)

(4)

where parameter defines the allowable distance for triplets to form a triangle.

Moreover, the PageRank method can be employed to characterize and quantify changes in molecular cooperative associations at the network level, thereby enabling the assessment of significant variations in higher-order structures. Consider as a network/graph with K nodes, where denotes its adjacency (transition) matrix and represents the degree of node i. For each node i (corresponding to the i-th row), two scenarios are considered: (1) the degree , if node i is connected to node j in graph G, and 0 otherwise; and (2) when degree , if and 0, otherwise. In the graph G, the PageRank vector , which reflects the importance score of each node, is determined by finding the fixed point of the following iterative process [34]:

(5)

Here, the damping factor is usually taken as 0.85, while the symbol “” indicates the matrix transpose operation. In light of the statistical characteristics inherent to DNB theory, this study introduces a modified personalized PageRank model as described below:

(6)

where (with denoting the outer product operation), the matrix (as detailed in Step 2 of the following HNPP method) represents the personalized transition matrix constructed from the time-specific network at time point T, is a K-dimensional column vector compensating for errors introduced by isolated nodes (see Step 3 of the HNPP method), and denotes a K-dimensional row vector of all ones. The TF-based vector , constructed based on transcription factors (TF), serves to highlight the significance of TF-associated nodes in the network, with its weighting coefficient 𝛽 set to 0.05. The vector acts as the personalized vector, as indicated in Step 3 of the HNPP method. To disentangle the effects of the major components of HNPP, we compared three settings: a pairwise DNB-based method without simplicial structures, an HNPP variant retaining the simplicial component but excluding the personalized PageRank modification (i.e., without the TF-based vector ), and the full HNPP model as proposed (S1 Fig and S2 Text). Our findings suggest that the higher-order simplicial construction makes an independent contribution to the observed performance gains beyond those achieved by standard pairwise DNB-based analysis, while personalized PageRank modification with the TF-based vector offer further complementary improvements. Overall, the improved performance of HNPP arises from the joint contribution of these components, with the higher-order structure playing a central role in the improvement.

HNPP method designed for critical phase detection in complex biological process

In the context of a biological system with K variables/genes, the proposed HNPP method is applied to pinpoint critical phase or state transitions in complex biological processes, with its detailed procedure outlined below.

1. [Step 1] Constructing a time-specific higher-order (simplicial) network/structure . We implemented the process of constructing simplicial complexes following the criteria set out in Equations (1) and (2), where the pairwise distance is defined as follows:

(7)

Here, represents the Pearson correlation coefficient (PCC) of expression value of genes and at the given time point T. It is evident that the distance function is negatively correlated with ; that is, the higher the Pearson correlation coefficient , the smaller the distance , which is consistent with the statistical properties of DNB used in network or graph construction. Moreover, we determined whether each set of three genes (gene triplets) can be connected into a triangle (2-simplex) based on Equations (3) and (7), along with the adjustable parameter 𝜖 mainly set at 0.2, thereby constructing the time-specific higher-order (simplicial) structure at time point T. Our analysis shows that the parameter 𝜖 within a specific range has little effect on the general pattern of the signal curve (S2 Fig and S3 Text). Although TDA is often used to characterize complex geometric or topological structure, in our framework the underlying metric space is constructed from Pearson-correlation-based gene associations. As a result, our HNPP implementation primarily captures higher-order structural information from a Pearson-derived association space based on pairwise linear correlations, which may limit its ability to recover genuinely nonlinear dependencies. Thus, in our research, nonlinear primarily reflects the higher-order geometric and topological structure represented by simplicial complexes, rather than the explicit modeling of arbitrary nonlinear gene–gene relationships.

1. [Step 2] Constructing the network’s transition matrix based on the time-specific higher-order (simplicial) structure . Specifically, the transition matrix is derived from the higher-order simplicial structure , where the matrix element is given by:

(8)

or

(9)

Here, refers to the number of triangles (2-simplices) jointly constructed by the gene pair and with the remaining genes.

1. [Step 3] Building the K-dimensional vectors (isolated-node adjustment), (TF-based information) and (personalization), respectively. More precisely, the isolated-node adjustment vector designed to compensate for errors introduced by isolated nodes. The i-th element/row of the normalized vector is defined as:

(10)

where represents the weight between nodes i and j.

The TF-based vector assigns a value of 1 to elements corresponding to transcription factors and 0 to all others, thereby emphasizing the regulatory importance of TF-associated nodes. The normalized vector is obtained by normalizing each element as:

(11)

The personalized vector reflects the variability of each gene/node by calculating the standard deviation of its expression levels across cells at the sampling time point T. Each element of the normalized vector is calculated as:

(12)

1. [Step 4] Calculating the PageRank vector (composed of the importance scores/local HNPP assigned to each node). As presented as in Equation (6), the local HNPP (gene-specific local HNPP) can be calculated for each gene/node based on the network’s transition matrix , the isolated-node adjustment vector , the TF-based vector , and the personalized vector , as obtained above. Moreover, the HNPP at the given time point T is computed as follows:

(13)

where the adjustable parameters L represents the count of genes falling within the top 5% in terms of local HNPP, while refers to the average of the PageRank score vector . Our results indicate parameter L within this range (typically from the top 3% to 10%) do not alter the overall trend of the signal curve, thereby demonstrating the robustness of HNPP to the choice of parameter L (S3 Fig). Near the critical state, DNB molecules exhibit collective variations in network-level associations, leading to significant dynamic changes in higher-order structures. Such shifts ultimately yield a marked increase in HNPP.

1. [Step 5] Assessing the critical phase via a one-sample t-test. To examine the capability of HNPP in capturing critical signals, we use one-sample t-test to evaluate whether the critical phase differs significantly from the before-transition phase. The statistic ST, defined in Equation (14), measures the significance of the deviation of a constant z from the mean of the n-dimensional vector .

(14)

Here, and denote its average value and standard deviation, respectively. The p-value derived from ST assesses how significantly z deviates from the average of . A p-value smaller than 0.05 indicates a statistically significant difference, whereas a p-value greater than 0.05 does not. The HNPP index indicates a critical state once it meets two criteria: (i) HNPP HNPP; and (ii) HNPP presents a statistical deviation from prior values (p-value 0.05), as detailed in S4 Text.

Overview of data and functional analysis

To evaluate the performance of the HNPP method, it was applied to both numerical simulations and six diverse real-world single-cell datasets: including embryonic developmental processes such as pericyte-to-neuron reprogramming [35] (GEO: GSE113036), differentiation of human embryonic stem cells (hESC) into definitive endoderm cells (DEC) [36] (GEO: GSE75748), the transition from inner cell mass (ICM) to visceral endoderm cells (VEC) [37] (GEO: GSE100597), and human retinal pigment epithelium (HRPE) development [38] (GEO: GSE107618), as well as complex disease-related phenomena like erlotinib resistance in lung cancer [39] (GEO: GSE149383) and the progression from hepatitis to liver cancer [40] (PMID: 36221095). The selection of real-world datasets was based on two criteria. First, all of them contain time-course information or dynamic cellular trajectories that describe complete biological processes. Second, they provide the essential information needed to identify key landmarks of these processes, such as key transitions from experimental observations. All scRNA-seq datasets were preprocessed prior to HNPP analysis, including log-transformation (i.e.,) and subsequent gene selection based on zero-expression filtering criteria. Genes with zero expression in more than 50% of cells were removed, which is a commonly adopted practice in scRNA-seq preprocessing to retain informative genes and reduce noise [41]. Detailed information on these datasets can be found in S5 Text. We carried out functional enrichment analyses based on the Metascape [42] and the ClusterProfiler [43], with pathway annotations sourced from the KEGG database.

HNPP validation based on simulation data

We applied the HNPP approach to an 8-node synthetic network (S4 Fig), described by stochastic differential equations (Eq. S4), to showcase its performance and capability in identifying early-warning indicators near critical transitions. The dynamic behavior of gene regulatory systems is often modeled using Michaelis–Menten or Hill equation-based frameworks [44,45], which have been used to depict complex biological processes such as transcription [46], cyclic biochemical reactions [47], and various other regulatory functions [48]. The system experiences a critical transition controlled by the parameter p, where marks the bifurcation point. Additional details regarding the dynamical system are provided in S6 Text. Numerical simulations were carried out by sweeping the parameter p between −0.5 and 0.2 to demonstrate the HNPP method’s effectiveness in detecting the critical transition of system approaching its bifurcation point.

A distinct increase in the HNPP score is observed near a certain parameter value (Fig 2A), indicating the onset of a critical phase. In addition, the mean HNPP values (red curve in Fig 2A) continue to provide a clear signal of the impending critical point, further demonstrating the robustness of our proposed approach. To clearly illustrate the distinction between the normal phase and the critical phase, Fig 2B shows the overall pattern of local HNPP across different nodes. Local HNPP scores remain consistently low across all nodes when the system is not near a critical point. In contrast, approaching the critical state, a subset of signaling molecules, identified as DNBs, show a sharp rise in their scores. Furthermore, as the system is close to the tipping point, the dynamical evolution of the regulatory network reveals significant reorganization of DNB subnetworks where every trio of DNBs forms a 2-simplicial complex (Fig 2C), signalling an imminent shift in the higher-order structure. Besides, as shown in Fig 2D–2F, a comparison between HNPP and molecular expression was carried out under different noise levels to demonstrate the robustness of the proposed method. With higher levels of noise, our HNPP maintained strong performance and effectively identified critical signals. The findings from the numerical experiments confirm that HNPP effectively captures early indicators of state transitions or critical phases.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Verification of the HNPP method through the numerical simulation.

(A) The average HNPP values from simulated trials indicates a marked increase in the vicinity of the bifurcation point . (B) The node-specific patterns of local HNPP are presented, revealing that DNB members undergo a rapid increase in HNPP score as the system transitions toward the bifurcation point. (C) Close to the bifurcation point, the regulatory network shows substantial reconfiguration of DNB subnetworks, with every trio of DNBs constituting a 2-simplicial complex. (D)-(F) A performance comparison between HNPP and molecular expression demonstrates that HNPP exhibits greater robustness and efficacy in pinpointing critical states.

https://doi.org/10.1371/journal.pcbi.1014475.g002

Evaluation of HNPP for critical phase detection in different biological processes

To evaluate the applicability of our proposed method, HNPP was utilized to six distinct scRNA-seq datasets of different biological processes, including pericyte-to-neuron reprogramming [35], hESC-to-DEC differentiation [36], ICM-to-VEC transition [37], human retinal pigment epithelium (hRPE) development [38], lung cancer cells erlotinib-resistance (LCCER) [39], and hepatitis-to-liver-cancer (HELC) progression [37]. Time-specific HNPP scores (as defined in Equation (13)) was applied to detect potential critical phase. The successful detection of critical states across various biological processes demonstrates the accuracy and robustness of HNPP.

For the pericyte-to-neuron data, as shown by the rose-red curve in Fig 3A, a notable increase in the mean HNPP score is observed on day 7 (=), before the lineage bifurcation into DLX- and NEUROG2-dominated neuronal fates at day 14 [35]. The hESC-to-DEC data (the rose-red curve of Fig 3B) exhibit a sharp HNPP transition from 24 h to 36 h (= ), which indicates that definitive endoderm fate commitment occurred at 72h [36]. In ICM-to-VEC data, the rose-red curve in Fig 2C shows a statistically significant shift at embryonic day 4.5 (E4.5) (), preceding the epiblast-to-primitive streak transition at E6.5 [37]. For hRPE data, two critical states are detected by HNPP (Fig 3D): the first tipping point at 7 w () precedes the transition around 9 w associated with early retinal pigment epithelium (RPE) differentiation, while the second at 11 w () serves as an indicator of the onset of visual-cycle-related functional maturation and metabolic reprogramming around 13 w [38]. When applied to disease-related LCCER data, it is seen from the rose-red curve of Fig 3E that a pronounced rise () in the average HNPP appears at day 4, marking a critical transition that precedes the emergence of the resistant state in erlotinib-treated PC9 cells at day 9 [39]. For the non-time-series single-cell dataset of HELC, the progression from hepatitis to liver cancer can be partitioned into four distinct clusters via the pseudo-temporal trajectory analysis (see S5 Fig and S7 Text). Fig 3F illustrates a sharp rise () in the average HNPP score at cluster 3 (C3), indicating a critical shift toward the liver cancer state at C4 [40]. In contrast, the light-blue curves in Fig 3A–3F illustrate dynamic changes in the mean expression levels of the top 5% differentially expressed genes (DEGs), yet such variations were insufficient to offer a reliable indicator of the critical transition. Moreover, the landscape of gene-specific local HNPP was employed to illustrate overall dynamic changes in signaling and non-signaling genes (Fig 3G–3L), revealing sharp elevation in local HNPP for signaling genes. In addition, compared with existing approaches, including Gaussian graphical optimal transport (GGOT) [7], single-sample landscape entropy (SLE) [49], BioTIP [50], directed-network rank score (DNRS) [2], and module-based dynamic network biomarker (M-DNB) [4] (See S8 Text for details), HNPP detects more effective critical signals and provides earlier warning signals before the known biological transition (Table 1), thereby demonstrating its enhanced ability to pinpoint critical phases in complex biological processes. Sensitivity analyses across different cell numbers and data sparsity settings further support the robustness of HNPP (S6 and S7 Figs).

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Comparison of the performance among different critical phase detection methods.

https://doi.org/10.1371/journal.pcbi.1014475.t001

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Critical phases of various biological processes revealed by the proposed HNPP.

Comparison of dynamic changes between HNPP (rose-red curve) and mean gene expression (light-blue curve) across six real-world single-cell datasets: (A) Pericyte-to-neuron, (B) hESC-to-DEC, (C) ICM-to-VEC, (D) hRPE, (E) LCCER and (F) HELC. Dynamic patterns of signaling and non-signaling genes are illustrated through local HNPP landscape for these datasets: (G) Pericyte-to-neuron, (H) hESC-to-DEC, (I) ICM-to-VEC, (J) hRPE, (K) LCCER and (L) HELC.

https://doi.org/10.1371/journal.pcbi.1014475.g003

Analysis of the evolving dynamics of signaling genes

At the identified critical phase, we chose the top 5% of genes exhibiting the highest local HNPP values as potential signaling genes, to further explore their roles in the evolving dynamics of biological processes. Specifically, we illustrated the evolution of signaling genes, focusing on their dynamic changes at the network level and how these alterations contribute to the emergence of critical states in complex biological systems. For pericyte-to-neuron data, it can be seen from Fig 4A that there occurs a notable shift in the network structure on day 7, where every trio of the most prominent signaling genes forms the higher-order structures (2-simplicial complex), indicating the cell fate determination for neuronal differentiation after day 7 [35]. Similarly, for the disease-related LCCER data, a marked shift in the network structure occurs on day 4 (Fig 4B), signaling a critical transition of erlotinib-treated PC9 cells towards an irreversible resistance state [39]. The overall dynamic changes of the regulatory network for these datasets are further detailed inS8 Fig. Moreover, a gene cluster comprising high-HNPP or signaling genes (top 5% molecules with the highest local HNPP values) together with low-HNPP genes (top 5% molecules with the lowest local HNPP values), was selected to carry out RNA velocity analysis using expression profiles. In particular, RNA velocities were computed with scVelo [51], which employs a gene-specific kinetic model to estimate cell-state transitions from RNA splicing dynamics. For the pericyte-to-neuron, ICM-to-VEC, and LCCER datasets, as shown in Fig 4C–4E, RNA velocities are projected onto the UMAP embedding as streamlines. These streamline patterns consistently reveal the directional progression from the before-transition to the after-transition phase, thereby uncovering the developmental trends of complex biological processes such as cell differentiation and disease evolution.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Dynamic evolution analysis of signaling genes.

Network dynamics of signaling genes are presented for (A) pericyte-to-neuron and (B) LCCER. it can be seen that there occurs a notable shift in the network structure on critical state, where every trio of the most prominent signaling genes forms the higher-order structures (2-simplicial complex). Based on the expression profiles of values of the top high- and low-HNPP genes, RNA velocity fields of the before-transition, critical, and post-transition phases are presented for (C) pericyte-to-neuron, (D) ICM-to-VEC, and (E) LCCER. The inferred streamline flows consistently reveal the progression from the before-transition toward the after-transition phase.

https://doi.org/10.1371/journal.pcbi.1014475.g004

Revealing potential regulatory mechanism of erlotinib-resistance in lung cancer cells

To reveal insights into the molecular mechanisms of erlotinib-resistance in lung cancer cells, we carried out functional analysis of the signaling genes. Specifically, KEGG pathway enrichment analysis revealed that the signaling genes were significantly enriched in pathways closely associated with tumor erlotinib resistance, including the Rap1 signaling pathway [52], MicroRNAs in cancer [53], and TGF-beta signaling pathway [54] (Fig 5A). Moreover, as shown in Fig 5B, GSVA analysis of signaling genes indicated that the enrichment scores of pathways such as the B cell receptor signaling pathway, Rap1 signaling pathway, Adherens junction, Apoptosis, and MAPK signaling pathway showed an increasing trend during the development of cancer resistance, which may promote erlotinib resistance by enhancing tumor cell proliferation, self-renewal, apoptosis suppression, and adhesion. Additionally, GO enrichment analysis also showed that the signaling molecules are predominantly involved in biological processes associated with tumor resistance, including the intrinsic apoptotic signaling pathway, response to transforming growth factor beta, and regulation of apoptotic signaling pathway (Fig 5C). Additional results of the GO enrichment analysis for cellular components (CC) and molecular functions (MF) are provided in S9 Fig and S9 Text. In addition, we further investigate the functional relevance of signaling genes in embryonic developmental datasets, including the pericyte-to-neuron and hESC-to-DEC datasets (S1-S2 Tables, S10 Fig and S10 Text). To further investigate the potential regulatory mechanisms underlying tumor erlotinib resistance, we analyzed the dynamic behavior of the subnetwork formed by signaling molecules together with their first-order DEG neighbors within the PPI network. Marked alteration of gene expression was observed in the subnetwork before and after the critical phase, implying a major change in regulatory dynamics (Fig 5D). Furthermore, the 1st-order DEG neighbors were found to be enriched in tumor resistance–associated pathways (Fig 5E), including Rap1 signaling, Proteoglycans in cancer, and FoxO signaling pathway.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. The potential mechanisms underlying erlotinib resistance in lung cancer cells.

(A) KEGG-based enrichment analysis was conducted to investigate the signaling genes. (B) GSVA-based functional assessment reveals that signaling molecules play diverse roles in cancer resistance progression. (C) Gene Ontology (GO) analysis suggested that signaling molecules are significantly enriched in biological pathways relevant to tumor resistance. (D) Dynamic alterations in the regulatory network formed by signaling molecules and 1st-order DEG neighbors were analyzed during the process of cancer resistance. (E) KEGG-based enrichment analysis was performed on the 1st-order DEG neighbors. (F) The analysis of signaling molecules and 1st-order DEG neighbors highlighted their involvement in cancer resistance through the Rap1 signaling pathway. (G) Cell–cell communication between CTNNB1 + critical-phase cells and after-transition phase cells is mediated via the ncWNT and SPP1 pathways. (H) Prognostic significance of CTNNB1 is evaluated in the TCGA-LUAD data.

https://doi.org/10.1371/journal.pcbi.1014475.g005

Our analysis revealed that the Rap1 signaling pathway functions as a major regulatory mechanism underlying cancer resistance. Specifically, upregulation of the signaling gene CTNNB1 drives the elevated expression of 1st-order DEG neighbor MAGI2 and RAPGEF6, which serve as a membrane-associated scaffold protein and a guanine nucleotide exchange factor (GEF), respectively, and collectively mediate the downstream activation of RAP1A and RAP1B (Figs 5F and S11). As members of the small GTPase family, RAP1A and RAP1B are regarded as prerequisites for the initiation of cell survival and proliferative signaling pathways [55]. Activation of RAP1A and RAP1B amplifies two critical signaling cascades: (i) the MAP2K3–MAPK12 axis, which triggers the MAPK pathway to sustain tumor cell proliferation and survival [56]; and (ii) the PIK3R1–Akt axis, which maintains PI3K–Akt signaling way to enhance cellular adaptation to drug exposure [57]. The results suggest that the signaling gene CTNNB1 facilitates acquired resistance in lung cancer cells via the Rap1 pathway, by maintaining pro-survival signaling, inhibiting apoptosis, and reinforcing cell adhesion and motility [58]. To further investigate the role of cell–cell communication in resistance, critical-phase cells were stratified into CTNNB1+ and CTNNB1- subsets based on CTNNB1 expression, and receptor–ligand interactions between CTNNB1 + critical-phase cells and cells of after-transition phase were analyzed to assess their contribution to resistance. The analysis revealed that the WNT5A–FZD2 interaction within the ncWNT pathway [59] promotes the transition of CTNNB1 + critical-phase cells toward resistant states, while the SPP1–CD44 pair in the SPP1 pathway [60] provides negative feedback regulation, together establishing a positive–negative feedback mechanism that accelerates erlotinib resistance (Figs 5G and S12). Besides, High CTNNB1 expression was associated with poor prognosis in lung cancer, consistent with the findings (Fig 5H) [61], highlighting its role as a cancer marker of adverse outcomes.