Introduction

In the field of cellular biology, the signal recognition and response of cells to environmental stimuli are key concepts. These processes are commonly involved in intricate biochemical signaling networks, where the strength and concentration of extracellular signals can influence cell fate and intercellular phenomena [1–3]. By comprehending the intricacies of such interactions, we can improve our understanding of cellular behavior, which may lead to novel approaches for the diagnosis and treatment of various pathological conditions. Therefore, it is crucial to study the mechanisms involved in this process to gain deeper insights into cellular decision-making. Molecular noise is a common phenomenon in biological systems [4], and its presence can impede the ability of cells to accurately sense and respond to various input signals [5]. The inherent stochasticity of biological noise renders cellular decision-making probabilistic in nature [5]. A wide range of potential outcomes can impact the cell’s behavior and the system’s overall function, each with varying degrees of likelihood. Even though biological noise presents significant challenges, researchers are actively seeking new ways to reduce its effects and improve the accuracy of cellular signaling and decision-making processes. Specifically, it has been shown that signal transduction noise arises from both extrinsic and intrinsic sources. Intrinsic noise is caused by the stochastic nature of biochemical reactions, which involve a small number of molecules. On the other hand, extrinsic noise results from variations in the levels of TNF receptors and cellular heterogeneity upon stimulation [6]. To gain a better understanding of the probabilistic nature of cellular decision-making, we employ a system model with explicit statistical parameters for alternative outcomes and quantify detection performance from single-cell data. Recently, similar models have used decision theory to derive optimal thresholds and error probabilities from such measurements. The objective of such a model is to assess the system’s capability to accurately detect the presence or absence of the desired signal. In those studies, the prior probabilities for signal presence/absence were assumed to be known, typically equal, that is 0.5/0.5, and then the maximum likelihood decision approach was applied [7–9]. It is important to consider a different modeling approach when the prior probabilities of signal presence are unknown. This paper introduces a signal detection model where the cell maximizes the probability of detecting the signal for any given false alarm probability. The signal detection model used in this study is based on the Neyman-Pearson signal detection theorem, which takes into account the probability of falsely detecting a signal when it is absent, also known as the false alarm probability. These quantitative and probabilistic models also have the potential to shed light on the stochastic signaling mechanisms and phenotype induction associated with genetic mutations that cause diseases [10]. By providing a deeper understanding of these mechanisms, our proposed model may contribute to the development of novel approaches for the diagnosis and treatment of genetic disorders. In order to validate such a method in a biologically meaningful context, we utilized a signaling pathway involving the regulation of the transcription factor Nuclear Factor κB (NF-κB) by tumor necrosis factor (TNF) (Fig 1) [5]. This allowed us to investigate the method’s effectiveness in a real-world scenario and assess its potential for future applications. TNF is a crucial antiviral cytokine that can cause significant damage to healthy host tissues. In addition to its role in activating NF-κB, a critical gene regulator involved in cell survival, viral replication, and pathological processes such as autoimmune diseases, various types of cancer, and inflammation, TNF has also been shown to mediate both anti-apoptotic and pro-apoptotic signals and can even trigger necroptosis, a type of pro-inflammatory cell death [11]. Upon activation, NF-κB can act to inhibit apoptosis in cells. The presence of the A20 molecule can also help reduce noise in the signaling pathway by acting as negative feedback to the information bottleneck. However, the effect of A20 activation on the amount of information attained in the system is variable and can either increase or decrease information [5]. The loss or mutation of the A20 molecule, inhibitory feedback of the NF-κB pathway, has been implicated in various pathological conditions such as cancer and chronic inflammation [12,13]. The negative feedback mediated by A20 can result in the reduction of NF-κB levels, leading to downstream effects on cellular signaling. Also, A20 deficiency may contribute to cancer development and lead to significant pathological conditions. This effect is believed to occur due to the persistent activation of anti-apoptosis genes mediated by NF-κB, as described in [14]. In response to NF-κB activation, the transcription of early genes, including cytokines, can be upregulated [7]. Furthermore, tumor microenvironments often contain heterogeneous cell populations with varying degrees of A20 expression, which makes mixed populations particularly relevant for understanding signal detection in cancer contexts.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 1. The Role of A20 in TNF–NF-κB signaling pathway.

https://doi.org/10.1371/journal.pcbi.1014608.g001

The objective of this paper is to demonstrate the applicability of the proposed statistical decision-making framework to diverse molecular signaling systems. We aim to showcase the versatility and effectiveness of this framework across different biological contexts when there is no knowledge of priors. The aim is twofold: first, to observe how the suggested decision model is expanded to examine various time points in the process of signaling outcomes using statistical multi-dimensional analysis for the TNF–NF-κB. Second, using the widely recognized graphical tool, the receiver operating characteristic (ROC), to visualize cell decision-making performance in both normal and abnormal conditions. The overall objective is to determine the probabilities of cell decision-making and establish a correlation between statistical parameters and metrics with biological discoveries.

Results

Based on single-cell data analysis of the TNF–NF-κB signaling pathway, it has been revealed that a cell can differentiate between low and high TNF level concentrations at the input [5]. The presence of molecular noise may impede the cell’s ability to accurately discriminate the strength of the input signals and external stimulations. Consequently, it affects the cellular decision-making results [5]. As previously noted, due to numerous diseases and pathological conditions that may arise from unanticipated and altered cellular decisions, there is a keen interest in devising suitable models for the characterization and quantification of molecular signal detection factors and cellular decision-making.

To analyze the proposed cellular decision-making framework for the signaling pathway in Fig 1, using nuclear NF-κB responses measured at the output upon receiving various TNF concentration levels at the system input, we focus on two types of decision probabilities as the desired metrics: probability of false alarm P_FA, and probability of detection P_D (defined in METHODS). We evaluate the proposed model by choosing 0.0021, 0.0052, and 0.013 ng/mL as the low TNF levels (null hypothesis H₀) vs. 0.2, 0.51, 1.3, and 50 ng/mL for high TNF levels (alternative hypothesis H₁). For instance, Fig 2A shows the histograms of NF-κB responses of hundreds of cells after 30 minutes of exposure to a low TNF level of 0.0052 ng/mL vs. a high TNF level of 0.2 ng/mL. Fig 2B shows the associated univariate PDFs of the responses in Fig 2A, including the optimal decision threshold value when P_FA = 0.1 obtained by the Neyman-Pearson theorem (details in METHODS). The vertical blue line as the optimal decision threshold is the best value that maximizes the probability of detection P_D, at P_FA = 0.1. Based on the definition in the Equation (4), the right-side area of the optimal decision threshold line, shaded in light red, represents the probability of false alarm P_FA region. Also, the probability of detection P_D region is on the right side of the obtained decision threshold under the associated PDF curve of high TNF level (hypothesis H₁), which is not shaded due to possible overlap with the P_FA region. Fig 2C also illustrates the same PDFs for both low and high TNF levels in Fig 2B, but includes the shaded false alarm region when P_FA = 0.7. The expansion of the shaded P_FA region is expected. This increase in threshold enlarges the P_D region as well, which means a higher P_D value.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 2. Univariate Neyman–Pearson detection of TNF level from single-cell NF-κB responses.

A) Histograms of measured NF-κB concentrations for low TNF 0.0052 ng/mL vs. high TNF of 0.2 ng/mL. B) Gaussian PDF of NF-κB responses to low and high TNF levels after 30 minutes of exposure including the optimal decision threshold when P_FA = 0.1 shown as blue vertical line and the P_FA associated region. C) Same PDFs when P_FA = 0.7 and its corresponding region.

https://doi.org/10.1371/journal.pcbi.1014608.g002

As a concise demonstration of the utilized experimental data, Fig 3A demonstrates the mean values of cells’ responses to different TNF levels in NF-κB after 30 minutes of TNF exposure. That is one of the important variables that can help to justify ROCs behaviors in Fig 4, as well. Also, this figure tells us that as the TNF level increases in each case, the corresponding mean increases as well due to the shifting of histograms to the right in the coordinate system. Fig 3B shows the standard deviation of cells’ responses to different TNF levels in NF-κB after 30 minutes of TNF exposure, where there are no high differences and fluctuations between various TNF levels’ standard deviations. When it comes to analyzing data, we invoke ROC (explained in METHODS) curves, here to see how the signaling system performs. Fig 4 shows the hypothesis testing problems of 0.0052 ng/mL as the low TNF level vs. 0.2 ng/mL and 50 ng/mL as the two high TNF levels over two different time points after TNF exposure, 30 minutes and 4 hours (individually and jointly analyzed) in both wild-type and A20^−/− cells. First, by comparing Fig 4C with Fig 4A associated with wild-type cells for different high TNF levels, 50 ng/mL and 0.2 ng/mL, respectively, this suggests that there is a visible reduction in all ROCs when the high TNF level decreases from 50 ng/mL to 0.2 ng/mL, which is consistent with our earlier analysis. A lower concentration of high TNF, like 0.2 ng/mL, has a closer histogram to the low TNF of 0.0052 ng/mL than the highest high TNF, which is 50 ng/mL.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 3. Dynamic range of responses to various TNF levels.

A) Nuclear NF-κB means for various TNF levels. B) Standard deviations of Nuclear NF-κB in response to different TNF levels.

https://doi.org/10.1371/journal.pcbi.1014608.g003

Download:

• PNG
  larger image
• TIFF
  original image

Fig 4. Receiver operating characteristic (ROC) curves for TNF–NF-κB signaling pathway evaluated using Neyman-Pearson theorem concepts when low TNF, H₀ is 0.0052 ng/mL.

A) Wild-type cells result for high TNF, H₁ equal to 0.2 ng/mL. B) A20^−/− cells results for high TNF, H₁ equal to 0.2 ng/mL. C) Wild-type cells result for high TNF, H₁ equal to 50 ng/mL. D) A20^−/− cells results for high TNF, H₁ equal to 50 ng/mL.

https://doi.org/10.1371/journal.pcbi.1014608.g004

Another important observation could be about the comparison of 4-hour ROCs with 30-minute ROCs for both high TNF levels, 0.2 ng/mL and 50 ng/mL, where we can say the 4-hour curves are below the 30-minute curves due to the activation of the negative feedback over time that reduces NF-κB level after 4 hours. In A20^−/− cells, we expect a shallower late decline than in wild-type, because the negative feedback that normally lowers the 4-hour response is largely absent; the A20^−/− 4-hour P_D nonetheless remains below its 30-minute value (Tables 1 and 2), since the H₀–H₁ separation is still smaller at 4 hours. The transition from wild-type to A20^−/− is like shifting the corresponding histograms to the right-hand side of the coordinate system. The means figure, Fig 3A, justifies this claim very well through a comparison of associated mean values of low and high TNF levels in the corresponding hypothesis testing. Both H₀ and H₁ histograms in A20^−/− have shifted to the right, and the means are good representatives of their histograms or their PDFs. In those figures, we can clearly see that the difference between the H₀ means and H₁ means plays an important role in the P_D of the Neyman-Pearson decision-making framework. Because, e.g., as the means become farther away, it seems like the histograms or PDFs are well-separated, and we can say the probability of detection is higher because the PDFs or histograms are nearly well separated. Also, these can be pointed out as the opposite, as well. Those cases with lower P_D, in each case, have a closer means of H₀ and H₁. As we can see, analyzing both time points, 30 minutes and 4 hours together, is like a bivariate PDF that can improve the decision-making performance. The bivariate case almost always stays either very close to the one with higher P_D or at the top of each individual 30-minute and 4-hour time points, reasonably. Moreover, by comparing the above ROCs from left (Fig 4A and 4C) to the right (Fig 4B and 4D), cell type transition from wild-type to A20^−/−, we always notice a reduction in all cases, 30 minutes, 4 hours, and the bivariate scenario, as well.

Download:

• PNG
  larger image
• TIFF
  original image

Table 1. Decision Probabilities of wild-type and A20^−/− cells for TNF = 0.0052 ng/mL vs. TNF = 0.2 ng/mL.

https://doi.org/10.1371/journal.pcbi.1014608.t001

Download:

• PNG
  larger image
• TIFF
  original image

Table 2. Decision Probabilities of wild-type and A20^−/− cells for TNF = 0.0052 ng/mL vs. TNF = 50 ng/mL.

https://doi.org/10.1371/journal.pcbi.1014608.t002

To better sense the excellence of bivariate analysis against univariate analysis in our cellular decision-making framework, Fig 5 is illustrated in which for a fixed P_FA = 0.1, where low TNF is 0.0052 ng/mL, the probability of detection P_D for several high TNF levels is graphed, e.g., for wild-type cells, the bivariate P_D(30 minutes & 4 hours) = 0.8, while the individual P_D(30 minutes) = 0.71 and P_D(4 hours) = 0.42 (Table 1). Another noteworthy impression we mentioned above is more obvious here when we compare the wild-type and A20^−/− cases for each high TNF levels, e.g., for the high TNF level of 0.2 ng/mL, the P_D drop is noticeable for individual time point comparison, P_D(wild-type at 30 minutes) = 0.71 while P_D(A20^−/− at 30 minutes) = 0.24. When it comes to 4-hour analysis, we can notice the same behavior, P_D(wild-type at 4 hours) = 0.42 while P_D(A20^−/− at 4 hours) = 0.14 (Table 1). Besides that, Table 2 shows the same behavior when the high TNF level is 50 ng/mL vs. the same low TNF of 0.0052 ng/mL. The value P_FA = 0.1 is used throughout as a somewhat conventional engineering reference (analogous to a 5% Type-I error in classical statistics), not a biologically privileged choice; the WT-versus-A20^−/− ROC curve ordering is preserved across all values of P_FA ∈ {0.01, 0.05, 0.10, 0.20, 0.50} (S3 Fig, S1 Table).

Download:

• PNG
  larger image
• TIFF
  original image

Fig 5. Probability of Detection vs. various TNF levels for both wild-type and A20^−/− cells at different time points.

https://doi.org/10.1371/journal.pcbi.1014608.g005

These analyses are also done for another two different low TNF levels, 0.0021 ng/mL and 0.013 ng/mL, vs. the aforementioned high TNF levels, 0.2 ng/mL and 50 ng/mL, whose results are all in good agreement.

The corresponding ROCs, along with their eigenvalue decompositions to be used in the Gil-Pelaez lemma for decision-making analysis, are fully provided in SUPPLEMENTARY MATERIALS. These findings provide additional validation of the significance of this parameter within the system model, as we observed biologically meaningful outcomes.

Throughout, the P_D values in Tables 1 and 2 are reported with bootstrap 95% confidence intervals (CI) obtained from 1000 resamples of the single-cell data.

Analysis of mixed cell populations

To further demonstrate the biological relevance of the framework, we analyzed a scenario motivated by cancer biology, where malignancies can contain heterogeneous populations of cells with varying A20 expression. We modeled a mixed population in which a fraction β are A20^−/− cells and (1 − β) are wild-type, with both subpopulations exposed to either low (H₀: 0.0052 ng/mL) or high (H₁: 50 ng/mL) TNF — i.e., under each hypothesis the population is the same (1 − β) WT + β A20^−/− blend, differing only in the TNF level. Because A20^−/− cells mount elevated NF-κB responses even under low TNF, mixing the two genotypes erodes a population-level observer’s ability to distinguish low- from high-TNF conditions, most severely at intermediate fractions (quantified below). We emphasize that this is a statistical property of the population readout, not a claim that any individual cell ‘misinterprets’ the signal — single-cell commitment to apoptosis or proliferation is a separate question.

Fig 6 shows the detection probability P_D of the optimal population observer — a single shared Neyman–Pearson threshold applied to the mixed-population readout — at P_FA = 0.1 versus the A20^−/− fraction β. At the boundaries the results match our pure-population findings: for β = 0 (WT), P_D = 0.99 (30 min), 0.65 (4 h), ≈ 1 (bivariate); for β = 1 (A20^−/−), P_D = 0.67, 0.46, 0.79 (Table 2). Between these limits the behavior is not monotonic. At 30 minutes P_D declines smoothly with β. At 4 hours P_D falls steeply to a minimum of ≈0.30 near β ≈ 0.3 and then partially recovers toward the pure-A20^−/− value as β → 1, because at intermediate mixtures the elevated low-TNF (H₀) baseline of A20^−/− cells overlaps the high-TNF (H₁) responses of wild-type cells, which a single shared threshold cannot separate; once the population is homogeneous again (β → 1) the problem simplifies and P_D rises. The bivariate curve shows only a shallow version of this dip and remains the highest of the three across the whole range of β.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 6. Probability of Detection P_D at P_FA = 0.1 versus A20^−/− fraction β for a mixed population (H_₀ = 0.0052 vs. H_₁ = 50 ng/mL), using a single shared (population-observer) Neyman–Pearson threshold.

At 4 hours P_D is non-monotonic, with a minimum near β ≈ 0.3.

https://doi.org/10.1371/journal.pcbi.1014608.g006

Fig 7 presents the complementary view — the P_FA required to reach P_D = 0.8 — as β increases. Mirroring Fig 6, the 4-hour curve is non-monotonic: the false alarm cost peaks near β ≈ 0.65, where the mixed-population detection is hardest, then declines as β → 1. The combined 30-minute-and-4-hour analysis (green) keeps the false alarm rate the lowest of the three across the entire range — roughly half that of the best single time point at high β — confirming the robustness of multi-time-point detection in heterogeneous populations.

Download:

• PNG
  larger image
• TIFF
  original image

Fig 7. Probability of False Alarm P_FA at P_D = 0.8 versus A20^−/− fraction β for the same mixed population and single shared threshold; the 4-hour curve peaks at intermediate β, mirroring Fig 6.

https://doi.org/10.1371/journal.pcbi.1014608.g007

Figs 6 and 7 adopt the perspective of an external observer applying a single, shared threshold to the mixed population — the biologically pertinent case, since a mutated A20^−/− cell cannot recalibrate its own decision threshold but inherits the setpoint of the normal downstream machinery; this inability to reset the threshold is itself a route by which such cells fail to commit to apoptosis and over-proliferate. A genotype-informed observer that gives each subpopulation its own optimal threshold dominates the single shared-threshold observer at every β (S5 Fig): the two nearly coincide at 30 minutes, where the mixture is close to Gaussian, and diverge sharply at 4 hours, where the genotype-informed test avoids the shared-threshold dip. This requires single-cell genotype information; when that information is available the problem decouples into two ordinary single-population detections, which can only match or exceed the shared-threshold result.

To make the underlying distributions explicit, S6 Fig shows the full Gaussian-mixture response density at representative mixing fractions (β = 0.3, 0.5, 0.7), overlaid on the single-Gaussian approximation for both hypotheses at both time points; the mixture is near-Gaussian at 30 minutes but becomes visibly bimodal at 4 hours for intermediate β. Figs 6 and 7 use the exact mixture-LLR (GMM) detector derived in METHODS; the single-Gaussian moment-matched approximation tracks it within |ΔP_D| ≤ 0.017 at 30 minutes and ≤ 0.065 at 4 hours across all β (S7 Fig).

An assumption of this analysis is that we treat the wild-type and A20^−/− responses as an independent linear mixture, without paracrine signal exchange between the two subpopulations. This is biologically appropriate for the prevailing clinical context of A20 loss — B-cell lymphoma [14] — a hematologic malignancy in which cells circulate and are essentially well-mixed in a homogeneous medium, so the dense paracrine coupling characteristic of solid-tumor stroma is largely absent. In settings with substantial paracrine exchange between wild-type and A20^−/− cells, such as solid tumors or dense tissue, a coupled model would instead be required — for example, a paracrine-feedback term modifying the effective TNF input experienced by the A20^−/− cells [15] — which we leave to future work.

Discussion

The introduced approach in this work proposes a set of statistical signal processing and decision-theoretic methods and metrics to model and measure the cellular decision-making errors in a TNF–NF-κB signaling system, under normal conditions and under abnormalities caused by noise or signaling uncertainties. This study showed biologically relevant and meaningful results that agreed with the functionality of the above-mentioned signaling pathway. The important transcription factor NF-κB regulates many genes, controls cell fate, and is involved in apoptosis and the transition of an antiviral state. Therefore, it is crucial for the cell to make an accurate decision in the information transmission of TNF–NF-κB signaling pathway. Here, we focused on such an important cell signaling system for which we presented univariate and bivariate methods to analyze the cell decision-making process and outcomes using Neyman-Pearson theorem, a statistical signal processing method which does not require knowledge of prior probabilities, whereas previous works by Habibi et al., Ozen et al., and Emadi et al. [7–9] require the knowledge of prior probabilities. The signaling pathway could transmit the information of changes in TNF concentration level from 0.0021 ng/mL to 50 ng/mL. The mean estimates in Fig 3A are derived from (and represent) the histograms in Fig 2A. The difference between the means of H₀ and H₁ plays a crucial role in the probability of detection P_D of the proposed model. When the means are further apart, the histograms appear more separated, indicating a higher probability of detection. Conversely, cases with lower P_D show closer means between H₀ and H₁.

A key finding is that when the TNF signal is strong in wild-type cells, the signal detection probability is higher compared to the weak TNF signal detection. The univariate results in Fig 4 demonstrate that in wild-type cells, the system outperforms A20^−/− cells in both time points, 30 minutes and 4 hours. Analyzing receiver operating characteristic (ROC), as a measure of system performance, indicates that both time points, 30 minutes and 4 hours, together, which can be considered as a joint or bivariate case, enhances the performance of the decision-making framework. The bivariate case consistently remains either very close to the one with higher P_D or above both univariate scenarios, the 30 minutes and 4 hours. Furthermore, when comparing the ROC curves from left to right panels (cell transition from wild-type to A20^−/− in Figs 4A–4D), a reduction in performance is observed in all cases: 30 minutes, 4 hours, and the bivariate case. This advantage persists even in heterogeneous cell populations containing mixed wild-type and A20^−/− cells, demonstrating the robustness of the multivariate approach in biologically realistic scenarios (Figs 6 and 7).

A natural question is whether the optimal decision boundary we derive corresponds to anything a real cell can implement. The boundary is a decision-theoretic optimum — an upper bound on the detectability that any downstream readout can in principle achieve. Whether cells actually approach this bound is an empirical question, and a substantial body of experimental work shows that signaling networks do implement stable decision thresholds, each defined by a dedicated molecular circuit and stable in unperturbed cells, shifting only under specific perturbations such as mutation or pharmacological treatment. In NGF signaling, a curved boundary in the two-dimensional pERK–pAKT plane separates proliferating from differentiating cells and remains invariant across stimuli and upstream perturbations [16]. In the control of cell death, the survival/death decision is set by a threshold in the dedicated apoptotic signaling circuit [17]. In cell migration, distinct migration and polarity patterns are governed by a threshold in the dedicated Rho-GTPase cytoskeletal machinery [18]. And for TNF specifically, the decision of blood-vessel cells to undergo angiogenesis is controlled by a TNF-dose threshold defined by the dedicated angiogenic machinery [15]. In each case the threshold is a stable, reproducible feature of the unperturbed network that shifts predictably only under mutation or drug — exactly the behavior our framework anticipates when such perturbations alter the underlying means and variances. Variability in the downstream network therefore shifts the attainable operating point along the ROC, lowering P_D at a fixed P_FA, but it does not move the upper bound; the distance between the two is a direct measure of how efficiently a cell’s downstream machinery employs the information available to it.

Because these decision thresholds are set by dedicated downstream networks — the apoptotic and proliferative machinery, largely separate from the upstream signaling cascade — they are typically stable whether a cell is healthy or transformed, and a mutation’s consequence depends on which layer it perturbs. A mutation that perturbs signaling alone, as A20 loss largely does by removing a brake on NF-κB, leaves the threshold fixed but shifts the distribution of inputs reaching it; the population is then read against a single shared threshold, and the non-monotonic loss of discriminability we find for mixed wild-type/A20^−/− populations (Figs 6 and 7) is a candidate mechanism by which such signaling lesions drive incorrect survival or proliferation decisions. A mutation or drug that instead alters the threshold-defining network itself shifts the threshold, effectively giving the affected subpopulation its own decision boundary; in that regime discriminability improves and the problem reduces to the genotype-informed (two-threshold) case of S5 Fig. Because A20 is a multifunctional node, both modes may operate, so we present the shared-threshold case in the main text and the genotype-informed case in the supplement to cover either possibility.

The developed framework also makes quantitative, testable predictions. Because detectability depends mainly on the mean separation |μ₁ − μ₀| relative to the noise, any perturbation that narrows this separation or inflates the variance should lower P_D at a fixed P_FA by an amount computable from the perturbed model parameters alone. Two pharmacological experiments follow directly. First, partial inhibition of IKK in wild-type cells should compress 30-minute discriminability: IKK activity governs the temporal profile of TNF-induced NF-κB activation [19], and because IKK is the kinase driving the response, lowering it pulls the high-TNF (H₁) response toward the unstimulated baseline. In our data the wild-type H₀–H₁ mean gap at 30 min (0.0052 vs 0.2 ng/mL) is 0.32 versus only 0.14 in A20^−/− — the latter already compressed by an elevated H₀ baseline (8.32 vs 8.09) from the missing A20 brake on basal NF-κB — so titrated IKK inhibition should drive the wild-type 30-min ROC toward the A20^−/− curve. Second, restoring A20 activity in A20^−/− or A20-low cells should rebuild late-phase negative feedback, since A20 terminates NF-κB signaling by editing the ubiquitin chains on RIP1 and targeting it for proteasomal degradation [20]; in our data the A20^−/− 4-hour means are elevated under both hypotheses (H₀ 8.54 vs 7.98; H₁ 8.83 vs 8.32), with the larger shift at H₀, so restoring feedback should widen the H₀–H₁ gap and partially rescue the 4-hour detection deficit (P_D from 0.46 toward the wild-type 0.65 at P_FA = 0.10). Each prediction is a quantitative ROC shift, not a qualitative expectation, and can be checked by re-fitting the Gaussian parameters after perturbation in new experiments.

The NP framework and the Bayesian/ML approach used previously [7–9] are complementary rather than competing. Both rest on the same likelihood ratio test statistic; they differ in how the decision threshold is chosen, to achieve a different decision objective. The Bayesian/ML route requires committing to the prior probabilities P(H₀) and P(H₁) which are assumed equal in the earlier studies, and yields a single operating point, whereas the NP route varies the false alarm rate to construct the entire ROC curve without invoking any prior (See also the Comparison with Maximum Likelihood Detection subsection in RESULTS). When the priors are genuinely unknown, as in most single-cell settings, NP provides an assumption-free score. When a prior is meaningfully known — for example a defined experimental ratio of stimulated to unstimulated cells, or an evolutionarily encoded expectation — the appropriate choice is dictated by the decision criterion to be optimized.

There are some limitations to consider. The current analysis assumes that the log-transformed NF-κB responses follow Gaussian distributions, which, while a reasonable approximation supported by the data, may not hold for all signaling systems or under all conditions. Although normality tests reject strict Gaussianity at the large single-cell sample sizes available here [21], the resulting deviations do not practically affect the detection-related results and conclusions: a non-parametric KDE-based detector reproduces the lognormal-based P_D values to within |ΔP_D| ≤ 0.092 (median |ΔP_D| of 0.03), across all the 48 decision pairs (S2B Table). This is expected because detection performance at a fixed false alarm rate is governed by the ordering the decision statistic imposes on the observations — the ROC is invariant to strictly monotone transformations of that statistic [22,23] — so a good lognormal fit yields nearly the optimal ordering, and the KDE comparison confirms that P_D is essentially unchanged. Additionally, the framework was validated using a single pathway (TNF–NF-κB) with two time points; extending it to higher-dimensional readouts with more time points or multiple downstream reporters would further demonstrate its generality. Future work could apply this framework to other signaling pathways and disease-relevant contexts, such as characterizing how specific oncogenic mutations degrade signal detection, or integrating it with information-theoretic measures to provide a more complete picture of cellular decision-making fidelity.

The logic of the framework is independent of the number of jointly analyzed readouts N; what change with N are the computational and statistical costs. Computationally, the Gil-Pelaez lemma remains tractable for N ≥ 3 via the characteristic function [24], requiring an N × N eigen decomposition of the class covariance matrices Σ and Θ; this is numerically stable for N up to ≈ 10 with the covariance matrix condition numbers seen in our data (maximum 1.63; S3 Table). Anyway, the Monte-Carlo quantile estimation introduced for the mixture analysis avoids the eigen decomposition altogether. With regard to the statistical aspect, we note that reliable estimation of an N × N covariance matrix requires roughly n ≥ 10N samples [25]. With our per-condition sample sizes (n ≈ 137–671), N ≈ 5 is a safe number; regularized estimators such as Ledoit–Wolf shrinkage or the graphical lasso extend this to N ≈ 20–30 at modest sample sizes. These are the costs of scaling; the benefit side is equally quantifiable. Each additional readout carries an experimental cost — reporter lines, immunostaining channels, or sequencing depth — but because adding a readout can increase the information available to the detector, the framework can calculate the amount of P_D increase with N. This makes NP method a tool for experimental design: starting with a small N, one can compute the marginal increase in P_D at fixed P_FA from the (N + 1)-th readout and stop adding readouts once that gain falls below a chosen threshold.

These findings are also consistent with biological experiments, where A20 deficiency can potentially lead to chronic inflammation or even contribute to the development of cancer. Overall, the introduced decision-theoretic model is particularly suited to settings where the stimulus prior probabilities are unknown — often, though not always, the biological case; and whenever priors are known, it is complementary to a Bayesian/ML analysis, with which it shares the underlying likelihood ratio test.

Methods

Neyman-Pearson decision framework

To enhance comprehension of the proposed model in molecular biology and cellular decision-making processes, we applied statistical signal processing and decision theory approaches to analyze the TNF–NF-κB signaling system. A cell is able to decide whether the TNF concentration is low or high when exposed to varying levels of TNF as an input signal [5]. However, due to potential signaling abnormalities or noise in signal transduction, the cell may respond differently to the same input, leading to incorrect decision-making. To address this challenge, we pose a binary hypothesis testing problem in which the cell should determine the true hypothesis based on the TNF input level [7].

(1)

In the previous works by Habibi et al. [7], Ozen et al. [8], and Emadi et al. [9] the focus was on minimizing the average of the probabilities of the erroneous decisions that a cell can make, P_FA and P_M = 1 – P_D as the statistical metric of interest, where the first one corresponds to the probability of when a cell incorrectly determines that the input level of TNF is high when it is actually low, and the second one refers to the probability of when a cell overlooks a high level of TNF at the input. The optimal decision-making based on the ML approach needed a knowledge of the prior probabilities of the hypotheses H₀ and H₁, P(H₀) and P(H₁), as a prerequisite to determine the optimal decision threshold and compute the error probabilities. But typically, in cell signaling studies, such information is unavailable; consequently, they were fairly assumed equal, P(H₀) = P(H₁) = 0.5 in previous works [7–9]. But in this study, we introduce a new approach, based on the Neyman-Pearson theorem, as an alternative approach for cellular decision-making studies which does not ever rely on prior knowledge of the probabilities of the hypotheses H₀ and H₁, P(H₀) and P(H₁), and aims to maximize the probability of detection P_D, meaning the probability of identifying the presence of a signal when it does in fact exist [28].

Let P_FA, probability of false alarm, and P_D, probability of detection, be the metric decision probabilities of interest for a cell. P_FA is indeed the probability of deciding the high level of TNF at the input of the system while it is low (i.e., declaring H₁ while H₀ is true), and P_D is the probability of deciding the high level of TNF when in fact, it is high (i.e., declaring H₁ while H₁ is true), as shown below:

(2)

The optimal decision-making framework strives to maximize P_D for any given P_FA. To better understand how it works, let vector y, the N-element observation data (as discussed later, y is composed of NF-κB responses), be used to form the likelihood ratio L(y) in Equation(3). It declares hypothesis H₁ when the likelihood ratio L(y) is greater than the threshold , and declares H₀ otherwise, according to the Neyman-Pearson theorem [28–30].

(3)

To further analyze the performance of the proposed framework, we need to compute the optimal decision threshold. First step is to define the metric parameters of interest, probability of false alarm P_FA, and probability of detection P_D which are computed as follows:

(4)(5)

where p(y; H₀) and p(y; H₁) are the probability density functions (PDFs) of y under hypotheses H₀ and H₁, respectively. There is a specific value to be calculated for any given P_FA equal to in Equation (4) by which the maximized P_D can be obtained as shown in Equation (5). Upon obtaining these decision probabilities, we can also take advantage of a well-known graphical representation, the Receiver Operating Characteristic (ROC), to visualize the performance of the proposed decision-making framework (Fig 4).

The utilized experimental data were first modeled using lognormal PDFs, for mathematical convenience and also to gain insights. In other words, the natural logarithm of data, ln(data), was first considered to be Gaussian. The lognormal model was initially examined by formal goodness-of-fit analysis (Shapiro–Wilk, Anderson–Darling and Jarque–Bera tests; S2A Table) and quantile–quantile (Q–Q) plots (S8 Fig). Then the usefulness of the lognormal model in the detection context was verified by showing that a non-parametric kernel density estimator (KDE) applied to the raw single-cell data reproduced the lognormal P_D values to within |ΔP_D| ≤ 0.092 (median |ΔP_D| of 0.03), across all the 48 decision pairs (S2B Table). The multivariate Gaussian PDF of the measured response quantity of data, for y under the hypothesis H_i is shown below [28]:

(6)

where y represents the column vector of the ln(data), is the mean vector of y under hypothesis H_i and is the covariance matrix of y under hypothesis H_i. and demonstrate the determinant and inverse of the matrix , respectively. And ^T denotes the transpose operator.

As mentioned earlier, to assess the performance of the proposed framework, the optimal decision threshold and the probabilities of interest in Equations (4) and (5) need to be computed in the following subsections, in a simpler case of univariate, and the general multivariate case.

1. A. Univariate Decision Analysis

We use nuclear NF-κB responses of thousands of cells exposed to various TNF concentrations measured at 30 minutes and 4 hours after exposure [5]. For univariate analysis, we consider y = ln (nuclear NF-κB) for each time point, individually (for simplicity, we may drop the term “nuclear” in some instances). As mentioned earlier, a Gaussian PDF can well represent the data, so for the case of univariate analysis, when N = 1 in Equation (6):

(7)

Where is the mean and is the variance of data under i-th hypothesis H_i. Under H₀, TNF concentration levels 0.0021, 0.0052, and 0.013 ng/mL, and under H₁, TNF concentration levels of 0.2, 0.51, 1.3, and 50 ng/mL were used.

To analyze the univariate decision, first, we need to define the random variable Y as below:

(8)

Where stands for the normal (Gaussian) PDF. Equation (8)indicates that the random variable Y under hypothesis H₀ has a normal distribution with mean and variance . It also has a normal distribution with mean and variance , under hypothesis H₁. Then, the transformation is applied to make the normal distribution under H₀, a normal distribution of zero mean and unit variance. This transformation helps us in further simplification of equations and makes analytics more straightforward when it comes to the case of multivariate decision analysis. So, the hypothesis testing problem in Equation (8)for random variable Y turns into the new hypothesis testing problem of Equation (9)for X, which is actually the new transformed univariate random variable Y.

(9)

Where .

To find the test statistic Z (a statistic, as a function of the data), we need to form the likelihood ratio first. As mentioned above, the importance of the transformation above will become more apparent in the next subsection B, when we look at N > 1, where finding the PDF of the test statistic is rather complicated.

The optimal decision rule declares H₁, if:

(10)

to find the optimal decision threshold for x, and after further algebraic simplification of the Equation (10), we obtain the following quadratic equation:

(11)

so, using the Equation (11), the test statistic can be defined as below:

(12)

It is worth mentioning that the log-likelihood ratio for Gaussian PDFs is quadratic in the observation, the decision region is generally specified by its two roots, and consequently the Neyman-Pearson decision region can comprise two thresholds rather than a single threshold. We compute both roots of the quadratic equation and determine the decision region that attains the target P_FA. In practice, typically one threshold falls in a region of negligible probability mass, reducing the decision rule to a single effective threshold: most of the 48 (cell-type × time × H₀ × H₁) conditions follow this pattern (S4 Table), so the single-threshold approximation is numerically accurate and sufficient for our data. Anyway, for every condition, the reported detection probability is computed using the exact decision region obtained by Monte Carlo, so no single-threshold approximation is made. Also, for the bivariate case (N = 2), the L(x) = γ contour forms a continuous boundary in the two-dimensional log-response plane (see our previous study in [9]).

To obtain the optimal decision threshold Z_th from P_FA in Equation (4), we use the Cumulative Distribution Function (CDF) of the test statistic Z, F_Z (z):

(13)

So, the decision probabilities of interest in the Equations (4) and (5) can be calculated using Equation (13), as follows:

(14)(15)

where the F_Z (z; H_i) is the CDF of the test statistic Z given hypothesis H_i. As discussed earlier, for any given P_FA = α, the z_th can be obtained from the Equation (14), but first the according F_Z (z) needs to be calculated.

To calculate F_Z (z) through the Gil-Pelaez lemma [24,31], the characteristic function of the test statistic Z is required:

(16)

where the Im[x] is the imaginary part of x and j = .

As mentioned above, the characteristic functions of the test statistic Z are needed for computing P_FA and P_D. Then, under hypothesis H₀ and H₁ can be obtained as below:

(17)(18)

Here, E[x] denotes the expected value of x. This route is especially useful when N > 1. In principle, one could obtain F_Z(z) by integrating the PDF p_z(z), but for our test statistic Z, the PDF has no simple closed form when N > 1 (see subsection B). Instead, we compute F_Z(z) numerically by using the characteristic function from Equations (17) and (18) together with the Gil-Pelaez inversion in Equation (16). With F_Z(z) in hand, P_FA and P_D can be computed using Equations (14) and (15).

1. B. Multivariate Decision Analysis:

In the general case, we further expand the study to multivariate decision analysis when N > 1, e.g., when N = 2, the decision probabilities of interest when NF-κB responses are analyzed for the concurrent case of two different time points of 30 minutes and 4 hours after TNF exposure in the RESULTS section. To compute the P_FA and P_D for this case, we need to take all the steps of univariate decision analysis (subsection A). First, for the N-element vector Y below, we use the multivariate Gaussian PDF of Equation (6) to form the likelihood ratio for the hypothesis testing problem in Equation (19). So, for the multivariate random vector Y, we have:

(19)

where is the mean vector and is the covariance matrix under hypothesis H₀. Also, is the mean vector and represents the covariance matrix under hypothesis H₁. Note that these mean vectors and covariance matrices are the same as those in Equation (6), however they are introduced here to make the notation simpler for the multivariate formulation and analysis.

In the bivariate case considered here (N = 2), where the two components of Y are cell’s 30-minute and 4-hour readouts, both intrinsic noise and extrinsic noise — for example cell-to-cell variation at receptor or pathway-component levels — are absorbed into the estimated mean vectors and , and covariance matrices and , and therefore integrated into the decision test statistic. Each off-diagonal entry of each covariance matrix captures the correlation between a cell’s 30-minute and 4-hour responses — if extrinsic fluctuations induced such cross-time correlation, the bivariate likelihood ratio test would exploit it. In the present dataset, however, these two readouts come from independent fixed-cell populations with no tracking of individual cells across the two time points. For experiments that do track single cells over time, such as live-cell imaging, the developed decision test statistic is capable of capturing and taking into account any possible cross-time correlations.

As discussed earlier, to normalize the PDF under H₀, making it a zero-mean and unit-variance Gaussian PDF, we do a transformation that obviously affects the PDF under H₁ as well. First, we start with an eigenvalue decomposition of , the covariance matrix under hypothesis H₀ as follows [28]:

(20)

where U is the orthonormal matrix of eigenvectors, and D is the diagonal matrix of eigenvalues of the covariance matrix . The vector Y under the hypothesis H₀ is transformed through three steps; first, the mean is subtracted to obtain Y- ; second, elements of the zero-mean Y are de-correlated according to U^T(Y- ); and third, the de-correlated elements are made unit-variance by calculating where is the diagonal matrix of the square root of the inverse of the eigenvalues. Clearly, upon repeating all the steps mentioned above, the mean vector and covariance matrix of Y under the hypothesis H₁ will be transformed accordingly, as shown below:

(21)

where is the covariance matrix of under the hypothesis H₁ in Equation (21), which is determined by the two original covariance matrices. The next step would be applying another transformation to reach the new vector X, as the desired transformed version of Y. The new covariance matrix C of the transformed Y, under hypothesis H₁ can be decomposed into its eigenvector and eigenvalue matrices as where V is the orthonormal matrix of eigenvectors, and is the diagonal matrix of eigenvalues of the covariance matrix C. Then, X is constructed by de-correlating under the hypothesis H₁ in Equation (21) according to . Therefore, X as shown below, has zero-mean and unit-variance uncorrelated Gaussian variables under H₀, whereas under H₁ it has uncorrelated Gaussian variables whose means and variances are given by m and Λ, respectively:

(22)

where is the new mean vector and is the new covariance matrix under the hypothesis H₁ in Equation (22).

As mentioned earlier, such a transformation can help us in further analytical simplifications. The optimal likelihood-based decision framework for the vector Y in Equation (19) decides H₁ if:

(23)

But the optimal likelihood-based decision rule for X, the transformed variable Y, declares H₁, if:

(24)

Similar to the case of univariate decision analysis (subsection A), after some simplifications, the test statistic Z can be obtained as follows:

(25)

and the binary hypothesis testing problem for the multivariate decision-making framework can be defined below:

The next step is finding the PDF of the test statistic Z, then computing the decision probabilities P_FA and P_D. To do so, the characteristic function of the data under each hypothesis is needed. For the data under hypothesis H₀, we have:

(26)

and the characteristic function of the data hypothesis H₁, will be calculated as below:

(27)

According to Equations (3)-(5), and similar to univariate decision analysis, F_Z(z) can be numerically obtained using the characteristic function of Z in Equations (26) and (27), together with the Gil-Pelaez lemma. Eventually, we can find P_FA and P_D.

Here, in the case of N = 2, we consider two time points, 30 minutes and 4 hours, to be simultaneously considered as the two desired variables for analyzing the decision-making framework using the experimental data set. The eigenvalue decompositions of all possible conditions analyzed here are provided in the SUPPLEMENTARY MATERIALS section.

Confidence intervals on P_D were obtained by non-parametric bootstrap: within each condition the single cells were resampled with replacement (1000 resamples), the lognormal parameters refit, and P_D recomputed; we report the 2.5–97.5 percentile interval.

Also, this approach can be developed to larger cellular signaling networks, including different in puts like secondary messengers or ligands, and the output could be various transcription factors [32].

Optimal NP test for Gaussian-mixture hypotheses

In the mixed-population analysis (Figs 6 and 7), a fraction β of the cells are A20^−/− and the remaining 1 − β fraction are wild-type, so under each hypothesis the distribution of y = ln(nuclear NF-κB) is a two-component Gaussian mixture rather than a single Gaussian. With 𝒩(·; μ, σ²) being the Gaussian (normal) PDF introduced in Equation (7), the PDFs of the observations under the low- and high-TNF hypotheses are the two-component mixtures given in Equations (28) and (29), and the corresponding Neyman–Pearson-optimal test statistic is the log-likelihood ratio (LLR) of these two mixture PDFs, given in Equation (30):

(28)(29)(30)

In these equations, the subscripts W and A denote the wild-type and A20^−/− components, respectively, and the second subscript of 0 and 1 refer to the low and high TNF hypotheses, respectively. Unlike the single-Gaussian case this test statistic is not quadratic in the observation, so the characteristic-function (Gil-Pelaez) inversion used for the single-Gaussian case cannot be pursued in closed form here. We therefore set the threshold by Monte-Carlo: drawing 2 × 10⁵ samples from the H₀ mixture, evaluating ℓ_mix for each, and taking the 1 − P_FA empirical quantile as the threshold γ. The detection probability is the fraction of an equally large H₁ sample such that ℓ_mix ≥ γ_th. Comparing this GM detector with the single-Gaussian detector used for Figs 6 and 7 (S7 Fig, S5 Table) gives agreement within |ΔP_D| ≤ 0.017 at 30 minutes and ≤ 0.065 at 4 hours across all β, therefore, the single-Gaussian approximation closely tracks the optimal test while remaining analytically tractable.

Supporting information

Acknowledgments

NA