Skip to main content
Advertisement
Browse Subject Areas
?

Click through the PLOS taxonomy to find articles in your field.

For more information about PLOS Subject Areas, click here.

  • Loading metrics

CoRoPINN: Cognitive region optimized physics-informed neural networks

  • Jingcong Li,

    Roles Conceptualization, Methodology, Validation, Visualization, Writing – original draft, Writing – review & editing

    Affiliation College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China

  • Yu Liao ,

    Roles Conceptualization, Resources, Supervision, Writing – original draft, Writing – review & editing

    yliao@hbmzu.edu.cn

    Affiliations College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China, College of Electronics and Information Engineering, Sichuan University, Chengdu, China

  • Ximeng Wang,

    Roles Investigation, Resources, Validation, Writing – original draft

    Affiliation College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China

  • Jie Deng

    Roles Investigation, Methodology, Software

    Affiliation College of Intelligent Systems Science and Engineering, Hubei Minzu University, Enshi, China

Abstract

The solution of partial differential equations (PDEs) remains a significant problem in scientific computing. Although physics-informed neural networks (PINNs) provide a mesh-free paradigm, pointwise constraints offer limited supervision between collocation points and uniform training can under-resolve regions with greater learning difficulty. To address these coupled limitations, this paper introduces Cognitive Region Optimized Physics-Informed Neural Networks (CoRoPINN). Rather than treating region optimization and cognitive learning as independent additions, CoRoPINN forms a closed feedback loop. Gradient-stability statistics calibrate a globally stable baseline radius, PDE residuals estimate pointwise learning difficulty, and the same difficulty signal jointly adjusts sample-specific neighborhoods and cognitive weights. This coupling expands continuous physical constraints around under-resolved collocation points while progressively reallocating optimization from easier to more difficult regions. We evaluate CoRoPINN on three one-dimensional PDEs with five backbone architectures, Poisson equations in two, three, and five dimensions, and Burgers2D. Across the five backbones, CoRoPINN achieves the lowest geometric-mean relative mean absolute error (rMAE) on all three one-dimensional benchmarks, reducing the aggregate error relative to the second-best method by 21.0%, 61.6%, and 52.9% on 1D-Reaction, 1D-Wave, and Convection, respectively. It also ranks first in both metrics for 2D and 5D Poisson, remains second in 3D, and attains the lowest rMAE and relative root mean square error (rMSE) among all four methods on Burgers2D. These results establish a clear overall accuracy advantage for CoRoPINN among the evaluated baselines.

Introduction

Partial differential equations (PDEs) play a pivotal role in the mathematical modeling of complex phenomena across various disciplines, including physics, chemistry, biology, and financial engineering. The numerical solution of PDEs presents a fundamental challenge in scientific computing [13]. Traditional numerical methods, such as the finite element method [4], finite difference method [5], and finite volume method [6], have matured and become the standard approaches. However, these methods primarily rely on grid discretization, which poses significant challenges, including high computational costs and the curse of dimensionality when dealing with high-dimensional problems, complex geometric boundaries, or the resolution of inverse problems [7,8].

In recent years, PINNs have emerged as a mesh-free deep learning framework, offering a novel paradigm for solving PDEs [9]. PINNs embed governing equations, initial conditions, and boundary conditions into the neural network’s optimization objective as residual losses. By leveraging automatic differentiation techniques to compute partial derivatives, they encourage the neural network to satisfy physical laws while fitting data. This integration of physical prior knowledge with data-driven approaches demonstrates immense potential for solving direct and inverse problems in data-scarce scenarios [8,10].

The success of PINNs has prompted refinements along four complementary directions. The first direction focuses on network architectures and seeks to improve expressiveness or computational efficiency. The separable physics-informed neural network (SPINN) [11] processes coordinate dimensions independently to reduce the cost of high-dimensional PDEs, whereas the deep fuzzy physics-informed neural network (FPINN) [12] introduces fuzzy-logic layers to improve robustness to data uncertainty.

The second direction focuses on optimization strategies and seeks to make the PINN loss landscape easier to train. The gradient-enhanced physics-informed neural network (gPINN) [13] regularizes residual gradients, while the scale-invariant MultiAdam optimizer [14] balances the optimization rates of heterogeneous loss terms.

The third direction focuses on balancing the multiple objectives in the PINN loss. Neural tangent kernel (NTK) weighting [15] estimates the relative importance of individual loss terms, whereas self-adaptive loss-balanced PINN [16] uses convergence status and uncertainty to update the weights.

The fourth direction focuses on changing the training paradigm itself. The variational physics-informed neural network (vPINN) [17] expresses PDE constraints in integral form, residual-based adaptive refinement (RAR) [18] adds points where PDE residuals are large, and physics-informed generative adversarial networks (PI-GANs) [19] combine physical constraints with adversarial training.

Despite this progress, two interconnected limitations remain. First, conventional PINNs impose PDE constraints at finite, discrete points, although the target equations are defined over continuous domains [9,20]. This spatial mismatch weakens supervision between collocation points and impairs the representation of local fine-scale structures (Fig 1). Second, learning difficulty varies substantially across the domain, but conventional training assigns equal importance to all samples [21]. The resulting Unbalanced Prediction Problem (UPP) causes optimization to favor smooth or simple regions while under-resolving challenging structures such as boundary layers. Existing approaches address parts of these limitations, but they do not jointly coordinate continuous neighborhood constraints, pointwise difficulty, and training-stage progression within one feedback loop.

To address these challenges, two distinct approaches to expanding training paradigms have emerged (Fig 2). The region-optimized physics-informed neural network (RoPINN) [20] adopts a spatial continuity perspective, effectively enhancing local constraints and improving generalization capabilities by extending the optimization domain from discrete points to their continuous neighborhoods. Conversely, the cognitive physics-informed neural network (CoPINN) [21] focuses on temporal dynamics and introduces a self-paced learning mechanism that dynamically evaluates sample difficulty and adjusts learning weights, thereby effectively mitigating the UPP [22].

Existing adaptive PINN methods differ in both their optimization units and their adaptation targets. Residual-based adaptive refinement (RAR) reallocates discrete collocation points according to residual magnitude, but its optimization unit remains an isolated point [18]. RoPINN expands each point into a neighborhood and calibrates a shared region scale through gradient stability, but it does not explicitly model pointwise difficulty or training-stage progression [20]. CoPINN schedules pointwise loss weights from easy to hard, but it retains point-based constraints [21]. CoRoPINN differs by coupling a globally calibrated stable radius with residual-based sample difficulty, point-specific regions, and progressive cognitive weights in one feedback loop. This coupling links spatial continuity with temporal curriculum learning instead of treating region optimization and cognitive weighting as independent additions.

Accordingly, this paper introduces Cognitive Region Optimized Physics-Informed Neural Networks (CoRoPINN). Its central advantage targets PDEs whose learning difficulty is distributed unevenly over the solution domain. Difficulty-adaptive neighborhoods strengthen continuous supervision around under-resolved collocation points, while cognitive weights redirect optimization toward those regions as training progresses. The present benchmarks evaluate this capability through local fine-scale structures, sparse coverage in higher dimensions, and coupled nonlinear dynamics. Within this evaluated scope, the closed-loop coordination addresses limitations that neither region optimization nor pointwise cognitive weighting resolves alone.

The main contributions of this paper are as follows:

  • We develop a cognition-guided region optimization mechanism that links global trust-region calibration, sample-specific residual difficulty, adaptive region sampling, and progressive cognitive weighting within one training loop.
  • We formulate a dual radius update that first identifies a globally stable region scale and then adapts each sampling neighborhood to local learning difficulty.
  • Across five backbone architectures, CoRoPINN achieves the lowest geometric-mean rMAE on all three one-dimensional benchmarks. Under the stated loss-ranking rule, it leads or ties in 37 of 45 reported metric cells and also attains the lowest rMAE and rMSE on Burgers2D.

Preliminaries

PDE definition

Consider a system of partial differential equations that incorporates equation constraints, boundary conditions, and initial conditions. This can be formalized as follows:

(1)

where , , and denote the governing PDE, boundary-condition, and initial-condition operators, respectively [1]. The symbols , , and denote the spatiotemporal domain, its boundary, and the initial-time domain at t = 0, respectively. The target solution is , where d is the number of spatial dimensions and m is the number of solution components. The coordinate combines spatial position and time, i.e., , where t denotes time.

PINNs have been demonstrated to approximate the true solution through neural network parameterization, with the training objective being to minimize the loss function [9].

Region optimization method

Region optimization is a training paradigm that extends the PINN optimization process from discrete sampling points to continuous neighborhoods. Conventional PINNs impose PDE constraints solely at finite sampling points, whereas region optimization theoretically reduces generalization error and enhances constraint satisfaction by applying continuous constraints within the neighborhood of each sampling point [20]. Region optimization comprises two iterative steps: Monte Carlo approximation and trust-region correction. The former effectively approximates the optimization objective, while the latter efficiently controls estimation error.

Region optimization definition.

For a set of sampling points S, with |S| denoting its cardinality, region optimization extends each point to its neighborhood , where r is the region radius and d denotes the spatial dimension. The region loss function can be defined as:

(2)

where denotes the point optimization loss at x, is the neural network parameterized by , is the measure (volume) of the extended neighborhood, and is an offset within that neighborhood. This definition extends the original point constraint to a region integral constraint, requiring the network to satisfy the PDE not only at the sampling point but also maintain physical consistency within the surrounding neighborhood.

Monte Carlo approximation.

Since the region integral in Eq (2) cannot be computed directly, region optimization employs Monte Carlo approximation. At each iteration, an offset is uniformly sampled from the neighborhood . The region loss gradient is approximated using the expected loss gradient at the sampled point [23], as follows:

(3)

Here, denotes the uniform distribution over , denotes expectation with respect to , and denotes the gradient with respect to the network parameters .

This approximation method has been demonstrated to efficiently implement region optimization while implicitly introducing a higher-order regularization effect [20]. The application of Taylor expansion analysis reveals that Monte Carlo sampling is equivalent to constraining the original loss and its first-order derivative terms. Such a constraint aids in suppressing unphysical oscillations in the solution, making it particularly suitable for PDE problems characterized by drastic variations.

Trust region calibration.

To prevent excessive sampling areas from causing optimization instability, region optimization employs a trust region calibration mechanism to adjust region size dynamically. The trust region is defined as the input domain area characterized by low gradient variance, where optimization is more stable and reliable [24]. The region radius r is dynamically adjusted based on gradient variance:

(4)

where denotes the gradient variance when uniformly sampling within the neighborhood . By approximating the variance through multiple consecutive iterations, the system adaptively adjusts the radius to balance generalization performance and optimization stability. In scenarios where the gradient variance is high, the region radius automatically shrinks to ensure convergence stability. Conversely, when optimization is stable, a larger region is permitted, thus enhancing generalization capability. This mechanism ensures stronger local constraints while maintaining training stability. The implementation-specific statistic, initialization, update frequency, and clipping controls used by CoRoPINN are specified in the later Trust region evaluation and Difficulty-based region adaptation subsections.

Cognitive learning method

Cognitive learning is a machine learning training paradigm that emulates human learning processes. Its fundamental principle is the dynamic adjustment of learning strategies based on the difficulty level of learning materials [22]. In the domain of physical information, neural networks employ cognitive learning methods to facilitate a progressive training process, ranging from simplicity to complexity, by discerning variations in learning difficulty across different regions.

Difficulty evaluation in CoPINN.

The primary task for cognitive learning to achieve an easy-to-difficult progression involves accurately assessing the learning difficulty across various spatial locations. Difficulty levels are typically determined based on predefined heuristic rules, such as training loss or specific statistical properties of the data. In CoPINN, the difficulty metric for sampling points at the t-th iteration is defined by the gradient norm of the PDE loss as:

(5)

Here, i indexes a sampling point, t indexes the training iteration, is the PDE residual loss at iteration t, and is the Euclidean norm. This metric dynamically reflects the difficulty for the network to satisfy physical constraints at any given point. It does not rely on static, predefined difficulty measures but can continuously adjust its focus during training, emphasizing more challenging regions. A larger gradient norm indicates greater difficulty in satisfying the physical equations at that location. This phenomenon is often associated with boundary layers and other regions characterized by rapid changes [25,26].

Cognitive weight allocation.

Based on difficulty assessment results, the core task of cognitive learning is to dynamically assign evolving weights to different samples over time. In the early training phase, the differences in weights are maximal. At this stage, the network’s capacity to adapt to challenging regions is weak. The system assigns higher weights to simple samples to rapidly establish a stable, correct global solution structure. During the mid-training phase, weight differences reach a minimum. At this point, the model engages in comprehensive, balanced learning, consolidating existing knowledge while preliminarily exploring difficult areas to prevent premature convergence to local optima. In the late training phase, weight differences gradually widen again but in the opposite direction, shifting focus toward challenging samples. This concentrates optimization resources on overcoming residual local difficulties, achieving refined fitting [21]. The dynamic weight adjustment mechanism ensures a learning process that is both robust and efficient, effectively avoiding initial instability and late-stage underfitting.

Method

Problem analysis and motivation

PINNs are confronted with two fundamental challenges when attempting to solve PDEs: the local constraints of point optimization and the imbalance in spatial learning.

Limitations of point optimization.

Conventional PINNs minimize PDE residuals based on discrete sampling points, with the loss function defined as:

(6)

Here, , , and denote the PDE residual, initial-condition, and boundary-condition losses, respectively, while , , and are their nonnegative weighting coefficients.

The PDE loss term is defined as:

(7)

Here, is the number of sampling points in , is the PDE operator, and is a discrete point. This point-based optimization imposes constraints solely on a finite set of discrete points, which results in insufficient supervision between these sampling points. This issue is particularly pronounced for PDEs with highly variable solutions, where the local nature of point constraints can lead to significant generalization problems. It has been demonstrated that the network may exhibit minimal residuals at the sampling points, while simultaneously generating nonphysical oscillations in the intervals between these points [2729].

Unbalanced prediction problem (UPP).

An analysis of error distributions during training reveals a severe spatial imbalance in PINNs when solving PDEs [21,29]. Notable disparities in learning difficulty have been observed across various spatial regions. In areas such as boundary regions and points of solution discontinuity, the residuals of PDEs are typically more than an order of magnitude higher than those in smooth internal regions. This phenomenon arises from drastic changes in physical quantities, leading to a highly uneven spatial distribution of prediction accuracy [15].

This imbalance stems from conventional PINN methods that apply identical optimization strategies across all training samples, neglecting the fundamental differences in physical properties and learning difficulty between regions. Since samples from simpler regions typically dominate the training process and gradient updates converge easily, uniform weight training strategies result in an overemphasis on these simpler regions. Consequently, this traps the model in local minima, hindering effective learning in more challenging regions that are critical to overall accuracy, thereby inducing the UPP.

Motivation analysis.

Region optimization has been shown to enhance generalization by expanding the constraint domain. However, this approach does not address imbalances in spatial learning. In contrast, cognitive learning can accommodate variations in difficulty through dynamic weight adjustments, yet it remains limited by the point optimization framework. These approaches are inherently complementary, enhancing PINNs from the perspectives of spatial continuity and temporal dynamics. The proposed CoRoPINN algorithm seamlessly integrates the strengths of both approaches. By concurrently adjusting the constraint scope and optimizing weights, the model adaptively allocates computational resources and constraint strength according to regional characteristics. This dual-adaptive mechanism facilitates customized treatment for regions with varying difficulty levels. Additionally, it establishes a dynamic equilibrium between local accuracy and global convergence, thus addressing the fundamental issues of locality and imbalance.

Method framework

Fig 3 presents the end-to-end workflow of the tightly coupled cognitive-regional optimization loop. The workflow contains five stages. First, trust-region calibration evaluates optimization stability from the historical variation of network-parameter gradients and determines a stable baseline radius. Second, the PDE residual magnitude is used to evaluate the difficulty of each sample. Third, the raw and batch-normalized difficulty values adapt the pointwise sampling radius and cognitive weight, respectively. Fourth, Monte Carlo sampling is performed within each adaptive region, and the resulting residuals are combined into a cognitively weighted loss. Fifth, the limited-memory Broyden–Fletcher–Goldfarb–Shanno (L-BFGS) optimizer updates the network parameters [30], and the new gradient information is returned to the calibration module for the next iteration. These stages form a closed “calibration–evaluation–adaptation–sampling–feedback” loop.

thumbnail
Fig 3. Overall workflow of CoRoPINN.

The diagram shows the five-stage cognitive-regional optimization loop and the gradient-feedback path used to update trust-region calibration.

https://doi.org/10.1371/journal.pone.0358646.g003

Theory analysis

Generalization error bound.

CoRoPINN effectively reduces generalization error by extending constraints from discrete points to continuous neighborhoods through region optimization. First, we define the model’s expected generalization error as [27]:

(8)

where denotes the expected generalization error, and denotes expectation over the sampled set S and any randomness in optimization algorithm A. The notation A(S) represents the parameters returned by A after training on S, and is the corresponding model.

For positive constants L and , assume that the point optimization loss is L-Lipschitz and -smooth with respect to the model parameters. For arbitrary parameter vectors and and every sample point x in the global domain , the following inequality holds [20]:

(9)

Assuming is L-Lipschitz and -smooth with respect to , performing T steps of stochastic gradient descent with step size at step t yields the following conclusion [27,28]:

If is convex for and , then holds.

If is bounded by a constant C for all and is non-convex for with step size monotonically non-increasing , then holds.

Moreover, if is bounded and convex for all and x, and is L-Lipschitz and -smooth with respect to the model parameters , then the region optimization loss function is also bounded and convex for all and x, and is L-Lipschitz and -smooth with respect to [20].

We assume is L-Lipschitz and -smooth with respect to . Then, performing T steps of stochastic gradient descent on with step size at step t yields the following conclusion [20]:

If is convex for and , then holds.

If is bounded by a constant C for all and is non-convex for with step size monotonically non-increasing , then holds, where J is a finite number determined by behavior in the early training phase.

It can be observed that, in comparison to point optimization, the generalization error bounds of region optimization introduce a negative term in both convex and non-convex scenarios, effectively reducing generalization error. This finding indicates that the region optimization paradigm offers a more consistent gradient optimization direction than point optimization at each iteration, thereby enhancing generalization performance.

Implicit regularization.

In solving PDEs, traditional point optimization imposes constraints on the values of derivative terms at specific points, resulting in significant locality concerns. In contrast, region optimization implicitly achieves high-order regularization effects within neighborhoods through Monte Carlo sampling. This approach is analogous to the concept of region constraints in Taylor expansion [13,20,31]:

(10)

where defines the first-order loss function, and represents the truncation error of the Taylor expansion, encompassing all second-order and higher-order terms.

In cognitive region optimization, the constraint strength for higher-order derivatives is adjusted through difficulty adaptation. The globally calibrated baseline radius is expanded for points with larger residual difficulty and is bounded by the pointwise clipping rule defined below. This mechanism strengthens neighborhood constraints in challenging regions while retaining the stability imposed by trust-region calibration.

Convergence rate.

We give a first-order stationarity guarantee for a theoretical stochastic-gradient update of the time-varying CoRoPINN objective [20,32]. Let

(11)

where and are the cognitive weight and pointwise region radius at iteration t, respectively. Assume that (i) each is lower bounded by and is -smooth; (ii) the weights and radii are bounded; (iii) the Monte Carlo gradient is conditionally unbiased, , with ; and (iv) a finite bound independent of T uniformly controls the cumulative objective drift:

(12)

Proposition. For the theoretical update with and , let R be sampled uniformly from . Then

(13)

Proof. By -smoothness, conditional unbiasedness, and the variance bound,

(14)

Since , the gradient coefficient is at least . Replacing by incurs at most . Summing over t, applying and , and dividing by gives the result. This follows the RoPINN expected-descent structure [20], with the drift term accounting for CoRoPINN’s adaptive weights and radii.

Cognitive region optimization

The cognitive region optimization framework constitutes the primary innovation of CoRoPINN. The integration of cognitive learning strategies with region optimization methods enables adaptive optimization across regions exhibiting varying degrees of difficulty.

Optimization objective.

The objective function of CoRoPINN integrates cognitive weight and region loss, as defined below:

(15)

where i indexes the point , M represents the set of Monte Carlo samples for each region, |M| is the number of such samples, and denotes the offset of the m-th sample. For the three one-dimensional experiments, define the training-progress factor and the easy/hard schedules as

(16)

Using the normalized residual difficulty defined below, the interpolated and implemented cognitive weights are [21]

(17)

Thus, and are scalar base weights that respectively decrease and increase during warm-up. The coefficient scales the cognitive deviation from unit weight. The same coefficient also scales the pointwise region-radius correction below.

This objective function adjusts the contribution of samples with varying difficulty through cognitive weights, simultaneously enhancing local constraints through region sampling.

Cognitive difficulty evaluation in CoRoPINN.

Conventional PINN methods apply a uniform optimization strategy to all training samples, thus disregarding variations in learning complexity across regions. To explicitly capture such heterogeneity, CoRoPINN introduces a dynamic difficulty assessment mechanism based on PDE residuals [26].

Let denote the collocation batch at training step t. CoRoPINN measures the difficulty of using the absolute PDE residual:

(18)

where denotes the network parameters and denotes the PDE residual operator evaluated at step t. Thus, is the residual magnitude rather than an input-gradient norm. A larger value indicates that the current network violates the governing equation more strongly at that location.

For cognitive weighting, the residual magnitudes are normalized within each batch:

(19)

where and . The stabilizer avoids division by zero, and provides a common scale for the cognitive weights.

The implementation also records the batch-average difficulty . For the most recent evaluations, the linear trend is obtained by least-squares fitting:

(20)

Here, q indexes the retained evaluations, while and are the fitted slope and intercept, respectively. The sign of indicates whether the average residual difficulty is rising or falling. This trend is used as a bounded correction to the global trust-region statistic defined below.

Trust region evaluation.

To prevent excessive regions from destabilizing optimization, CoRoPINN first determines a globally stable baseline radius. Let be the number of optimizer closure evaluations at step t, and let P be the number of flattened trainable parameters. The step gradient is

(21)

Here, is the full closure-k loss at outer step t (weighted PDE, plus applicable boundary/initial terms), and is the mean flattened closure gradient. For the history window , the implementation calculates the mean coefficient of gradient variation:

(22)

where is the retained gradient-history length, p indexes a flattened parameter component, and denote the standard deviation and mean over , and is a numerical stabilizer. The difficulty-trend correction and calibrated statistic are

(23)(24)

Here, is the difficulty-trend correction, is its bounded amplitude, is the sign function, and is the calibrated gradient-variation statistic. If , it is set to 1 for numerical stability, as in the implementation. The statistic computed after step t calibrates the baseline radius used at step t + 1:

(25)

where r0 is the user-specified initial region size and restricts its first argument to the stated lower and upper bounds. The first step uses the initialized calibration value . A larger gradient variation therefore contracts the baseline region, whereas a stable gradient history permits a larger region within the stated bound.

Difficulty-based region adaptation.

After global calibration, CoRoPINN adapts the baseline radius to each collocation point using its raw residual difficulty:

(26)

where is the cognitive difficulty coefficient used by the implementation. It is distinct from the smoothness constant in the theoretical analysis. The same scales both the cognitive weight and the pointwise radius: the raw value controls the latter, whereas controls the former. For each point, M Monte Carlo offsets are sampled as

(27)

and the governing-equation loss is evaluated at . The clipping interval prevents the pointwise correction from changing the calibrated baseline by more than a factor of two.

Together, these equations specify the update order. CoRoPINN begins step t with the baseline radius calibrated from the previous gradient history, evaluates the current residual difficulty, adapts each pointwise radius, samples its neighborhood, and updates the network. The resulting gradients and average difficulty are then stored to calibrate the radius for step t + 1. This “calibration–evaluation–adaptation–sampling–feedback” sequence couples global optimization stability with local learning difficulty without altering the underlying PDE loss.

At each outer L-BFGS step, remains fixed across all closure evaluations. Each closure recomputes and from the current network state. After the optimizer step, the closure gradients are averaged to obtain , and the retained gradient history and difficulty trend are updated. The resulting statistic then calibrates . Thus, global calibration occurs once per outer step, whereas pointwise adaptation follows the current residual during each closure evaluation.

Progressive learning strategy.

To mitigate the instability that may arise from prematurely concentrating on challenging samples during the initial stages of training [33], CoRoPINN employs a progressive learning strategy governed by above, where Twarmup denotes the warm-up period. During the warm-up phase (t < Twarmup), the model prioritizes learning from simpler samples to establish a stable foundational solution structure. As training progresses to , the system gradually shifts its focus toward more challenging samples, thereby facilitating a natural transition from easy to hard.

Cognitive region sampling.

Based on the above mechanism, CoRoPINN employs a stratified region sampling strategy. The system continuously evaluates sample difficulty and tracks its evolving trends. For regions demonstrating persistent escalation in difficulty, the system not only expands the sampling scope through adaptive mechanisms but also augments their contribution to the loss function via cognitive weighting. Conversely, in regions where difficulty is reduced, the system dynamically reduces resource allocation. This cognition-guided sampling strategy ensures that computational resources remain concentrated on areas most in need of improvement, thus enhancing prediction accuracy while maintaining training efficiency.

Experiments

Experimental setup

Benchmarks.

To comprehensively validate the effectiveness of CoRoPINN, we select three representative one-dimensional time-dependent problems, three multidimensional Poisson problems, and Burgers2D as benchmarks.

The 1D-Reaction problem describes nonlinear reaction dynamics, the 1D-Wave problem represents hyperbolic wave propagation, and the Convection problem tests high-frequency transport. Together, they expose distinct optimization difficulties, including established failure modes of conventional PINNs [20,29,34].

Burgers2D complements these problems with coupled nonlinear transport–diffusion dynamics in two spatial dimensions and time [35].

The Poisson benchmarks are second-order elliptic equations on with the manufactured solution . Increasing the dimension from 2D to 5D makes a fixed number of randomly sampled collocation points progressively sparser in the domain. The Poisson-5D setting uses 5000 interior points and therefore provides an explicit higher-dimensional sparse-coverage test. Table 1 summarizes the dimensionality, differential order, and principal characteristic evaluated by each benchmark.

thumbnail
Table 1. Summary of the benchmark problems used in this study.

https://doi.org/10.1371/journal.pone.0358646.t001

Base models and baselines.

To validate the broad applicability of CoRoPINN, five distinct neural network architectures were employed as baseline models. These included the canonical PINN [9], the quadratic residual network (QRes) [36], FLS (a sinusoidal-space PINN) [37], PINNsFormer (a Transformer-based PINN framework) [38], and the Kolmogorov–Arnold network (KAN) [39]. Based on these network architectures, we conducted comparative experiments between CoRoPINN and five baseline methods. Vanilla PINN represents the conventional physics-informed neural network algorithm, employing uniform sampling and standard loss functions. gPINN [13] enhances gradient information based on variational principles. vPINN [17] transforms point constraints into integral constraints using a variational formulation. RoPINN implements a region optimization strategy, employing Monte Carlo region sampling to expand the constraint domain. CoPINN employs a cognitive learning mechanism to dynamically adjust sample weights, thereby enabling progressive training from easy to difficult. CoRoPINN, the method proposed in this paper, integrates region optimization with cognitive learning to achieve dual spatial and temporal adaptive optimization.

Training configuration.

Training data are generated within each solution domain from the governing equation, boundary conditions, and initial conditions. For the three one-dimensional time-dependent problems, a uniform space–time grid provides residual, boundary, and initial points. The Poisson experiments use randomly sampled interior and boundary points. The Burgers2D experiment uses randomly sampled interior, initial-condition, and periodic-boundary points. Table 2 summarizes the backbone configurations used in the comparisons.

thumbnail
Table 2. Neural network architectures used in the experiments.

https://doi.org/10.1371/journal.pone.0358646.t002

All models are trained in PyTorch [40] for 1000 outer optimization epochs. Optimization uses L-BFGS with a Strong-Wolfe line search. The initial learning-rate parameter (step size) is 1.0, and subsequent step sizes are selected by the line search.

Evaluation metrics.

Model performance is evaluated using three complementary metrics. The “Loss” metric assesses adherence to the physical constraints and is calculated as a weighted sum of the PDE residual, boundary-condition, and initial-condition losses. Relative mean absolute error (rMAE) denotes the average absolute deviation between the predicted and true values, normalized to the range of the true solution. This metric is robust to outliers and readily interpretable. Relative root mean square error (rMSE) measures the second-order moment of the prediction errors, making it more sensitive to large errors and informative about prediction quality in critical regions [41,42].

Results

Comparison with standard and adaptive PINN baselines.

In this section, we compare six PINN training methods across fifteen backbone–PDE configurations, comprising five base architectures and three PDE problems. In addition to Vanilla PINN, the comparison includes gPINN, vPINN, RoPINN, and CoPINN. RoPINN and CoPINN are the most direct adaptive baselines because they isolate region optimization and cognitive weighting, respectively. The resulting matrix assesses CoRoPINN’s relative gains across backbone architectures and PDEs.

Table 3 presents a comprehensive comparison of CoRoPINN versus baseline methods across multiple typical PDE problems. Lower values for the rMAE, rMSE, and Loss metrics indicate superior model performance. Furthermore, to clearly present the results of the model comparisons, the optimal model results under the same experimental configuration are displayed in bold, while the second-best model results are shown in italics with underlines. The “Promotion” row records the percentage improvement or degradation of CoRoPINN in performance metrics relative to the best-performing model among the baselines.

thumbnail
Table 3. Comparison between CoRoPINN and standard or adaptive PINN baselines under different base models.

https://doi.org/10.1371/journal.pone.0358646.t003

To compare the six methods across backbone architectures at a glance, we aggregate the positive rMAE values in Table 3 using the geometric mean

(28)

This ratio-scale summary avoids allowing a single result from a different error magnitude to dominate the cross-backbone comparison. Fig 4 shows that CoRoPINN achieves the lowest aggregate rMAE on all three problems: 0.012913 on 1D-Reaction, 0.037698 on 1D-Wave, and 0.074009 on Convection. Relative to the second-best aggregate in each problem, these values are lower by 21.0%, 61.6%, and 52.9%, respectively. The gPINN result for 1D-Wave is computed from four successful backbones because its PINNsFormer configuration was out of memory (OOM). No value is imputed for that configuration.

thumbnail
Fig 4. All-method comparison of geometric-mean rMAE across the five backbone architectures in Table 3.

Lower values are better. The gPINN result for 1D-Wave uses the four successful configurations (n = 4) after excluding the PINNsFormer OOM entry. All other bars use n = 5. The rMAE values are shown on a logarithmic y-axis.

https://doi.org/10.1371/journal.pone.0358646.g004

Under the PINN backbone, the direct adaptive-baseline comparison is consistent across the three PDEs. For 1D-Reaction, CoRoPINN obtains an rMAE of 0.025, compared with 0.056 for RoPINN and 0.038 for CoPINN. For 1D-Wave, the corresponding values are 0.014, 0.063, and 0.039. For Convection, they are 0.585, 0.635, and 0.624. These results demonstrate consistent gains under the canonical backbone across all three PDEs.

It is noteworthy that CoPINN, as a baseline method for cognitive learning, generally exhibits lower loss metrics. This phenomenon can be attributed to a global coefficient in its loss function that decays linearly toward zero, which can compress late-stage loss values without implying superior prediction accuracy. In contrast, CoRoPINN progressively redistributes relative emphasis between easy and difficult samples while retaining the bounded affine weight range . This bounded rescaling avoids the global decay-to-zero behavior and reduces the resulting loss-scale bias. Because CoPINN loss values are therefore not directly comparable with those of the other methods, we exclude CoPINN from the Loss ranking. Under this stated rule, Table 3 shows that CoRoPINN is best or tied for best in 13 of 15 Loss cells and 24 of 30 rMAE/rMSE cells. CoRoPINN therefore leads or ties in 37 of the 45 reported metric cells across the evaluated settings.

Error distribution visualization.

This section evaluates the error distribution characteristics of CoRoPINN and baseline methods in solving PDEs. To achieve this, we systematically visualize the error distributions across four methods (PINN, RoPINN, CoPINN, CoRoPINN) for three problems (1D-Reaction, 1D-Wave, Convection). Our examination of the spatial distribution of absolute errors on both linear and logarithmic scales aims to reveal the differences in the methods’ capabilities to control local error maxima and enhance the overall smoothness of solutions. The central objective of this study is to determine whether CoRoPINN more effectively suppresses the UPP through the synergistic effects of region optimization and cognitive learning.

Fig 5(a) presents a heatmap of absolute error distributions for the four methods on a linear scale, utilizing color gradients to visually represent the magnitude of errors. On this scale, the conventional PINN demonstrates the most significant fluctuations in error, characterized by substantial error ranges and notable clustering. RoPINN demonstrates distinct edge error clustering, manifesting as clear stripes in the boundary regions. CoPINN exhibits relatively uniform overall error distribution but still shows visible error fluctuations and localized high-error regions. In contrast, CoRoPINN demonstrates the most uniform error distribution, with the most consistent global color tone.

thumbnail
Fig 5. Absolute error distribution heat maps on different PDEs and diverse base models.

(a) Results on the linear scale. (b) Results on the logarithmic scale.

https://doi.org/10.1371/journal.pone.0358646.g005

Fig 5(b) presents a replot of the spatial distribution of absolute error on a logarithmic scale. This scaling method enhances the contrast in regions with minimal error, thereby facilitating a more precise evaluation of subtle variations in approximating the exact solution. CoRoPINN exhibits superior spatial uniformity in error distribution and reduced overall error levels, notably mitigating error concentration in boundary areas.

The visualization of error distribution indicates that CoRoPINN exhibits superior spatial uniformity in both linear and logarithmic scales, providing qualitative support for the effectiveness of region optimization and cognitive learning mechanisms in improving error distribution.

Prediction results visualization.

To illustrate the performance of CoRoPINN, Fig 6 compares predictions obtained with the conventional PINN backbone on the 1D-Wave problem. The figure presents spatiotemporal heatmaps of the exact solution, RoPINN prediction, and CoRoPINN prediction, together with the loss curves for the two methods.

thumbnail
Fig 6. Comparison between CoRoPINN and RoPINN on the 1D-Wave equation using the PINN backbone.

(a) Exact solution of the 1D-Wave equation. (b) RoPINN prediction and loss curves. (c) CoRoPINN prediction and loss curves.

https://doi.org/10.1371/journal.pone.0358646.g006

The visualization shows that the CoRoPINN prediction heatmap closely follows the exact solution and captures the propagation and boundary-reflection patterns of the wave equation. RoPINN reduces the limitations of point optimization through region optimization but retains visible prediction deviations. The loss curves provide a consistent quantitative comparison: CoRoPINN attains a 90% lower final loss than RoPINN, and all three loss components reach lower final values.

Notably, the loss curve of CoRoPINN exhibits a more stable descent trend, particularly during the mid-to-late training stages. Boundary loss and initial condition loss continue to decrease steadily, while RoPINN experiences significant stagnation in convergence during the same period. This phenomenon validates the effectiveness of incorporating cognitive learning mechanisms. By progressively shifting the focus of optimization from simple regions to challenging boundary areas, CoRoPINN avoids local optima traps during training and achieves comprehensive learning of critical regions such as boundaries.

Multidimensional PDE results.

To evaluate the generalization potential of CoRoPINN beyond the one-dimensional benchmarks, we include Poisson equations in 2D, 3D, and 5D. We further compare the four methods on Burgers2D using a shared PINN backbone and training protocol. The 5D case provides a direct higher-dimensional evaluation. The optimal result for each problem is highlighted in bold, while the second-best result is italicized and underlined.

As shown in Table 4, CoRoPINN demonstrates a clear overall advantage across the evaluated 2D, 3D, and 5D Poisson equations. In 2D, it ranks first in both metrics, reducing rMAE and rMSE by 29.0% and 41.2%, respectively, relative to the second-best CoPINN results. In 3D, CoRoPINN ranks second in both metrics and remains more accurate than Vanilla PINN and RoPINN. In the 5D case, it ranks first again and reduces rMAE by approximately 25% and 32% relative to RoPINN and CoPINN, respectively. Recovering the lead in 5D, where a fixed collocation budget leaves progressively sparser domain coverage, highlights CoRoPINN’s ability to use difficulty-adaptive neighborhoods to extract more information from limited collocation points. The coupled mechanism is therefore not tied to a single dimensional setting: it remains competitive in 3D and becomes most advantageous when sparse coverage makes isolated point constraints less effective.

thumbnail
Table 4. Results of CoRoPINN and comparison algorithms on multidimensional PDEs using the PINN backbone.

https://doi.org/10.1371/journal.pone.0358646.t004

On Burgers2D, CoRoPINN attains the lowest rMAE (0.481) and rMSE (0.531) among the four methods. These values are 1.2% and 2.0% lower, respectively, than the corresponding second-best results. This extends the observed accuracy advantage to a coupled nonlinear transport–diffusion system and shows that the same core mechanism remains effective beyond scalar Poisson equations.

Ablations

In this section, we validate the independent contributions and synergistic effects of the two core components–cognitive learning and region optimization–through systematic ablation studies conducted on three PDEs. The primary focus of these studies is to assess model performance and efficiency.

Performance analysis.

As illustrated in Fig 7, we present a comparative analysis of rMAE across five distinct base network architectures. The conventional PINN approach is represented by Vanilla, while RoPINN and CoPINN denote methods employing region optimization and cognitive learning, respectively. CoRoPINN represents the comprehensive method proposed in this paper.

thumbnail
Fig 7. Accuracy ablation study of CoRoPINN on different PDEs and diverse base models.

rMAE is recorded. The rMAE values are shown on a logarithmic y-axis.

https://doi.org/10.1371/journal.pone.0358646.g007

Overall, CoRoPINN achieves optimal or near-optimal performance across all test configurations. When compared to the Vanilla baseline, the introduction of either region optimization or cognitive learning individually yields significant improvements. This ablation study validates our core hypothesis: region optimization and cognitive learning enhance PINNs from the perspectives of spatial continuity and temporal dynamics, respectively, demonstrating their inherent complementarity.

Efficiency analysis.

Although CoRoPINN achieves favorable prediction accuracy, cognitive evaluation and region sampling introduce additional computation. Table 5 reports both rMAE and wall-clock training time in seconds per 100 epochs for the four methods. All entries use the same PINN backbone, collocation data, optimizer family, and evaluation interval within each PDE.

thumbnail
Table 5. Accuracy and wall-clock training-time comparison using the PINN backbone.

https://doi.org/10.1371/journal.pone.0358646.t005

The results show that CoRoPINN requires more training time than standard PINN. On 1D-Reaction, PINN is the fastest method at 10.5 s per 100 epochs but has an rMAE of 0.982. CoRoPINN attains an rMAE of 0.025 at 26.0 s, compared with 0.056 at 23.2 s for RoPINN and 0.038 at 22.9 s for CoPINN. In this setting, CoRoPINN attains the lowest rMAE among the four methods at the cost of a moderate increase in training time relative to the adaptive baselines.

The computational trade-off also depends on the PDE and comparator. On 1D-Wave, CoRoPINN requires 78.4 s per 100 epochs, compared with 90.6 s for CoPINN, while attaining a lower rMAE of 0.014 versus 0.039. On Convection, the four methods have closer training times, whereas CoRoPINN records the lowest rMAE among the compared methods.

Overall, the table shows that the measured accuracy–cost balance varies across the three evaluated PDEs. The additional operations therefore remain a practical limitation that should be considered when CoRoPINN is transferred to larger problems.

Hyperparameter analysis

This section systematically evaluates the impact of key hyperparameters on the performance of the CoRoPINN algorithm. The experiments focus on three core hyperparameters: the cognitive difficulty coefficient , the initial region size r0, and the warm-up period T warmup. These parameters collectively influence the dynamics and generalization capability of CoRoPINN. The sensitivity and optimal ranges of each parameter were assessed using various partial differential equations, with all results documented based on the conventional PINN model.

Initial region size r0.

The base region radius r0 is a critical hyperparameter in the CoRoPINN method’s region optimization mechanism. The value of r0 is directly correlated with the approximation accuracy of Monte Carlo region integration and the model’s generalization capability.

Fig 8 shows the rMAE, rMSE, and Loss curves for different values of r0 across three PDEs. When , the three metrics generally improve as r0 increases, indicating that moderate domain expansion can strengthen the continuity of the physical constraints. When , performance generally deteriorates, indicating that an excessively large domain can compromise optimization stability. Based on these evaluated problems, an initial radius in the range is recommended.

thumbnail
Fig 8. Performance of CoRoPINN on three PDEs using the PINN backbone under different initial region sizes.

https://doi.org/10.1371/journal.pone.0358646.g008

Cognitive difficulty coefficient .

The cognitive difficulty coefficient regulates the adaptability of region size to sample difficulty. This parameter directly influences the additional sampling range allocated to challenging samples and serves as a crucial factor in balancing local constraint strength and optimization stability.

Fig 9 evaluates CoRoPINN across three PDEs under different values of . The best overall performance on the evaluated problems occurs when lies in . Within this interval, the cognitive weighting mechanism balances the contributions of easy and difficult samples. Performance deteriorates when is below 10−5 or above 10−1. Based on these results, should be selected within and tuned for the target problem.

thumbnail
Fig 9. Performance of CoRoPINN on three PDEs using the PINN backbone under different cognitive difficulty coefficients.

https://doi.org/10.1371/journal.pone.0358646.g009

Warm-up period .

The warm-up period T warmup determines when the model begins to focus on challenging samples. An insufficient warm-up period may lead to instability during early training, whereas an excessively long warm-up period can delay learning in important areas such as boundaries.

Fig 10 shows the performance of CoRoPINN across three PDEs under different values of Twarmup. For 1D-Reaction, rMAE remains between 0.02 and 0.04 across the evaluated range. For 1D-Wave, the model remains stable when Twarmup is in [0,800], with rMAE and rMSE between 0.01 and 0.03. When Twarmup reaches 900 or more, rMAE and rMSE increase to 0.11 and 0.13, respectively. For Convection, rMAE and rMSE increase gradually over [0,1000]. Based on these evaluated problems, a practical initial range is [100,600].

thumbnail
Fig 10. Performance of CoRoPINN on three PDEs using the PINN backbone under different warm-up periods.

https://doi.org/10.1371/journal.pone.0358646.g010

Discussion

Limitations

The results should be interpreted in light of three limitations. First, gradient-statistic calibration, pointwise difficulty evaluation, and Monte Carlo region sampling add computation beyond standard PINN training. Table 5 quantifies this trade-off under a common setup and shows that its magnitude depends on the PDE and comparator. Second, performance depends on the initial region radius r0, cognitive difficulty coefficient , and warm-up period Twarmup. Figs 8Fig 9, Fig 10 identify useful ranges for the evaluated problems, but these ranges may require retuning for previously unseen PDEs or backbone architectures [43]. Third, although the experiments cover one-dimensional time-dependent equations, Poisson problems up to five dimensions, and one two-dimensional nonlinear coupled Burgers problem, they do not establish performance for arbitrary high-dimensional domains, complex geometries, or broader coupled multiphysics systems. These boundaries limit the extent to which the present results can be generalized.

Future work

Future work will address these limitations in three directions. More efficient reuse of gradient statistics and adaptive allocation of region samples may reduce the additional training cost. Joint or automatic selection of r0, , and Twarmup may improve transfer across PDEs while reducing manual tuning. Further evaluation should extend CoRoPINN to higher-dimensional equations, complex geometries, broader coupled and multiphysics systems, and a wider range of backbone architectures. The Poisson-5D and Burgers2D experiments provide initial tests of high-dimensional and nonlinear coupled settings, respectively, but broader validation is needed to determine the limits of these extensions.

Conclusion

This paper presents CoRoPINN, a cognition-guided region optimization method that couples the spatial continuity of RoPINN with the progressive difficulty scheduling of CoPINN. Rather than placing the two mechanisms side by side, CoRoPINN forms a closed feedback loop: gradient-stability statistics calibrate a globally stable radius, PDE residuals estimate pointwise difficulty, and this difficulty jointly controls adaptive neighborhoods and cognitive weights. The resulting optimization progressively redirects learning toward difficult regions while maintaining continuous physical constraints around collocation points.

Across the evaluated 1D-Reaction, 1D-Wave, and Convection problems, CoRoPINN obtains the lowest geometric-mean rMAE over all five backbone architectures, reducing the aggregate error relative to the second-best method by 21.0%, 61.6%, and 52.9%, respectively. In the multidimensional tests, CoRoPINN ranks first in both metrics for 2D and 5D Poisson and second in 3D. It also achieves the lowest rMAE and rMSE among all four PINN-based methods on Burgers2D. Taken together, these results show that CoRoPINN’s main advantage lies in using the same residual-derived difficulty signal to decide both where continuous constraints should be expanded and where optimization effort should be concentrated. This coordinated adaptation makes more effective use of sparse collocation information while preserving neighborhood continuity, allowing the method to retain strong performance across changes in backbone, dimensionality, and PDE dynamics. Within the evaluated scope, CoRoPINN is the strongest overall method among the compared baselines.

Acknowledgments

The authors have no acknowledgments to report.

References

  1. 1. Evans LC. Partial differential equations. vol 19. American Mathematical Society; 2022.
  2. 2. Roubíček T. Nonlinear partial differential equations with applications. Springer; 2005.
  3. 3. Solin P. Partial differential equations and the finite element method. Hoboken: Wiley-Interscience; 2006.
  4. 4. Jagota V, Sethi APS, Kumar K. Finite element method: an overview. Walailak J Sci Technol. 2013;10(1):1–8.
  5. 5. Thomas JW. Numerical partial differential equations: finite difference methods. Springer Science & Business Media; 2013.
  6. 6. Barth T, Ohlberger M. Finite volume methods: foundation and analysis. In: Encyclopedia of computational mechanics. vol 1. John Wiley & Sons; 2004. p. 1–60.
  7. 7. Hao Z, Liu S, Zhang Y, Ying C, Feng Y, Su H, et al. Physics-informed machine learning: A survey on problems, methods and applications. arXiv preprint. 2022;arXiv:2211.08064.
  8. 8. Karniadakis GE, Kevrekidis IG, Lu L, Perdikaris P, Wang S, Yang L. Physics-informed machine learning. Nat Rev Phys. 2021;3(6):422–40.
  9. 9. Raissi M, Perdikaris P, Karniadakis GE. Physics-informed neural networks: A deep learning framework for solving forward and inverse problems involving nonlinear partial differential equations. J Computat Phys. 2019;378:686–707.
  10. 10. Cuomo S, Di Cola VS, Giampaolo F, Rozza G, Raissi M, Piccialli F. Scientific Machine Learning Through Physics–Informed Neural Networks: Where we are and What’s Next. J Sci Comput. 2022;92(3).
  11. 11. Cho J, Nam S, Yang H, Yun SB, Hong Y, Park E. Separable physics-informed neural networks. Adv Neural Inform Process Syst. 2023;36:23761–88.
  12. 12. Wu W, Duan S, Sun Y, Yu Y, Liu D, Peng D. Deep fuzzy physics-informed neural networks for forward and inverse PDE problems. Neural Netw. 2025;181:106750. pmid:39427411
  13. 13. Yu J, Lu L, Meng X, Karniadakis GE. Gradient-enhanced physics-informed neural networks for forward and inverse PDE problems. Comput Methods Appl Mech Eng. 2022;393:114823.
  14. 14. Yao J, Su C, Hao Z, Liu S, Su H, Zhu J. MultiAdam: Parameter-wise scale-invariant optimizer for multiscale training of physics-informed neural networks. In: International conference on machine learning. PMLR; 2023. p. 39702–21.
  15. 15. Wang S, Yu X, Perdikaris P. When and why PINNs fail to train: A neural tangent kernel perspective. J Computat Phys. 2022;449:110768.
  16. 16. Xiang Z, Peng W, Liu X, Yao W. Self-adaptive loss balanced Physics-informed neural networks. Neurocomputing. 2022;496:11–34.
  17. 17. Kharazmi E, Zhang Z, Karniadakis GE. Variational physics-informed neural networks for solving partial differential equations. arXiv preprint. 2019;arXiv:1912.00873.
  18. 18. Wu C, Zhu M, Tan Q, Kartha Y, Lu L. A comprehensive study of non-adaptive and residual-based adaptive sampling for physics-informed neural networks. Comput Methods Appl Mech Eng. 2023;403:115671.
  19. 19. Yang L, Zhang D, Karniadakis GE. Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations. SIAM J Sci Comput. 2020;42(1):A292–317.
  20. 20. Wu H, Luo H, Ma Y, Wang J, Long M. RoPINN: Region optimized physics-informed neural networks. Adv Neural Inform Process Syst. 2024;37:110494–532.
  21. 21. Duan S, Wu W, Hu P, Ren Z, Peng D, Sun Y. CoPINN: Cognitive Physics-Informed Neural Networks. ICML; vol. 267. 2025. p. 14759–77.
  22. 22. Jiang L, Meng D, Zhao Q, Shan S, Hauptmann A. Self-paced curriculum learning. In: Proceedings of the AAAI conference on artificial intelligence. vol. 29. 2015. p. 2694–700.
  23. 23. Rubinstein RY, Kroese DP. Simulation and the Monte Carlo method. John Wiley & Sons; 2008.
  24. 24. Yuan YX. A review of trust region algorithms for optimization. ICIAM; vol. 99; 2000. p. 271–82.
  25. 25. Soviany P, Ionescu RT, Rota P, Sebe N. Curriculum Learning: A Survey. Int J Comput Vis. 2022;130(6):1526–65.
  26. 26. Wang X, Chen Y, Zhu W. A Survey on Curriculum Learning. IEEE Trans Pattern Anal Mach Intell. 2021;44(9):4555–76. pmid:33788677
  27. 27. Hardt M, Recht B, Singer Y. Train faster, generalize better: Stability of stochastic gradient descent. In: International conference on machine learning. PMLR; 2016. p. 1225–34.
  28. 28. Xiao J, Fan Y, Sun R, Wang J, Luo ZQ. Stability analysis and generalization bounds of adversarial training. Adv Neural Inform Process Syst. 2022;35:15446–59.
  29. 29. Krishnapriyan A, Gholami A, Zhe S, Kirby R, Mahoney M. Characterizing possible failure modes in physics-informed neural networks. Adv Neural Inform Process Syst. 2021;34:26548–60.
  30. 30. Liu DC, Nocedal J. On the limited memory BFGS method for large scale optimization. Math Programm. 1989;45(1–3):503–28.
  31. 31. Sirignano J, Spiliopoulos K. DGM: A deep learning algorithm for solving partial differential equations. J Comput Phys. 2018;375:1339–64.
  32. 32. Ghadimi S, Lan G. Stochastic First- and Zeroth-Order Methods for Nonconvex Stochastic Programming. SIAM J Optim. 2013;23(4):2341–68.
  33. 33. Rathore P, Lei W, Frangella Z, Lu L, Udell M. Challenges in training PINNs: A loss landscape perspective. arXiv preprint. 2024;arXiv:2402.01868.
  34. 34. Mojgani R, Balajewicz M, Hassanzadeh P. Lagrangian PINNs: A causality-conforming solution to failure modes of physics-informed neural networks. arXiv preprint. arXiv:2205.02902. 2022.
  35. 35. Hao Z, Yao J, Su C, Su H, Wang Z, Lu F, et al. PINNacle: A comprehensive benchmark of physics-informed neural networks for solving PDEs. Adv Neural Inform Process Syst. 2024;37:76721–74.
  36. 36. Bu J, Karpatne A. Quadratic residual networks: A new class of neural networks for solving forward and inverse problems in physics involving PDEs. In: Proceedings of the 2021 SIAM International Conference on Data Mining (SDM). SIAM; 2021. p. 675–83.
  37. 37. Wong JC, Ooi CC, Gupta A, Ong Y-S. Learning in Sinusoidal Spaces With Physics-Informed Neural Networks. IEEE Trans Artif Intell. 2022;5(3):985–1000.
  38. 38. Zhao Z, Ding X, Prakash BA. PINNsFormer: A transformer-based framework for physics-informed neural networks. arXiv preprint. arXiv:2307.11833. 2023.
  39. 39. Liu Z, Wang Y, Vaidya S, Ruehle F, Halverson J, Soljacic M, et al. KAN: Kolmogorov–Arnold networks. In: International conference on learning representations. vol. 2025. 2025. p. 70367–413.
  40. 40. Paszke A, Gross S, Massa F, Lerer A, Bradbury J, Chanan G, et al. PyTorch: An imperative style, high-performance deep learning library. Adv Neural Inform Process Syst. 2019;32.
  41. 41. Chai T, Draxler RR. Root mean square error (RMSE) or mean absolute error (MAE)? – Arguments against avoiding RMSE in the literature. Geosci Model Dev. 2014;7(3):1247–50.
  42. 42. Hodson TO. Root-mean-square error (RMSE) or mean absolute error (MAE): when to use them or not. Geosci Model Dev. 2022;2022:1–10.
  43. 43. Wang S, Sankaran S, Wang H, Perdikaris P. An expert’s guide to training physics-informed neural networks. arXiv preprint. arXiv:2308.08468. 2023.