acceptodds
Under review as a conference paper at ICLR 2027

Physics-Informed Stochastic Configuration Machine: A Backpropagation-Free Neural Network with Fast Training involving Nonlinear Differential Equations

Abstract

While Physics-Informed Neural Networks (PINNs) have emerged as a transformative paradigm for solving complex differential equations, their reliance on backpropagation-based gradient descent and automatic differentiation (AD) imposes significant computational bottlenecks and severe non-convex optimization challenges. To overcome these fundamental limitations, we propose the Physics-Informed Stochastic Configuration Machine (PI-SCM), a novel backpropagation-free framework for both forward and inverse problems in differential equations. The core mathematical contribution lies in the analytical evaluation of local Jacobians for nonlinear differential operators, which facilitates a linearized representation of the physical loss and projects it into a unified, linearized algebraic subspace. This reformulation allows for the explicit determination of network weights via a sequence of linearized least-squares problems, effectively bypassing the iterative traps of traditional nonlinear optimizers. We develop a progressive algorithmic suite comprising localized construction (PI-SC-I), sliding-window correction (PI-SC-II), and global correction (PI-SC-III), and establish their universal approximation properties. Extensive experiments demonstrate that PI-SCM achieves robust parameter identification and substantially higher predictive accuracy than the PIELM baselines while accelerating training by orders of magnitude compared with standard PINNs. Our work provides a highly efficient and scalable foundation for next-generation, real-time Scientific Machine Learning applications.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Nonlinear Differential Equations, Scientific Machine Learning, Physics-Informed Neural Networks, Stochastic Configuration Networks, Backpropagation-free

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_15zkOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Physics-Informed Stochastic Configuration Machine (PI-SCM), a backpropagation-free method for solving forward and inverse problems involving differential equations. The method uses randomly generated basis functions and incrementally constructs a single-hidden-layer network. For nonlinear PDEs, the authors linearize the physical residual around the current approximation and determine the output weights through least-squares problems.

Critical strengths
  1. By combining stochastic node construction with local linearization and least-squares optimization, the authors avoid backpropagation while retaining the ability to solve nonlinear differential equations.
  2. The paper includes theoretical analysis and extensive numerical tests. The authors establish residual convergence under stated assumptions and investigate the effects of activation functions, network width, collocation points, and correction frequency.
Critical weaknesses
  1. The five-dimensional experiment is not genuinely high-dimensional. The solution in Appendix D.4 depends only on x_1 and x_2, with no dependence on the other three spatial variables. This does not provide convincing evidence of the method’s ability to handle high-dimensional problems. A test involving all five spatial variables would be more appropriate.
  2. Both the Helmholtz and reaction–diffusion examples use solutions with trigonometric structure, which may favor the sine activation used by PI-SCM.
  3. The novelty over NNBM needs further clarification. Both methods use random basis functions and linearized least-squares optimization for nonlinear PDEs. The global correction in PI-SC-III is closely related to the Gauss–Newton procedure used in NNBM. The main difference appears to be the incremental node selection and correction strategy. An ablation study using the same randomly generated basis functions with and without incremental selection would help clarify the contribution of this strategy.
  4. Figure 6 shows residual reduction against the number of accepted nodes, but does not report how many candidates were evaluated before each node was accepted. The acceptance rate, average number of trials per node, and residual reduction against actual runtime would help assess the efficiency of the construction procedure.
  5. The runtime comparisons use different hardware and network settings. PI-SCM and NNBM are evaluated on CPU, whereas PINN and PISCN are evaluated on GPU. In addition, PI-SCM uses sine activation while the PINN baselines use tanh. For a fair comparison, all methods should be evaluated on the same hardware (either CPU or GPU), preferably with consistent activation functions, to better assess the computational advantage of the proposed method.
  6. The advantage over NNBM is less clear than the paper suggests. Table 1 shows that PI-SC-III is slower than NNBM on all five benchmarks. Although the accuracy improvement is substantial for Helmholtz and Allen–Cahn, it is relatively modest for reaction–diffusion and Bratu. Since NNBM constructs its basis functions at once, whereas PI-SC-III requires incremental selection and repeated corrections, an error-versus-total-training-time comparison would better establish whether the additional computational cost is justified.
  7. The theoretical results concern discrete residual convergence rather than solution accuracy. Theorem 1 proves that the residual at the collocation points converges to zero under a strong full-row-rank assumption. However, a small residual does not necessarily imply a small solution error, and the theorem does not establish convergence to the true PDE solution.
  8. The generality of the method for nonlinear PDEs is not sufficiently demonstrated. Although the paper claims to handle nonlinear differential equations, it does not test standard nonlinear problems such as the Burgers or Euler equations. It remains unclear how the method performs for nonlinear PDEs with sharp gradients or discontinuities.
Questions for authors

Please refer to the concerns in the weaknesses section.

6 comments

We appreciate your time and detailed feedback. Thank you for recognizing our combination of stochastic node construction and analytical output weight updates for solving nonlinear differential equations without backpropagation, together with the convergence analysis and extensive numerical evaluation. We have revised the manuscript and expanded the experimental evaluation. Revisions to the manuscript are highlighted in blue for ease of reference. Our point-by-point responses to your concerns and questions are provided below.

W1 The five-dimensional experiment is not genuinely high-dimensional. The solution in Appendix D.4 depends only on x_1 and x_2, with no dependence on the other three spatial variables. This does not provide convincing evidence of the method’s ability to handle high-dimensional problems. A test involving all five spatial variables would be more appropriate.

Response:

Thank you for your suggestion regarding the effective dimensionality of the five-dimensional benchmark. We have added a nonlinear reaction–diffusion problem involving all five spatial variables in Appendix D.4.3. Its solution explicitly depends on all five spatial variables, with each spatial direction contributing a nonzero term to the diffusion operator. We evaluated the method under both unsupervised and semi-supervised settings. As shown in Table 3, PI-SC-III achieves mean RMSEs of 6.30×10−36.30\times10^{-3} and 6.25×10−36.25\times10^{-3}, respectively, providing experimental evidence of PI-SCM’s ability to solve nonlinear PDEs involving all five spatial variables.

W2 Both the Helmholtz and reaction–diffusion examples use solutions with trigonometric structure, which may favor the sine activation used by PI-SCM.

Response:

Thank you for your suggestion regarding activation function selection. Your comment motivated us to repeat and expand the activation study in Appendix D.10. The updated study covers eight benchmarks, and systematically compares sine, sigmoid, and tanh activations.

Considering the different sensitivities of activation functions and their derivatives to hidden parameter scales, we tuned the scale parameter aa, which controls the sampling range of hidden parameters, separately for each activation function. This allows each activation to operate at an appropriate scale and fully realize its performance.

The results in Table 8 show that sigmoid and tanh also achieve high solution accuracy on the Helmholtz and reaction–diffusion cases highlighted in your comment. These findings further demonstrate the effectiveness of PI-SCM across different activation functions, showing that its physics-informed construction and analytical update mechanisms can support accurate solutions using multiple choices of activation.

W3.1 The novelty over NNBM needs further clarification. Both methods use random basis functions and linearized least-squares optimization for nonlinear PDEs. The global correction in PI-SC-III is closely related to the Gauss–Newton procedure used in NNBM. The main difference appears to be the incremental node selection and correction strategy.

Response:

Thank you for highlighting the connection between the global correction in PI-SC-III and the Gauss–Newton method. The global correction solves a linearized least squares problem within the currently accepted basis and shares a mathematical connection with a Gauss–Newton update. PI-SCM embeds this mechanism into physics-informed incremental construction, with three principal contributions.

1. Physical residuals directly guide basis construction. NNBM optimizes output weights within a preconfigured set of random basis functions. PI-SCM configures new nodes according to the current residuals of the governing equations, initial and boundary conditions, and optional observations. Combined with adaptive sampling scales, this progressively constructs a feature space suited to the current problem. The supervisory inequality determines candidate acceptance according to the actual contribution to the true nonlinear loss. Through this mechanism, every accepted new node reduces the current residual norm, directly linking network growth to improvement in the physics-informed objective.

2. Analytical weight updates are coupled with supervisory construction. NNBM uses iterative Gauss–Newton optimization to update output weights within a given basis. PI-SCM solves one linear least squares problem for each candidate configuration and each windowed or global correction, then combines directional damping with true residual evaluation to determine an acceptable update. The algorithm subsequently advances node construction, alternating expansion of the feature space with weight updates. This design unifies physics-informed basis construction and efficient analytical training within a single solution procedure.

3. Convergence analysis covers the coupled construction and correction process. We analyze the dependence of residual mappings on the current state and the Taylor remainder introduced by linearization. We establish monotonic reduction and convergence of the residual under the stated assumptions. We further prove that windowed and global corrections preserve the convergence achieved by local construction, providing theoretical support for the complete process of node selection, analytical updates, and weight correction.

W3.2 An ablation study using the same randomly generated basis functions with and without incremental selection would help clarify the contribution of this strategy.

Response:

Thank you for suggesting further evaluation of the contribution of incremental construction. Node selection and output weight updates in PI-SCM are coupled, with each accepted node changing the current true nonlinear residual and thereby influencing the configuration and selection of subsequent nodes.

To further evaluate the proposed mechanisms, we compare the solution accuracy of PI-SC-III and NNBM/PIELM at matched hidden layer widths across eight benchmarks in Figure 10 of Appendix D.8. This comparison evaluates the combined effects of their respective construction and output weight update procedures under matched node budgets.

PI-SC-III generally achieves lower mean RMSE, supporting the joint effectiveness of physics-informed incremental construction and analytical weight updates in making better use of the available model capacity and constructing a feature space adapted to the problem.

W4 Figure 6 shows residual reduction against the number of accepted nodes, but does not report how many candidates were evaluated before each node was accepted. The acceptance rate, average number of trials per node, and residual reduction against actual runtime would help assess the efficiency of the construction procedure.

Response:

Thank you for your attention to candidate evaluation costs and construction efficiency. We further clarify the candidate search mechanism. PI-SCM adds one node to the network at each successful construction stage, while each search round generates and evaluates a batch of Tmax⁡T_{\max} candidates. If the batch contains candidates satisfying the supervisory inequality for the true residual, the algorithm selects the one yielding the greatest residual reduction. Otherwise, it adjusts the sampling scale, supervisory parameter, and damping factor according to the prescribed procedure and evaluates new candidate batches.

Therefore, when the first batch succeeds, each newly added node requires Tmax⁡T_{\max} candidate evaluations. If configuring the LL-th node requires NLN_L batches, the actual number of candidate evaluations is NLTmax⁡N_L T_{\max}, and the ratio of selected nodes to evaluated candidates at that stage is 1/(NLTmax⁡)1/(N_L T_{\max}). In our implementation, candidates within the same construction stage share the current network state and residual information. Their node outputs, derivatives, analytical coefficients, and true residuals are evaluated through parallel batched tensor operations.

We have also added comparisons of solution error against actual training time across eight benchmarks in Appendix D.8 (Figure 9). For PI-SC-III, the reported training time includes the generation and evaluation of all candidates, supervisory selection, and output weight corrections, allowing the computational cost of candidate search to be assessed together with the achieved solution accuracy.

W5.1 The runtime comparisons use different hardware and network settings. PI-SCM and NNBM are evaluated on CPU, whereas PINN and PISCN are evaluated on GPU. 

Response:

Thank you for your comment on the experimental hardware configuration. Our experiments evaluate the practical training efficiency and computational resource requirements of different solution procedures. PI-SCM, PIELM, and NNBM run on the CPU, while PINN and PISCN are trained on the GPU. This configuration gives methods relying on backpropagation access to GPU acceleration, while demonstrating the ability of PI-SCM to achieve high accuracy and fast training on a standard CPU.

NNBM [1] and LocELM [2] also report performance evaluations on CPUs. For comparisons across devices, Frozen-PINN [3] likewise compares its method running on a CPU with PINNs trained on GPUs. These studies provide precedents for comparing the practical training efficiency and resource requirements of PDE solvers using hardware configurations suited to their computational workflows.

W5.2 In addition, PI-SCM uses sine activation while the PINN baselines use tanh.

Response:

Thank you for your comment on the activation function settings. Our PINN baselines use tanh following the recommendations of Wang et al. [4] . This paper explicitly recommends tanh for constructing continuously differentiable neural representations, providing a literature basis for our baseline configuration.

Appendix D.10 evaluates PI-SCM with sine, tanh, and sigmoid across eight benchmarks, with the hidden parameter sampling scale aa tuned separately for each activation function. The results show that PI-SCM also achieves high solution accuracy with tanh. In particular, tanh yields the lowest mean RMSE among the three activations for Allen–Cahn and Bratu, reaching 2.39×10−42.39\times10^{-4} and 4.70×10−64.70\times10^{-6}, respectively. These results demonstrate PI-SCM’s ability to obtain accurate solutions using the same tanh activation as the PINN baselines, as well as its adaptability to different activation functions.

Reviewer_vygkOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper propose PI-SCM, which extends the Stochastic Configuration Machine framework (Wang & Li, 2017) to solve nonlinear PDEs in PINN forms. While traditional randomized networks and Extreme Learning Machines (ELMs) fix hidden parameters and solve for output weights via linear least squares, nonlinear differential operators prevent the physical loss from being linear in output weights. PI-SCM addresses this by applying a first-order Taylor expansion around the previous state via analytically evaluated local Jacobians, projecting the physical residuals into a linearized algebraic subspace. Practical algorithms are proposed including localized construction (PI-SC-I), sliding-window correction (PI-SC-II), and global correction (PI-SC-III) to control Taylor truncation errors and ensure monotonic residual reduction and universal approximation properties. The methodology is evaluated on five benchmark systems for forward problems (Van der Pol oscillator, 2D Helmholtz, Allen-Cahn, 5D reaction-diffusion, and 2D Bratu equations) and three benchmark systems for inverse problems.

Critical strengths

Bypassing inner iterative nonlinear optimization loops (such as Gauss-Newton iterations) by analytically evaluating local Jacobians and using a single linear least-squares solve per candidate node provides a direct, computationally efficient engineering solution.

Critical weaknesses
  1. Deep PINN baselines utilize small architectures (e.g., 1×3001\times300 or 4×644\times64) and fixed, limited training budgets, raising concerns that they may be under-trained. Evaluating speedups against under-trained baselines without providing error-versus-time curves makes the efficiency claims less definitive.

  2. The SCM network is known to be earlier work. The author applies it to PINN training. The extension is directly for linear PDEs. The key novelty only lies in nonlinear PDEs with a first order Taylor expansion, which is limited.

Questions for authors
  1. Can you include strong baselines such as frozen-PINN (ICLR 2026), locELM (CMAME 2021), etc.?

  2. Can the authors provide error-versus-time Pareto curves comparing PI-SCM against fully converged PINN and randomized baseline models on identical hardware to substantiate the 36–107× acceleration claims under equivalent accuracy targets?

  3. How can Assumption 2 (regarding the full row rank of reference block matrices across the reachable state closure K\mathbb{K}) be validated in practical algorithm implementations?

  4. Can you extend the inverse problem evaluations to systematically analyze robustness and parameter identification errors under varying levels of observation noise (e.g., 1%, 5%, 10%)?

4 comments

We appreciate your time and careful review. Thank you for recognizing the computational efficiency of our method. We have revised the manuscript and expanded the experimental evaluation. Revisions to the manuscript are highlighted in blue for ease of reference. Our point-by-point responses to your concerns and questions are provided below.

W1 Deep PINN baselines utilize small architectures (e.g., 1×3001\times300 or 4×644\times64) and fixed, limited training budgets, raising concerns that they may be under-trained. Evaluating speedups against under-trained baselines without providing error-versus-time curves makes the efficiency claims less definitive.

Response:

Please refer to our answer to Q2.

W2 The SCM network is known to be earlier work. The author applies it to PINN training. The extension is directly for linear PDEs. The key novelty only lies in nonlinear PDEs with a first order Taylor expansion, which is limited.

Response:

Thank you for your comment on the novelty of our method. PI-SCM builds on the original SCN’s local, windowed, and global output weight update strategies and its supervised node construction. We extend this framework for target function approximation to network construction and efficient training under nonlinear physical constraints, together with a corresponding convergence analysis.

1. Extending supervision from target function approximation to physical constraints. The original SCN [1] configures hidden nodes using the target function approximation residual eL−1=f−fL−1e_{L-1}=f-f_{L-1}. Its implementation with finite training samples uses these data to compute supervisory criteria and estimate output weights. PI-SCM incorporates the differential equations, initial and boundary conditions, and optional observations into a unified objective. It selects nodes according to their contribution to the true physical residual, allowing physical constraints to guide both network construction and output weight correction. This also supports solution reconstruction in unsupervised settings. The mechanism progressively constructs basis functions adapted to the current equations and boundary conditions. Figure 10 in Appendix D.8 shows that PI-SC-III generally achieves lower mean RMSE than NNBM/PIELM at matched hidden layer widths across eight benchmarks, supporting the combined effectiveness of physics-informed construction and analytical weight updates in making better use of a given node budget.

2. Establishing fast analytical updates for nonlinear physical objectives. In the original SCN, once the hidden parameters are fixed, the network output depends linearly on the output weights. The local residual therefore admits an exact linear update, and the output weights can be determined directly through linear least squares. Nonlinear differential operators introduce nonlinear dependence of the physical residual on the output weights, resulting in mappings that depend on the current state and remainders from linearization.

PI-SCM integrates analytical Jacobian evaluation, a single linear least squares solve, directional damping, and true residual supervision into incremental construction. This mechanism identifies updates proposed by the local linear model that improve the original nonlinear objective. As discussed in Remark 1, each candidate configuration and each output weight correction requires only one linear least squares solve. Table 11 in Appendix D.11 further shows that directly introducing iterative Gauss–Newton solves requires approximately 6–26 times as much training time as our method, with limited gains in accuracy, substantially eroding SCN’s fast training advantage.

3. Establishing convergence analysis for nonlinear physics-informed construction. The original SCN theorems establish convergence of the target function approximation error using the linear structure of the output weights, supervisory inequalities, and least squares optimality. PI-SCM’s analysis further addresses the dependence on the current state and Taylor remainders introduced by nonlinear differential operators. Under the stated assumptions, we prove the existence of admissible candidate configurations and convergence of the true collocation residual, and show that windowed and global corrections preserve this property. These results extend SCN’s theory of supervised construction to the optimization of nonlinear physical residuals, providing a theoretical foundation for the proposed analytical training mechanism.

Q1 Can you include strong baselines such as frozen-PINN (ICLR 2026), locELM (CMAME 2021), etc.?

Response:

Thank you for suggesting stronger baselines. We have added comparisons of solution accuracy and training time on three problems to which both methods apply, covering Allen–Cahn, reaction–diffusion, and nonlinear diffusion on a tree-shaped irregular domain. The experiments include Frozen-PINN-elm, Frozen-PINN-swim, and a projected Frozen-PINN-swim variant for reaction–diffusion. The results are summarized in Table 1, with experimental settings provided in Appendices D.3, D.4, and D.7.

For nonlinear diffusion, PI-SC-III offers advantages in both accuracy and efficiency. Compared with Frozen-PINN-swim, PI-SC-III reduces the mean RMSE by factors of approximately 68–175 and achieves training speedups of approximately 199–218, demonstrating its effectiveness on this nonlinear problem with an irregular geometry.

For reaction–diffusion, PI-SC-III achieves training speedups of approximately 3.0–9.2 over the three Frozen-PINN configurations and obtains lower RMSE than Frozen-PINN-elm and unprojected Frozen-PINN-swim. The Frozen-PINN-swim variant using projection based on gradients of the initial profile achieves an RMSE of approximately 6.6×10−56.6\times10^{-5}, demonstrating the benefit of sampling guided by the solution structure in this benchmark.

For Allen–Cahn, Frozen-PINN trains faster and achieves lower RMSE in the unsupervised setting. In the semi-supervised setting, Frozen-PINN-swim and PI-SC-III achieve mean RMSEs of 1.52×10−31.52\times10^{-3} and 1.57×10−31.57\times10^{-3}, respectively, yielding comparable accuracy.

Q2 Can the authors provide error-versus-time Pareto curves comparing PI-SCM against fully converged PINN and randomized baseline models on identical hardware to substantiate the 36–107× acceleration claims under equivalent accuracy targets?

Response:

Thank you for suggesting an efficiency comparison after sufficient training and under matched accuracy targets. We have added curves of error versus training time across eight benchmarks in Appendix D.8 (Figure 9). The PINN baselines include 4×644\times64 and 8×648\times64 architectures, with training budgets extended to 20,000 epochs. We also expanded the range of network sizes evaluated for PI-SC-III and NNBM/PIELM so that the comparisons cover performance after convergence.

For each benchmark, we use the lower mean RMSE attained at convergence by the two PINN architectures as the target accuracy. Among the PI-SC-III configurations reaching this target, we select the one with the shortest mean training time. Comparing this time with the corresponding PINN’s mean convergence time yields training speedups of approximately 62.7–1317, quantifying computational efficiency under explicit accuracy targets.

Figure 9 also compares PI-SC-III with NNBM/PIELM. After both methods converge, PI-SC-III reduces the mean RMSE by factors ranging from approximately 1.401.40 to 7.57×1047.57\times10^4. Figure 10 further provides comparisons at matched hidden layer widths, showing that PI-SC-III achieves lower RMSE.

All experiments are conducted on the same workstation, with PI-SCM, NNBM and PIELM running on the CPU and PINNs running on the GPU. The reported training times and speedups correspond to this hardware configuration, which is specified in Appendix D.

Q3 How can Assumption 2 (regarding the full row rank of reference block matrices across the reachable state closure K\mathbb{K}) be validated in practical algorithm implementations?

Response:

Thank you for asking how Assumption 2 can be verified in practice. We have added sufficient conditions for Assumption 2 in Appendix C.2, with the results stated in Lemma 2 and Theorem 4. Under the stated conditions, a fixed finite family of reference nodes can be selected such that the concatenated reference matrix has full row rank throughout the reachable state closure K\mathbb{K}, while each individual reference block maintains constant rank.

We further provide a structural verification for the two dimensional Bratu equation, covering both unsupervised and semi-supervised settings. This demonstrates how the sufficient conditions can be established for a concrete nonlinear PDE.

Q4 Can you extend the inverse problem evaluations to systematically analyze robustness and parameter identification errors under varying levels of observation noise (e.g., 1%, 5%, 10%)?

Response:

Thank you for suggesting an evaluation of observation noise in inverse problems. We have added experiments on the Van der Pol, Helmholtz, and Bratu inverse problems in Appendix D.9, considering noise levels of 00\\%, 11\\%, 55\\%, and 1010\\%, with Nd∈100,500,1000N_d\in\\{100,500,1000\\} observations. Tables 5–7 report the means and standard deviations of the absolute parameter identification error, state RMSE, and training time.

The results show that parameter identification errors generally increase with observation noise, while additional observations mitigate its impact. For PI-SC-III, at 1010\\% noise with Nd=1000N_d=1000, the mean parameter identification errors for Van der Pol and Bratu are 2.53×10−22.53\times10^{-2} and 1.44×10−31.44\times10^{-3}, respectively, with corresponding state RMSEs of 1.42×10−21.42\times10^{-2} and 9.74×10−59.74\times10^{-5}. These results demonstrate PI-SCM’s ability to recover physical parameters and reconstruct states under substantial observation noise, providing experimental support for its robustness in solving inverse problems with noisy observations.

Thank you again for your careful review and constructive comments. Your feedback has helped us strengthen the theoretical justification and expand the experimental evaluation of PI-SCM. We welcome further discussion and questions.

[1] Dianhui Wang and Ming Li. Stochastic configuration networks: Fundamentals and algorithms. IEEE transactions on cybernetics, 47(10):3466–3479, 2017.

Thanks for the additional experiments and additional clarification. Some of my concerns have been concerned. For brevity I only list the remaining weakness here. The comparison with frozen-pinn only conducted on three PDEs. The eight PDE benchmarks are not the original frozen-pinn paper's eight PDE benchmarks. The authors clarification on W2 certificate that the novelty is incremental. The nonlinear PDEs with a first order Taylor expansion turn into linear PDEs. The physics-informed version is directly. The convergence results are weak, only the loss decays are proved, no decay rate, no solution convergence results. So I keep my score.

Reviewer_EeQZOpenReview ↗
Rating
3(+1)
Confidence
1(−1)
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes the Physics-Informed Stochastic Configuration Machine (PI-SCM), a framework for solving forward and inverse differential-equation problems without backpropagation. The method combines incremental hidden-node construction with local linearization of nonlinear physical residuals, allowing output-weight updates to be computed through linear least squares. Candidate nodes are accepted based on the actual nonlinear residual. The paper presents a universal approximation result for forward problems under the stated assumptions. The authors evaluate the framework on five forward and three inverse benchmarks, reporting substantial training-time reductions compared with the tested PINN baselines and accuracy improvements over PIELM/NNBM.

Critical strengths
  • Avoiding backpropagation in physics-informed learning is an interesting research direction, and I believe extending analytical output-weight updates to nonlinear differential equations is valuable. The combination of residual linearization and physics-informed node selection provides a clear approach to this problem.

  • While reading the paper, I initially was curious about the power of expressivity for the proposed method. I appreciate that the paper addresses this concern through a convergence analysis that accounts for the nonlinear residual and explains how the correction variants preserve convergence.

  • PI-SCM trains much faster than the PINN baselines that use backpropagation.

Critical weaknesses
  • My understanding is that the main advance is the combination of nonlinear residual linearization and physics-informed node selection, while the local (I), windowed (II), and global (III) update structure seems to follow existing SCN algorithms. I would appreciate a clearer explanation of what is new, as I am still unsure how much the method advances existing approaches.

  • Given the substantial speed differences already reported against PINNs, I do not view further PINN optimizer tuning as a major concern. For me, comparisons with other backpropagation-free methods need more attention. Could the authors compare accuracy and training time with Frozen-PINN on problems where both methods apply?

  • As a minor presentation suggestion, Appendix A would be more useful to readers unfamiliar with SCNs if it explained the main ideas behind the methods rather than simply listing related papers.

I would be happy to revise my assessment if the authors address these concerns.

Questions for authors
  • Could the authors clarify how PI-SCM differs from existing SCN construction and Gauss–Newton-based randomized methods? For linear PDEs, where the linearization is exact, what explains the substantial improvement over PIELM?

  • Could the authors compare with Frozen-PINN and similar methods on a few problems where all methods apply? The goal is to understand when each method works better. I do not expect PI-SCM to perform best in every case.

5 comments

We appreciate your time and thorough feedback. Thank you for recognizing the value of analytical weight updates to nonlinear differential equations, our convergence analysis, and the substantial training speedups over the PINN baselines. We have revised the manuscript and expanded the experimental evaluation. Revisions to the manuscript are highlighted in blue for ease of reference. Our point-by-point responses to your concerns and questions are provided below.

W1. My understanding is that the main advance is the combination of nonlinear residual linearization and physics-informed node selection, while the local (I), windowed (II), and global (III) update structure seems to follow existing SCN algorithms. I would appreciate a clearer explanation of what is new, as I am still unsure how much the method advances existing approaches.

Response:

Please refer to our answer to Q1.

W2. Given the substantial speed differences already reported against PINNs, I do not view further PINN optimizer tuning as a major concern. For me, comparisons with other backpropagation-free methods need more attention. Could the authors compare accuracy and training time with Frozen-PINN on problems where both methods apply?

Response:

Please refer to our answer to your Q2.

W3. As a minor presentation suggestion, Appendix A would be more useful to readers unfamiliar with SCNs if it explained the main ideas behind the methods rather than simply listing related papers.

Response:

Thank you for your suggestion regarding the background presentation. We have expanded Appendix A to explain the basic principles of randomized networks and SCNs, helping readers unfamiliar with SCNs understand the foundations of PI-SCM construction.

Q1.1 Could the authors clarify how PI-SCM differs from existing SCN construction and Gauss–Newton-based randomized methods?

Response:

Thank you for requesting clarification of how PI-SCM differs from existing SCN construction and Gauss–Newton-based randomized methods. PI-SCM builds on SCN’s local, windowed, and global output weight update strategies, together with its supervisory inequality for hidden node configuration. We extend this constructive framework to physics-informed learning for nonlinear differential equations through three developments.

1. Physics-informed node construction and weight updates. We incorporate nonlinear differential equation residuals, initial and boundary conditions, and optional observations into a unified objective. Physical constraints therefore guide both local node configuration and windowed or global output weight corrections. This enables construction without interior solution observations while supporting semi-supervised learning and inverse problems.

Methods such as PIELM, NNBM, and LocELM optimize output weights within a preconfigured feature space, whose suitability for the problem determines the attainable solution accuracy. PI-SCM extends the supervisory inequality to evaluate candidate configurations according to their actual contribution to the true nonlinear objective. Every accepted node reduces the current true residual, directly linking network growth to improvement in the physics-informed objective. Figure 10 in Appendix D.8 further compares solution accuracy at matched hidden layer widths across eight benchmarks. PI-SC-III generally achieves lower mean RMSE, supporting the effectiveness of physics-informed construction in making better use of a given node budget and constructing a feature space adapted to the problem.

2. Fast analytical updates integrated with incremental construction. Nonlinear differential operators make the physical loss nonlinear in the output weights. We analytically evaluate the local Jacobians to construct a linear least squares subproblem at the current state, then combine damping along the proposed update direction with true loss evaluation to determine an acceptable update efficiently.

Randomized methods using Gauss–Newton optimization, such as NNBM and LocELM, repeatedly linearize the optimization problem and update output weights within a given basis. PI-SCM integrates each linearization directly into node construction and output weight correction. As discussed in Remark 1, each candidate configuration and each windowed or global correction requires only one linear least squares solve, eliminating backpropagation-based training and repeated inner Gauss–Newton iterations during configuration. This provides an efficient computational mechanism that makes the SCN architecture practically viable for physics-informed learning with nonlinear differential equations.

The ablation study in Appendix D.11 (Table 11) shows that directly introducing iterative Gauss–Newton optimization into SCN construction substantially erodes its fast training advantage, with little or no improvement in solution accuracy over our method.

(continued in the next comment)

3. Convergence analysis for the true nonlinear residual. Nonlinear differential operators introduce residual mappings that depend on the current state and Taylor truncation errors, requiring analysis of how a linearized update affects the original nonlinear objective. Our convergence analysis differs fundamentally from SCN’s proof based on exact linear least squares approximation. Through damped updates and supervisory node selection, our proof explicitly controls nonlinear remainders and establishes convergence of the true collocation residual under the stated assumptions. It also shows that windowed and global corrections preserve this convergence property, providing a theoretical foundation for supervised construction under nonlinear physical constraints.

Q1.2 For linear PDEs, where the linearization is exact, what explains the substantial improvement over PIELM?

Response:

Physics-informed node construction remains effective for linear PDEs. PIELM estimates output weights within a feature space spanned by randomly configured hidden nodes. PI-SCM selects candidate nodes according to the current equation and combines this supervision with adaptive sampling scales to progressively construct a feature space adapted to the current problem. Exact linearization allows each candidate’s residual contribution to be evaluated accurately, so the supervisory criterion directly guides the selection of hidden nodes. This physics-guided construction explains why PI-SCM can achieve higher accuracy than PIELM even when the linearization is exact.

Q2 Could the authors compare with Frozen-PINN and similar methods on a few problems where all methods apply? The goal is to understand when each method works better. I do not expect PI-SCM to perform best in every case.

Response:

Thank you for suggesting a comparison with Frozen-PINN. We have added comparisons of solution accuracy and training time on three problems to which both methods apply, covering Allen–Cahn, reaction–diffusion, and nonlinear diffusion on a tree-shaped irregular domain. The experiments include Frozen-PINN-elm, Frozen-PINN-swim, and a projected Frozen-PINN-swim variant for reaction–diffusion. The results are summarized in Table 1, with experimental settings provided in Appendices D.3, D.4, and D.7.

For nonlinear diffusion, PI-SC-III offers advantages in both accuracy and efficiency. Compared with Frozen-PINN-swim, PI-SC-III reduces the mean RMSE by factors of approximately 68–175 and achieves training speedups of approximately 199–218, demonstrating its effectiveness on this nonlinear problem with an irregular geometry.

For reaction–diffusion, PI-SC-III achieves training speedups of approximately 3.0–9.2 over the three Frozen-PINN configurations and obtains lower RMSE than Frozen-PINN-elm and unprojected Frozen-PINN-swim. The Frozen-PINN-swim variant using projection based on gradients of the initial profile achieves an RMSE of approximately 6.6×10−56.6\times10^{-5}, demonstrating the benefit of sampling guided by the solution structure in this benchmark.

For Allen–Cahn, Frozen-PINN trains faster and achieves lower RMSE in the unsupervised setting. In the semi-supervised setting, Frozen-PINN-swim and PI-SC-III achieve mean RMSEs of 1.52×10−31.52\times10^{-3} and 1.57×10−31.57\times10^{-3}, respectively, yielding comparable accuracy.

Thank you again for your time and valuable feedback. Your comments have helped us clarify our contributions and strengthen the comparisons with related methods. We welcome any further questions or feedback.

W1. Thank you for the detailed explanation. However, the response seems to list the method’s strengths rather than clarify what is new. My understanding is that the proposed PI-SCM extends the existing framework (SC-I, II, and III) by using physics-informed residuals to select nodes and linearization to handle nonlinear PDEs. This feels like a natural and somewhat incremental extension, although I am not familiar enough with the SCN literature to judge its novelty with confidence.

W2. Frozen-PINN is more accurate in some settings, while PI-SCM is faster on reaction–diffusion. I have to admit that PI-SCM’s broader scope also remains a strength. I understand that accuracy can vary across PDEs, but could the authors explain why the speed comparison changes so much? Frozen-PINN is much faster on Allen–Cahn but much slower on reaction–diffusion and nonlinear diffusion. What causes this difference?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.