- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Physics-Informed Stochastic Configuration Machine (PI-SCM), a backpropagation-free method for solving forward and inverse problems involving differential equations. The method uses randomly generated basis functions and incrementally constructs a single-hidden-layer network. For nonlinear PDEs, the authors linearize the physical residual around the current approximation and determine the output weights through least-squares problems.
Critical strengths
- By combining stochastic node construction with local linearization and least-squares optimization, the authors avoid backpropagation while retaining the ability to solve nonlinear differential equations.
- The paper includes theoretical analysis and extensive numerical tests. The authors establish residual convergence under stated assumptions and investigate the effects of activation functions, network width, collocation points, and correction frequency.
Critical weaknesses
- The five-dimensional experiment is not genuinely high-dimensional. The solution in Appendix D.4 depends only on x_1 and x_2, with no dependence on the other three spatial variables. This does not provide convincing evidence of the method’s ability to handle high-dimensional problems. A test involving all five spatial variables would be more appropriate.
- Both the Helmholtz and reaction–diffusion examples use solutions with trigonometric structure, which may favor the sine activation used by PI-SCM.
- The novelty over NNBM needs further clarification. Both methods use random basis functions and linearized least-squares optimization for nonlinear PDEs. The global correction in PI-SC-III is closely related to the Gauss–Newton procedure used in NNBM. The main difference appears to be the incremental node selection and correction strategy. An ablation study using the same randomly generated basis functions with and without incremental selection would help clarify the contribution of this strategy.
- Figure 6 shows residual reduction against the number of accepted nodes, but does not report how many candidates were evaluated before each node was accepted. The acceptance rate, average number of trials per node, and residual reduction against actual runtime would help assess the efficiency of the construction procedure.
- The runtime comparisons use different hardware and network settings. PI-SCM and NNBM are evaluated on CPU, whereas PINN and PISCN are evaluated on GPU. In addition, PI-SCM uses sine activation while the PINN baselines use tanh. For a fair comparison, all methods should be evaluated on the same hardware (either CPU or GPU), preferably with consistent activation functions, to better assess the computational advantage of the proposed method.
- The advantage over NNBM is less clear than the paper suggests. Table 1 shows that PI-SC-III is slower than NNBM on all five benchmarks. Although the accuracy improvement is substantial for Helmholtz and Allen–Cahn, it is relatively modest for reaction–diffusion and Bratu. Since NNBM constructs its basis functions at once, whereas PI-SC-III requires incremental selection and repeated corrections, an error-versus-total-training-time comparison would better establish whether the additional computational cost is justified.
- The theoretical results concern discrete residual convergence rather than solution accuracy. Theorem 1 proves that the residual at the collocation points converges to zero under a strong full-row-rank assumption. However, a small residual does not necessarily imply a small solution error, and the theorem does not establish convergence to the true PDE solution.
- The generality of the method for nonlinear PDEs is not sufficiently demonstrated. Although the paper claims to handle nonlinear differential equations, it does not test standard nonlinear problems such as the Burgers or Euler equations. It remains unclear how the method performs for nonlinear PDEs with sharp gradients or discontinuities.
Questions for authors
Please refer to the concerns in the weaknesses section.
6 comments
W3.1 The novelty over NNBM needs further clarification. Both methods use random basis functions and linearized least-squares optimization for nonlinear PDEs. The global correction in PI-SC-III is closely related to the Gauss–Newton procedure used in NNBM. The main difference appears to be the incremental node selection and correction strategy.
Response:
Thank you for highlighting the connection between the global correction in PI-SC-III and the Gauss–Newton method. The global correction solves a linearized least squares problem within the currently accepted basis and shares a mathematical connection with a Gauss–Newton update. PI-SCM embeds this mechanism into physics-informed incremental construction, with three principal contributions.
1. Physical residuals directly guide basis construction. NNBM optimizes output weights within a preconfigured set of random basis functions. PI-SCM configures new nodes according to the current residuals of the governing equations, initial and boundary conditions, and optional observations. Combined with adaptive sampling scales, this progressively constructs a feature space suited to the current problem. The supervisory inequality determines candidate acceptance according to the actual contribution to the true nonlinear loss. Through this mechanism, every accepted new node reduces the current residual norm, directly linking network growth to improvement in the physics-informed objective.
2. Analytical weight updates are coupled with supervisory construction. NNBM uses iterative Gauss–Newton optimization to update output weights within a given basis. PI-SCM solves one linear least squares problem for each candidate configuration and each windowed or global correction, then combines directional damping with true residual evaluation to determine an acceptable update. The algorithm subsequently advances node construction, alternating expansion of the feature space with weight updates. This design unifies physics-informed basis construction and efficient analytical training within a single solution procedure.
3. Convergence analysis covers the coupled construction and correction process. We analyze the dependence of residual mappings on the current state and the Taylor remainder introduced by linearization. We establish monotonic reduction and convergence of the residual under the stated assumptions. We further prove that windowed and global corrections preserve the convergence achieved by local construction, providing theoretical support for the complete process of node selection, analytical updates, and weight correction.
W3.2 An ablation study using the same randomly generated basis functions with and without incremental selection would help clarify the contribution of this strategy.
Response:
Thank you for suggesting further evaluation of the contribution of incremental construction. Node selection and output weight updates in PI-SCM are coupled, with each accepted node changing the current true nonlinear residual and thereby influencing the configuration and selection of subsequent nodes.
To further evaluate the proposed mechanisms, we compare the solution accuracy of PI-SC-III and NNBM/PIELM at matched hidden layer widths across eight benchmarks in Figure 10 of Appendix D.8. This comparison evaluates the combined effects of their respective construction and output weight update procedures under matched node budgets.
PI-SC-III generally achieves lower mean RMSE, supporting the joint effectiveness of physics-informed incremental construction and analytical weight updates in making better use of the available model capacity and constructing a feature space adapted to the problem.
W4 Figure 6 shows residual reduction against the number of accepted nodes, but does not report how many candidates were evaluated before each node was accepted. The acceptance rate, average number of trials per node, and residual reduction against actual runtime would help assess the efficiency of the construction procedure.
Response:
Thank you for your attention to candidate evaluation costs and construction efficiency. We further clarify the candidate search mechanism. PI-SCM adds one node to the network at each successful construction stage, while each search round generates and evaluates a batch of candidates. If the batch contains candidates satisfying the supervisory inequality for the true residual, the algorithm selects the one yielding the greatest residual reduction. Otherwise, it adjusts the sampling scale, supervisory parameter, and damping factor according to the prescribed procedure and evaluates new candidate batches.
Therefore, when the first batch succeeds, each newly added node requires candidate evaluations. If configuring the -th node requires batches, the actual number of candidate evaluations is , and the ratio of selected nodes to evaluated candidates at that stage is . In our implementation, candidates within the same construction stage share the current network state and residual information. Their node outputs, derivatives, analytical coefficients, and true residuals are evaluated through parallel batched tensor operations.
We have also added comparisons of solution error against actual training time across eight benchmarks in Appendix D.8 (Figure 9). For PI-SC-III, the reported training time includes the generation and evaluation of all candidates, supervisory selection, and output weight corrections, allowing the computational cost of candidate search to be assessed together with the achieved solution accuracy.
W5.1 The runtime comparisons use different hardware and network settings. PI-SCM and NNBM are evaluated on CPU, whereas PINN and PISCN are evaluated on GPU.
Response:
Thank you for your comment on the experimental hardware configuration. Our experiments evaluate the practical training efficiency and computational resource requirements of different solution procedures. PI-SCM, PIELM, and NNBM run on the CPU, while PINN and PISCN are trained on the GPU. This configuration gives methods relying on backpropagation access to GPU acceleration, while demonstrating the ability of PI-SCM to achieve high accuracy and fast training on a standard CPU.
NNBM [1] and LocELM [2] also report performance evaluations on CPUs. For comparisons across devices, Frozen-PINN [3] likewise compares its method running on a CPU with PINNs trained on GPUs. These studies provide precedents for comparing the practical training efficiency and resource requirements of PDE solvers using hardware configurations suited to their computational workflows.
W5.2 In addition, PI-SCM uses sine activation while the PINN baselines use tanh.
Response:
Thank you for your comment on the activation function settings. Our PINN baselines use tanh following the recommendations of Wang et al. [4] . This paper explicitly recommends tanh for constructing continuously differentiable neural representations, providing a literature basis for our baseline configuration.
Appendix D.10 evaluates PI-SCM with sine, tanh, and sigmoid across eight benchmarks, with the hidden parameter sampling scale tuned separately for each activation function. The results show that PI-SCM also achieves high solution accuracy with tanh. In particular, tanh yields the lowest mean RMSE among the three activations for Allen–Cahn and Bratu, reaching and , respectively. These results demonstrate PI-SCM’s ability to obtain accurate solutions using the same tanh activation as the PINN baselines, as well as its adaptability to different activation functions.
We appreciate your time and detailed feedback. Thank you for recognizing our combination of stochastic node construction and analytical output weight updates for solving nonlinear differential equations without backpropagation, together with the convergence analysis and extensive numerical evaluation. We have revised the manuscript and expanded the experimental evaluation. Revisions to the manuscript are highlighted in blue for ease of reference. Our point-by-point responses to your concerns and questions are provided below.
Response:
Thank you for your suggestion regarding the effective dimensionality of the five-dimensional benchmark. We have added a nonlinear reaction–diffusion problem involving all five spatial variables in Appendix D.4.3. Its solution explicitly depends on all five spatial variables, with each spatial direction contributing a nonzero term to the diffusion operator. We evaluated the method under both unsupervised and semi-supervised settings. As shown in Table 3, PI-SC-III achieves mean RMSEs of 6.30×10−3 and 6.25×10−3, respectively, providing experimental evidence of PI-SCM’s ability to solve nonlinear PDEs involving all five spatial variables.
Response:
Thank you for your suggestion regarding activation function selection. Your comment motivated us to repeat and expand the activation study in Appendix D.10. The updated study covers eight benchmarks, and systematically compares sine, sigmoid, and tanh activations.
Considering the different sensitivities of activation functions and their derivatives to hidden parameter scales, we tuned the scale parameter a, which controls the sampling range of hidden parameters, separately for each activation function. This allows each activation to operate at an appropriate scale and fully realize its performance.
The results in Table 8 show that sigmoid and tanh also achieve high solution accuracy on the Helmholtz and reaction–diffusion cases highlighted in your comment. These findings further demonstrate the effectiveness of PI-SCM across different activation functions, showing that its physics-informed construction and analytical update mechanisms can support accurate solutions using multiple choices of activation.