- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper presentes a novel architecture to learn the physical parameter to solution map, especially in systems when the responses changes sharply and non-linearly. The authors propose two components: NRGD and GANR. The former is used to expose the non-lin response structute that a surrogate must represent; while the latter is aimed at improving the ability to represent the system non-linearity. Overall, the two are complementary and integrated into GA-NAR, in particular, the information from the NRGD is used to adapt the network nonlin computation using informayion extracted from the physical system. The method is evaluating separately each component and then, the overall performance of GA-NAR on three datasets: IEEE-118 AC, Kuramoto 64 and Non linear Heat equation.
Critical strengths
- The paper presents a physically grounded architecture to learn the physical parameter to solution map, especially in systems when the responses changes sharply and non-linearly. The description of the method is very detailed, which on one hand makes the paper more convincing, on the other hand, makes the paper more difficult to understand.
- The authors have a structured experimental evaluation, evaluating each component and the overall performance.
Critical weaknesses
- Some sections would need strong improvements from the presentation point of view: the related work section is little informative and sounds more like a collection of papers the authors want to cite, without a discussion on why they are relevant to the current work; the numerical results and conclusions are just few sentences with little interpretation of the results, more a description of the the three tables and a conclusion without any discussion on the results.
- The numerical verification of the final GA-NAR architecture with other state-of-the-art methods is missing. The authors have focused on activation-matched baselines.
Questions for authors
- The paper should be improved in the presentation point of view, especially in the related work section and the numerical results and conclusions sections.
- The verification of the method against other approaches is missing, GA-NAR performance assessment would be more convincing if it was compared with other state-of-the-art methods.
5 comments
I would like to thank the authors for the efforts in improving the presentation of the results. Also, I appreciate their detailed response and for conducting additional comparisons with DeepONet, FNO, and PINN.
However, I could not find these new comparisons reported or discussed in the paper. In particular, Section 5.4 still focuses on activation-matched baselines, which is perfectly reasonable. Nevertheless, since you performed this analysis I think it would be a valuable improvement of the work even as supplementary material...
Quick follow up question: the gains relative to the matched random-projector modules are quite small in several cases, could you comment on this?
Thank you for taking the time to read our previous response and examine the additional experiments so carefully. We are grateful for your positive assessment of the additional comparisons with DeepONet, FNO, and PINN.
Thank you also for suggesting that we include the comparisons with other state-of-the-art methods in the appendix. We initially presented the new results in our response because we wanted to hear your assessment of the experimental setup and how convincingly the results supported our conclusions before incorporating them into the manuscript. Following your suggestion, we have added Appendix K to the revised manuscript, where we describe the experimental setup and report the results of these comparisons.
We appreciate your follow-up about the small gains over the matched random-projector module. As you suggested, we kept the nonlinear module, parameter count, and training procedure fixed. Both variants use the same physical responses to construct their training targets, with the response allocation recomputed for each projector. The matched comparison therefore measures only the incremental benefit of using physically aligned subspaces and their associated response allocation. This incremental benefit is distinct from the overall gain of the full GA-NAR module over the original backbone. Small gains over the matched control can therefore coexist with substantially larger gains over the original backbone, because physical alignment and its associated response allocation account for only one part of GA-NAR’s overall improvement.
On the IEEE-118 power-flow problem, GA-NAR reduced the test mean squared error (MSE) of the one-dimensional Fourier neural operator (FNO-1D) by 14.56% relative to the original backbone and by 0.20% relative to the matched random-projector module. For the physics-informed neural network (PINN) on Kuramoto-64, the corresponding reductions were 37.34% and 0.08%. In both cases, the random-projector module already delivered nearly all of the improvement over the original backbone.
The activation experiments show that the size of this gain varies even within a fixed architecture. On Kuramoto-64, the physical projector reduced test MSE relative to the random projector by 53.89% with ReLU and 32.39% with GELU. The corresponding reductions in the governing-equation residual, which measures how closely the predictions satisfy the physical equations, were 62.25% and 24.07%. On the nonlinear heat benchmark with Softplus, the physical projector reduced test MSE by 10.17% relative to the random projector.
Thank you again for your careful reading and for raising this question. We hope our response clarifies why the gains over the matched control are small in several cases and addresses your question about how much physical alignment contributes in the tested settings.
We appreciate your recognition of our structured evaluation of individual components and overall performance. Below, we address your concerns with new experiments and explanations.
1. Presentation and interpretation. The nine-page limit led us to compress the related-work discussion and interpretation of results. In the resubmitted PDF, we now explain why the cited approaches matter to GA-NAR and what the numerical results support. Revised passages are marked in blue for easy identification.
2. Comparison with other state-of-the-art methods. GA-NAR is a general nonlinear enhancement for physical surrogates: its response-geometry conditioning can be incorporated into operator architectures and physics-informed networks. Activation-matched baselines hold the backbone fixed and directly test this enhancement. We therefore believe the original paper's comparison design is well motivated.
At the same time, we thank you for suggesting comparisons with other state-of-the-art methods and agree that they are valuable. We therefore added paired tests with DeepONet, FNO, and parameter-conditioned PINN on IEEE-118, Kuramoto-64, and Heat FEM. Each row fixes the backbone and task, comparing the original, a GA-NAR-style nonlinear module, and an equal-capacity random-projector module. The module arms share initialization, data, budget, optimizer, and validation rule. Random-projector targets are also derived from physical responses; the control tests projector orientation within the matched module. Inference needs no solver or Jacobian query. Pretrained or PDE-conditioned models use extra data or equation inputs and require a separate resource-matched comparison.
The table reports mean squared error (MSE) in standardized predicted states over three paired training seeds; lower is better. Gains are percentage reductions in mean MSE relative to the named reference. FNO-1D/2D indicate one-/two-dimensional spectral representations.
The comparison shows why GA-NAR is useful beyond the original activation study. Its nonlinear module lowers mean prediction error relative to the original backbone in all nine settings, including operator and physics-informed networks. Against an equal-capacity module, the response-derived projector has a clear advantage in Kuramoto DeepONet, with additional gains in IEEE PINN and Heat FNO-2D. These results support a contribution from physical response geometry in those settings. Heat PINN also shows that response conditioning can help when the network already trains with a governing-equation residual loss.
The effects differ across systems, clarifying where response conditioning contributes most. IEEE FNO and Kuramoto PINN show large module-level gains over their frozen originals, with close physical and random projector results; Heat DeepONet shows a smaller increment. Heat PINN favors the physical module in mean MSE, with variation across seeds. On a stronger Kuramoto FNO baseline, the physical and random modules are close in MSE, while the physical module lowers the mean governing-equation residual, a measure of physical consistency, in all three seeds. Together, the prediction and residual results show how the enhancement behaves across distinct physical tasks.
Taken together, these same-backbone comparisons show that GA-NAR's response-conditioned nonlinearity improves established DeepONet, FNO, and PINN surrogates across all three physical systems. Its additional gains over matched-capacity modules in several settings further demonstrate the value of adapting nonlinear computation to physical response geometry. Thus, the benefit of GA-NAR extends beyond activation-matched baselines to other strong scientific machine-learning methods.
Thank you again for your time and thoughtful review. Your comments have helped us improve the manuscript's clarity. We hope the new explanations and results address your concerns. Our anonymous repository linked in the submission will soon provide further explanations, experimental settings, and complete results.