acceptodds
Under review as a conference paper at ICLR 2027

GA-NAR: Geometry-Adaptive Nonlinearity Allocation and Representation for Complex Physical Systems

Abstract

Many complex physical systems exhibit a recurring dominant nonlinear-response structure whose local intensity and dominant/complement allocation vary across operating conditions, while standard supervised surrogates typically apply a fixed pattern of nonlinear computation. We introduce GA-NAR (Geometry-Adaptive Nonlinearity Allocation and Representation), a backbone-compatible framework that adapts neural nonlinear computation to the structure of the physical parameter-to-solution map. GA-NAR comprises two coupled components: Nonlinear Response Geometry Decomposition (NRGD), which identifies a recurring Dominant Nonlinear-Response Subspace together with state-dependent response intensity and subspace allocation, and Geometry-Adaptive Nonlinear Representation (GANR), which uses this information to organize nonlinear computation across subspaces and network depth. Independent experiments on NRGD across five systems verify reliable recovery of the proposed response descriptors, while separate no-physics and shared-physics factorial experiments show that GANR achieves the lowest NRMSE on four of five auxiliary systems under both conditions. The complete GA-NAR framework reduces standardized test MSE in all 18 activation–system comparisons, with median reductions of 11.5%, 2.1%, and 2.8% on IEEE-118, Kuramoto-64, and nonlinear heat FEM, respectively, alongside larger activation-specific gains in several cases. Physical residuals also decrease for most tested activations, showing that the predictive gains generally extend to physical consistency. We will release codes at https://anonymous.4open.science/r/GA-NAR.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Physical Systems, Parameter-to-Solution Map, Nonlinear Response Geometry, Nonlinear Representation.

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GypJOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper presentes a novel architecture to learn the physical parameter to solution map, especially in systems when the responses changes sharply and non-linearly. The authors propose two components: NRGD and GANR. The former is used to expose the non-lin response structute that a surrogate must represent; while the latter is aimed at improving the ability to represent the system non-linearity. Overall, the two are complementary and integrated into GA-NAR, in particular, the information from the NRGD is used to adapt the network nonlin computation using informayion extracted from the physical system. The method is evaluating separately each component and then, the overall performance of GA-NAR on three datasets: IEEE-118 AC, Kuramoto 64 and Non linear Heat equation.

Critical strengths
  • The paper presents a physically grounded architecture to learn the physical parameter to solution map, especially in systems when the responses changes sharply and non-linearly. The description of the method is very detailed, which on one hand makes the paper more convincing, on the other hand, makes the paper more difficult to understand.
  • The authors have a structured experimental evaluation, evaluating each component and the overall performance.
Critical weaknesses
  • Some sections would need strong improvements from the presentation point of view: the related work section is little informative and sounds more like a collection of papers the authors want to cite, without a discussion on why they are relevant to the current work; the numerical results and conclusions are just few sentences with little interpretation of the results, more a description of the the three tables and a conclusion without any discussion on the results.
  • The numerical verification of the final GA-NAR architecture with other state-of-the-art methods is missing. The authors have focused on activation-matched baselines.
Questions for authors
  • The paper should be improved in the presentation point of view, especially in the related work section and the numerical results and conclusions sections.
  • The verification of the method against other approaches is missing, GA-NAR performance assessment would be more convincing if it was compared with other state-of-the-art methods.
5 comments

We appreciate your recognition of our structured evaluation of individual components and overall performance. Below, we address your concerns with new experiments and explanations.

1. Presentation and interpretation. The nine-page limit led us to compress the related-work discussion and interpretation of results. In the resubmitted PDF, we now explain why the cited approaches matter to GA-NAR and what the numerical results support. Revised passages are marked in blue for easy identification.

2. Comparison with other state-of-the-art methods. GA-NAR is a general nonlinear enhancement for physical surrogates: its response-geometry conditioning can be incorporated into operator architectures and physics-informed networks. Activation-matched baselines hold the backbone fixed and directly test this enhancement. We therefore believe the original paper's comparison design is well motivated.

At the same time, we thank you for suggesting comparisons with other state-of-the-art methods and agree that they are valuable. We therefore added paired tests with DeepONet, FNO, and parameter-conditioned PINN on IEEE-118, Kuramoto-64, and Heat FEM. Each row fixes the backbone and task, comparing the original, a GA-NAR-style nonlinear module, and an equal-capacity random-projector module. The module arms share initialization, data, budget, optimizer, and validation rule. Random-projector targets are also derived from physical responses; the control tests projector orientation within the matched module. Inference needs no solver or Jacobian query. Pretrained or PDE-conditioned models use extra data or equation inputs and require a separate resource-matched comparison.

The table reports mean squared error (MSE) in standardized predicted states over three paired training seeds; lower is better. Gains are percentage reductions in mean MSE relative to the named reference. FNO-1D/2D indicate one-/two-dimensional spectral representations.

SystemBackboneOriginal+GA-NARMatched randomGain vs originalGain vs random
IEEE-118DeepONet1.162093e-051.150771e-051.161483e-05+0.97%+0.92%
IEEE-118FNO-1D1.053946e-059.005099e-069.023167e-06+14.56%+0.20%
IEEE-118PINN9.877070e-059.669337e-059.875704e-05+2.10%+2.09%
Kuramoto-64DeepONet1.767051e-021.678558e-021.767168e-02+5.01%+5.01%
Kuramoto-64FNO-1D7.215122e-037.161206e-037.159552e-03+0.75%-0.02%
Kuramoto-64PINN1.737157e-031.088454e-031.089321e-03+37.34%+0.08%
Heat FEMDeepONet4.300818e-064.298852e-064.299606e-06+0.05%+0.02%
Heat FEMFNO-2D3.459924e-063.273480e-063.321364e-06+5.39%+1.44%
Heat FEMPINN1.865184e-041.315994e-041.421128e-04+29.44%+7.40%

The comparison shows why GA-NAR is useful beyond the original activation study. Its nonlinear module lowers mean prediction error relative to the original backbone in all nine settings, including operator and physics-informed networks. Against an equal-capacity module, the response-derived projector has a clear advantage in Kuramoto DeepONet, with additional gains in IEEE PINN and Heat FNO-2D. These results support a contribution from physical response geometry in those settings. Heat PINN also shows that response conditioning can help when the network already trains with a governing-equation residual loss.

The effects differ across systems, clarifying where response conditioning contributes most. IEEE FNO and Kuramoto PINN show large module-level gains over their frozen originals, with close physical and random projector results; Heat DeepONet shows a smaller increment. Heat PINN favors the physical module in mean MSE, with variation across seeds. On a stronger Kuramoto FNO baseline, the physical and random modules are close in MSE, while the physical module lowers the mean governing-equation residual, a measure of physical consistency, in all three seeds. Together, the prediction and residual results show how the enhancement behaves across distinct physical tasks.

Taken together, these same-backbone comparisons show that GA-NAR's response-conditioned nonlinearity improves established DeepONet, FNO, and PINN surrogates across all three physical systems. Its additional gains over matched-capacity modules in several settings further demonstrate the value of adapting nonlinear computation to physical response geometry. Thus, the benefit of GA-NAR extends beyond activation-matched baselines to other strong scientific machine-learning methods.

Thank you again for your time and thoughtful review. Your comments have helped us improve the manuscript's clarity. We hope the new explanations and results address your concerns. Our anonymous repository linked in the submission will soon provide further explanations, experimental settings, and complete results.

I would like to thank the authors for the efforts in improving the presentation of the results. Also, I appreciate their detailed response and for conducting additional comparisons with DeepONet, FNO, and PINN.

However, I could not find these new comparisons reported or discussed in the paper. In particular, Section 5.4 still focuses on activation-matched baselines, which is perfectly reasonable. Nevertheless, since you performed this analysis I think it would be a valuable improvement of the work even as supplementary material...

Quick follow up question: the gains relative to the matched random-projector modules are quite small in several cases, could you comment on this?

Thank you for taking the time to read our previous response and examine the additional experiments so carefully. We are grateful for your positive assessment of the additional comparisons with DeepONet, FNO, and PINN.

Thank you also for suggesting that we include the comparisons with other state-of-the-art methods in the appendix. We initially presented the new results in our response because we wanted to hear your assessment of the experimental setup and how convincingly the results supported our conclusions before incorporating them into the manuscript. Following your suggestion, we have added Appendix K to the revised manuscript, where we describe the experimental setup and report the results of these comparisons.

We appreciate your follow-up about the small gains over the matched random-projector module. As you suggested, we kept the nonlinear module, parameter count, and training procedure fixed. Both variants use the same physical responses to construct their training targets, with the response allocation recomputed for each projector. The matched comparison therefore measures only the incremental benefit of using physically aligned subspaces and their associated response allocation. This incremental benefit is distinct from the overall gain of the full GA-NAR module over the original backbone. Small gains over the matched control can therefore coexist with substantially larger gains over the original backbone, because physical alignment and its associated response allocation account for only one part of GA-NAR’s overall improvement.

On the IEEE-118 power-flow problem, GA-NAR reduced the test mean squared error (MSE) of the one-dimensional Fourier neural operator (FNO-1D) by 14.56% relative to the original backbone and by 0.20% relative to the matched random-projector module. For the physics-informed neural network (PINN) on Kuramoto-64, the corresponding reductions were 37.34% and 0.08%. In both cases, the random-projector module already delivered nearly all of the improvement over the original backbone.

The activation experiments show that the size of this gain varies even within a fixed architecture. On Kuramoto-64, the physical projector reduced test MSE relative to the random projector by 53.89% with ReLU and 32.39% with GELU. The corresponding reductions in the governing-equation residual, which measures how closely the predictions satisfy the physical equations, were 62.25% and 24.07%. On the nonlinear heat benchmark with Softplus, the physical projector reduced test MSE by 10.17% relative to the random projector.

Thank you again for your careful reading and for raising this question. We hope our response clarifies why the gains over the matched control are small in several cases and addresses your question about how much physical alignment contributes in the tested settings.

Reviewer_LrKzOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes GA-NAR, a framework for adapting neural surrogate computation to the nonlinear-response geometry of physical systems. NRGD extracts a global Dominant Nonlinear-Response Subspace (DRS), local response intensity, and dominant/complement allocation from finite-scale Jacobian variations. GANR uses these descriptors through dual-subspace encoding and RMS-matched layer-wise preactivation modulation, while preserving the backbone activation family. The paper evaluates descriptor prediction and response-aware representation separately, and end-to-end experiments show that GA-NAR reduces standardized test MSE in every activation–system comparison.

Critical strengths

S1. Principled coupling of physical response geometry and neural nonlinearity. NRGD turns finite-scale Jacobian variation into response descriptors, which GANR maps onto layer-wise preactivation modulation without changing the activation (Sec. 4.1; Fig. 1). This coupling is a distinctive way to inject physical response structure into a surrogate.

S2. Consistent gains across activation families. GA-NAR lowers standardized test MSE in every activation–system comparison and reduces the physical residual in most of them (Table 3). This consistency supports the claim that, within the tested residual backbone, GA-NAR acts as an add-on across activation families rather than an activation-specific fix.

Critical weaknesses

W1. Main and controlled results disagree on heat FEM. Despite matching baselines and the same reported seeds and training budget, Table 3 and Tables 9–10 give substantially different Full GA-NAR results, including opposite Softplus residual trends, and the paper does not say whether the configurations differ. This calls into question both the trade-off reported in Sec. 5.4 and the App. H attribution, which is clearest in this setting.

W2. Limited attribution of the gains to physics-derived descriptors. The App. H controls remove the descriptors or change how they are used, but none keeps the GANR structure while replacing them with non-physical ones. Since the paper's stated distinction from generic conditioning is the physical origin of these descriptors (Sec. 1), the gains may stem from the structured modulation rather than the descriptors' physical content.

Questions for authors

Q1 (Related to W1; clarification). Could the authors reconcile the heat FEM results between Table 3 and Tables 9–10, particularly the opposite Softplus physical-residual trends? Were different configurations, training procedures, or evaluation protocols used, and which results are authoritative?

Q2 (Related to W2; clarification). How are ρG⋆(x)\rho^\star_G(x) and πG⋆(x)\boldsymbol{\pi}^\star_G(x) in Eq. (96) obtained for held-out test inputs, given that NRGD targets are not queried at inference (Sec. 4.1; App. J)? Relatedly, which scales, response ranks, and gate outcomes did App. J yield for the end-to-end benchmarks?

Q3 (Related to W2; clarification). How were the settings in Appendix H selected, and how representative are they of the full Table 3 evaluation?

Q4 (Related to W2; suggestion). On the App. H settings, could the authors evaluate the final GANR architecture with (i) a rank-matched random projector and allocation targets recomputed for that projector, or (ii) uniform allocation supervision while keeping the original DRS projector fixed? Keeping capacity and modulation structure unchanged would help isolate the contributions of the physics-derived subspace orientation and allocation information (cf. App. B.5).

If any part of this review reflects a misunderstanding of the paper, I would appreciate the authors pointing it out, and I will revise my assessment accordingly.

11 comments

We appreciate your recognition of our geometry-conditioned nonlinear representation and consistent cross-activation gains. We address your questions with protocol clarification and matched experiments.

1. Heat FEM results in Tables 3 and 9–10 (W1/Q1). Table 3 evaluates six-activation end-to-end GA-NAR with NRGD supervision (λρ=λπ=0.5\lambda_\rho=\lambda_\pi=0.5). Appendix H evaluates controlled attribution without these auxiliary losses and with physics-weighted checkpoint selection. These distinct protocols explain their Full values. All figures below follow the submitted paper (mean ± SD).

Heat FEM / FullMSEPhysical residual
Table 3 / ReLU0.0234 ± 0.0007 (×10−4\times10^{-4})0.0402 ± 0.0011 (×10−3\times10^{-3})
Table 3 / Softplus0.0148 ± 0.0005 (×10−4\times10^{-4})0.1169 ± 0.0033 (×10−3\times10^{-3})
Appendix H / ReLU1.283 ± 0.104 (×10−6\times10^{-6})3.321 ± 0.123 (×10−5\times10^{-5})
Appendix H / Softplus0.461 ± 0.045 (×10−6\times10^{-6})2.335 ± 0.086 (×10−5\times10^{-5})

In Table 3, Heat/Softplus reduces MSE by 66.8%, while its residual changes from 0.0720 to 0.1169 (×10−3\times10^{-3}). This Heat-specific prediction–residual trade-off follows from temperature-dependent conduction and cubic reaction: equation residuals depend on spatial gradients and local nonlinear errors, whereas MSE averages state errors. In Appendix H, Full GA-NAR improves both metrics.

2. Inference-time descriptors and geometry resolution (W2/Q2). NRGD constructs PRP_R and the ρG⋆,πG⋆\rho_G^\star,\pi_G^\star targets from training geometry. At inference, GANR predicts intensity and allocation from the input and frozen projector, without solver or Jacobian calls. Eq. (96) uses descriptors interpolated from four training centers.

Appendix J's 90%-mass setting is a conservative reference for resolution analysis. The additional evaluation adopts a training-only, resolution-aware 80%-mass response core, with h=0.02h=0.02 and ranks 4/1/3 for IEEE-118/Kuramoto-64/Heat FEM. This retains reliably resolved dominant directions while preserving the complementary channel. The criterion uses training geometry, not test predictions. Numerical stability checks support all three configurations, with an exact-trace check for IEEE-118. We have added a detailed explanation of this evaluation setting to the resubmitted PDF.

3. Appendix H coverage (Q3). The seven settings were selected to cover all three physical systems and contrasting activation mechanisms: piecewise-linear ReLU, smooth GELU/Softplus, and gated SwiGLU. The matched four-way comparison includes Kuramoto/ReLU and Heat/Softplus. The subset covers 3/6 IEEE-118, 2/6 Kuramoto-64, and 2/6 Heat FEM activation pairs in Table 3. To assess coverage against the full evaluation, we checked the other 11 pairs: MSE improves in 7/7 Appendix H-covered pairs and 11/11 remaining pairs. Appendix H tests attribution across these settings; Table 3 verifies gains across the six-activation grid.

4. Physical-geometry controls (W2/Q4). Thank you for proposing both GANR controls. P uses physical geometry. R uses a rank-matched random projector and recomputes training allocation targets. U keeps physical DRS and intensity and fixes inference allocation at (0.5,0.5)(0.5,0.5). Capacity, budget, and seeds match. Positive gains favor P (three-seed means).

System / activationMSE vs RMSE vs UResidual vs RResidual vs U
IEEE-118 / ReLU+13.92%+0.16%+1.40%+11.85%
IEEE-118 / GELU+5.18%+3.50%+1.88%−4.48%
IEEE-118 / SwiGLU+2.11%+1.47%−9.69%−15.77%
Kuramoto-64 / ReLU+53.89%+43.95%+62.25%+37.53%
Kuramoto-64 / GELU+32.39%+4.22%+24.07%+0.98%
Heat FEM / ReLU−5.85%+4.23%−13.72%−8.42%
Heat FEM / Softplus+10.17%+4.02%+3.28%+0.81%

P improves MSE relative to R in 6/7 comparisons and to U in 7/7; residual gains occur in 5/7 and 4/7. Kuramoto-64/ReLU shows residual reductions of 62.25% versus R and 37.53% versus U, demonstrating the value of physical orientation and adaptive allocation. We have included this additional experiment in the resubmitted PDF.

For IEEE-118/SwiGLU, AC power balance couples voltage magnitudes and angle differences, so lower MSE can coexist with larger power mismatches. In Heat FEM/ReLU, R leads on both metrics and U on residual. Temperature-dependent flux and cubic reaction make heat balance sensitive to the spatial pattern of temperature and gradient errors.

We sincerely thank you again for the time and effort devoted to reviewing our manuscript and for your thoughtful comments, which have improved its quality and clarity. We hope these explanations and additional results address your concerns. Our anonymous repository linked in the submission will soon provide detailed settings, explanations, and complete results.

Reviewer_LrKzOpenReview ↗

Response to Authors

Thank you for your response and the revised manuscript. The protocol differences explain why the heat FEM results differ. In the new control experiments, the physical geometry improves MSE in most settings, but the residuals show mixed results. However, since the controlled ablation experiment used GA-NAR trained without NRGD auxiliary losses, it fails to demonstrate that the local descriptors control the modulation. Yet, the revised main text makes that claim. The new control experiments also appear to use a different rank rule than the main method. Therefore, it is not yet clear that the improvements in the main experiments are due to the physical descriptors. Overall, I maintain my score of 2.

Thank you for reading our previous response and reviewing the additional experiments so carefully. Your follow-up made us realize that our previous reply left two points open to misinterpretation. We appreciate the opportunity to clarify how the model uses physical geometry when the auxiliary losses are disabled and which rank rule we used for the reported experiments.

Disabling the two NRGD auxiliary losses removes the penalties for matching the local response intensity and the allocation of response between the two subspaces. GANR still uses the fixed projector estimated from physical responses at training points to split each input into dominant and complementary components. Both components are encoded and used to make the prediction, so physical response geometry continues to enter the model when the auxiliary losses are disabled.

The end-to-end results in the revised main text and the new control experiments both use the 80% spectral-mass rule. Using training data alone, we retain the fewest leading response directions whose eigenvalues sum to at least 80% of the total. This gives ranks of 4 for IEEE-118, 1 for Kuramoto-64, and 3 for nonlinear heat FEM. The physical and random projectors have the same rank within each system, and these ranks are fixed before held-out evaluation.

The 90% setting in Appendix J serves as a conservative reference for resolution analysis. It asks whether a broader set of response directions can be resolved from a finite set of physical probes. Requiring 90% coverage can place the cutoff among weak, closely spaced eigenvalues, where small changes in the response estimate can change which directions enter the dominant subspace. At 80%, the subspaces estimated independently from two halves of the training geometry agree closely. Their overlaps are 0.988, 0.976, and 0.934, respectively, all above the 0.90 requirement, with 1 indicating identical subspaces. The relative changes introduced by normalizing the response operators to have nonnegative eigenvalues that sum to one are 0.094, 0.222, and 0.024, all below the 0.25 tolerance. These checks support using the 80% cutoff to identify the dominant directions for these experiments. The other directions remain available to the model through the complementary channel.

Thank you again for taking the time to raise these questions. We hope the details above clarify the experimental setup and address your concerns.

Reviewer_ABokOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

GA-NAR incorporates physical response information into parameter-to-state surrogate models. NRGD summarizes finite-scale Jacobian variation through a dominant response subspace, local intensity, and subspace allocation. GANR uses these descriptors to guide layer-wise preactivation modulation. The evaluation combines auxiliary descriptor and representation studies with end-to-end experiments on AC power flow, Kuramoto equilibria, and steady nonlinear heat conduction.

Critical strengths
  1. Distinguishing nonlinear response from first-order sensitivity provides an interesting motivation for surrogate design. The Hessian–Gram interpretation gives the descriptors a concrete mathematical basis.

  2. The paper includes useful sampling-budget diagnostics, multiple seeds, physical-residual evaluation, and capacity and generic-conditioning controls. The IEEE-118 results are encouraging, with a median MSE reduction of 11.5% across activation families.

Critical weaknesses
  1. The incremental performance benefit does not yet convincingly justify the framework’s complexity. GA-NAR combines derivative-based descriptor construction, dual-subspace encoding, auxiliary supervision, and layer-wise modulation. Table 3 reports improvements across all 18 activation–system pairs, but the median reductions on Kuramoto and Heat FEM are only 2.1% and 2.8%, respectively. As Appendix G acknowledges, these comparisons include additional capacity. Appendix H provides more informative controls: Full GA-NAR wins in most settings, but both controls outperform it on Kuramoto/ReLU, and the Heat/ReLU variants are nearly indistinguishable (Table 9). Heat/Softplus shows a substantial additional benefit, although the Full GA-NAR Heat results differ between Tables 3 and 9 without a clear configuration-level reconciliation. Thus, the evidence suggests useful but setting-dependent gains, while the advantage of the complete mechanism over simpler alternatives remains insufficiently established.

  2. The demonstrated applicability is limited relative to the methodological ambition and additional requirements. The final experiments cover three fixed-system equilibrium or steady-state mappings: fixed-topology power flow, a fixed Kuramoto graph, and heat conduction with a fixed domain and boundary condition (Appendix E.2; F.3). These are legitimate surrogate tasks, but they offer limited evidence across operating regimes or system configurations. The auxiliary GANR studies use a different formulation, as explicitly acknowledged in Appendix C.6 and F.2, so they do not directly broaden validation of the final adapter. Furthermore, the method requires offline physical solves and derivative evaluations (Section 4.1; Appendix A.1), but their total cost and benefit relative to additional supervised data are not clearly quantified. Since the response subspace depends on sampled operating conditions, downstream robustness to geometry-center selection is also relevant; the descriptor-level stability checks in Appendix B.3 and J do not fully answer this question.

Questions for authors
  1. Tables 3 and 9: After accounting for table scaling, Full GA-NAR’s Heat FEM MSE changes from (2.34\times10^{-6}) to (1.283\times10^{-6}) for ReLU, and from (1.48\times10^{-6}) to (0.461\times10^{-6}) for Softplus. What differences in configuration or model selection explain these results?

  2. Appendix B.3 and J: Do the descriptor-stability findings translate into consistent end-to-end prediction gains when geometry centers are independently resampled, with the supervised splits and construction budget held fixed?

  3. Section 4.1 and Appendix A.1: What is the total offline descriptor-construction cost relative to training-data generation and model training? Could comparison with allocating a similar physical-computation budget to additional supervised samples clarify the practical benefit?

2 comments

We appreciate your recognition of the Hessian–Gram motivation and systematic evaluation. We address your three questions with the reported results and new experiments.

1. Heat FEM results in Tables 3 and 9–10 (Q1). These tables assess different aspects of GA-NAR. Table 3 evaluates the six-activation end-to-end framework with NRGD intensity/allocation supervision (λρ=λπ=0.5\lambda_\rho=\lambda_\pi=0.5). Appendix H uses a four-arm attribution protocol with those auxiliary weights set to zero and a physics-weighted checkpoint-selection criterion. These distinct protocols explain the Full GA-NAR values.

All values below are from the submitted paper, in the respective table units:

Source / activationFull MSEFull physical residual
Table 3 / ReLU0.0234 ± 0.0007 (×10−4\times10^{-4})0.0402 ± 0.0011 (×10−3\times10^{-3})
Table 3 / Softplus0.0148 ± 0.0005 (×10−4\times10^{-4})0.1169 ± 0.0033 (×10−3\times10^{-3})
Tables 9–10 / ReLU1.283 ± 0.104 (×10−6\times10^{-6})3.321 ± 0.123 (×10−5\times10^{-5})
Tables 9–10 / Softplus0.461 ± 0.045 (×10−6\times10^{-6})2.335 ± 0.086 (×10−5\times10^{-5})

In Table 3, Heat/Softplus reduces standardized MSE by 66.8%, while the residual rises from 0.0720 to 0.1169 (×10−3\times10^{-3}). This reflects a Heat-specific prediction–residual trade-off: the governing equation combines temperature-dependent conduction, (1+T2)∇T(1+T^2)\nabla T, with a cubic reaction term, 0.1T30.1T^3. Its residual depends on spatial gradients and nonlinear terms, while MSE averages state errors. Lower state MSE can therefore coexist with a larger equation residual. Appendix H independently shows that, within its controlled comparison, Full GA-NAR improves both metrics and exceeds the capacity and generic-geometry controls.

2. Robustness to resampled geometry centers (Q2). We independently reconstructed three training-only geometry banks per system, fixing supervised data splits, model settings, validation, and construction budgets. The table reports three-bank mean ± SD of paired improvements over seed-matched baselines; projector overlap is compared with a rank/dimension-matched random reference.

SystemCentersOverlap / randomMSE gain (%)Residual gain (%)
IEEE-118 / ReLU1,0240.331 / 0.1011.96 ± 0.790.10 ± 0.06
Kuramoto-64 / ReLU5120.347 / 0.0485.63 ± 0.2321.15 ± 1.24
Heat FEM / Softplus5120.764 / 0.3330.92 ± 0.300.30 ± 0.07
Shallow-water960.141 / 0.14125.13 ± 0.3415.42 ± 0.39
Resistor network960.683 / 0.58332.87 ± 0.8319.10 ± 0.07
Allen–Cahn960.130 / 0.12035.38 ± 0.6222.74 ± 0.47
Reaction transport960.802 / 0.41744.96 ± 0.9225.78 ± 0.40
Hydraulic network960.517 / 0.50016.08 ± 0.656.74 ± 0.39

All 24 bank-level comparisons improve both metrics; across individual runs, 72/72 improve MSE and 65/72 improve residual. The five additional systems yield 16.08–44.96% MSE gains. These results connect the descriptor-stability analyses in Appendix B to stable end-to-end predictive improvement: independently reconstructed NRGD geometry consistently supports beneficial nonlinear allocation under fixed within-system budgets.

3. Geometry construction versus additional labels (Q3). We compared two uses of similar measured physical-generation time. G constructs NRGD geometry and trains with the original labeled set; L instead generates additional labeled solutions and trains a near-capacity-matched correction from the same seed-matched baseline. Both use the same validation procedure; stage time includes generation and incremental training.

SystemGeneration G/L (s)Samples G/LStage G/L (s)G-over-L MSE gainResidual gain
IEEE-118120.54/113.1220k/40k141.81/130.171.06 ± 0.82%3.31 ± 2.24%
Kuramoto-64180.63/180.2610k/3.66m222.15/211.1835.30 ± 10.27%0.19 ± 1.12%
Heat FEM208.85/211.2910k/32k255.37/268.1852.22 ± 1.08%28.42 ± 1.28%

At comparable generation time, G reduces mean MSE on all three systems with fewer labels. On Heat FEM, it also reduces the residual by 28.42% and shortens the full stage; on IEEE-118 and Kuramoto-64, the relative benefits vary by metric. This provides direct evidence that geometry construction can be a valuable use of physical-computation budget compared with acquiring more labels.

We sincerely thank you again for the time and effort devoted to evaluating our manuscript and for your thoughtful, valuable comments, which have improved its quality and clarity. We hope these clarifications, analyses, and new results address your concerns. Our anonymous repository linked in the submission will soon provide further explanations, experimental settings, and complete results.

Dear Reviewer,

We would like to briefly follow up on the three questions you raised. In our response above, we explain how differences in auxiliary supervision and checkpoint selection account for the Heat FEM results in Tables 3 and 9. We also report end-to-end experiments with three independently resampled geometry banks per system, keeping the supervised splits and construction budgets fixed. Finally, we compare geometry construction with generating additional supervised samples under similar physical-computation budgets and find that geometry construction yields lower mean MSE on all three systems tested.

As the author–reviewer discussion period draws to a close, we would be grateful if you could take a moment to read our response. If you have any further feedback or suggestions, especially on whether the clarifications and new experiments address your concerns, we would greatly appreciate hearing from you.

Thank you again for the time and care you have devoted to reviewing our work.

Comments on the paper

3 comments

Dear Program Chairs, Senior Area Chairs, Area Chairs, Reviewers

Thank you for overseeing the review of Submission 61524. We also thank all three reviewers for their careful assessment and constructive suggestions. Their feedback helped us clarify how GA-NAR uses physical response geometry and explain what the experiments establish.

We appreciate the specific strengths highlighted in reviews.

  • Reviewer ABok recognized the distinction between nonlinear response and first-order sensitivity, the mathematical grounding provided by the Hessian–Gram interpretation, and the use of sampling-budget diagnostics, multiple seeds, physical residuals, and controlled comparisons. We also appreciate the reviewer’s recognition of the 11.511.5\\% median MSE reduction on IEEE-118 across activation families.
  • Reviewer GypJ highlighted the physically grounded architecture for learning parameter-to-solution maps with sharply varying nonlinear responses, the detailed methodological description, and the structured evaluation of NRGD, GANR, and the complete GA-NAR framework.
  • Reviewer LrKz recognized the principled coupling of finite-scale physical response geometry with layer-wise preactivation modulation, as well as the consistent MSE gains across activation families and the physical-residual improvements in most settings.

The central contribution of GA-NAR is to use how a physical system responds locally to determine how a surrogate allocates nonlinear computation. NRGD extracts recurring dominant response directions, local response intensity, and dominant/complement allocation from finite-scale Jacobian variation. GANR uses these descriptors to modulate preactivations across subspaces and layers while preserving the backbone’s activation function. The independent GANR study achieves the lowest NRMSE on four of five systems under both auxiliary-physics conditions. In the end-to-end evaluation, GA-NAR reduces standardized test MSE in all 18 activation–system comparisons and lowers the governing-equation residual in 15 of 18.

The revised passages and new appendix text are marked in blue in the resubmitted PDF. In the main text, we expanded the related-work discussion, clarified the rationale for the experimental design, and explained the numerical results and their implications in greater detail. These changes improve the presentation while retaining the original numerical results in Tables 1–3 and the paper’s core conclusions.

The appendix provides four main additions.

  • Appendix B.5 visualizes nonlinear-response geometry on two-dimensional parameter sections across seven systems, showing how response intensity and directional allocation vary across operating points.
  • Appendix G.1 illustrates how geometry-conditioned preactivation shifts affect ReLU, GELU, Softplus, and SwiGLU across network layers.
  • Appendix H.1 adds the suggested rank-matched random-projector and uniform-allocation controls to assess the contributions of physical subspace orientation and state-dependent allocation within the final GANR architecture.
  • Appendix J.1 documents the additional training-only geometry-resolution evaluation, including the 8080\\% spectral-mass rank rule, the numerical checks for each system, and the information used at inference.

The individual rebuttals also report the DeepONet, FNO, and PINN comparisons, the geometry-center resampling study, and the comparison with generating additional supervised labels at a similar physical-computation cost.

Thank you for considering our responses and the resubmitted manuscript.

Sincerely,
The Authors

Dear Reviewers,

The authors have posted their rebuttals. As the author-reviewer discussion period is coming to a close, please read their responses and engage in the discussion. Thank you for your time and effort.

Best,

AC

Dear Area Chair and Reviewers,

We would like to sincerely thank you for the time and care you have devoted to our submission. We appreciate the Area Chair’s effort in overseeing the review and discussion, and we are grateful to each reviewer for reading the manuscript closely, considering its contributions, and raising thoughtful questions about the method and its evaluation.

We are especially grateful to the reviewers who continued the conversation with us during the rebuttal. Your follow-up questions prompted us to design more targeted controls, clarify how local physical information guides the model, and explain more precisely what the experimental results support. These exchanges helped us strengthen both the evaluation and the manuscript, and we appreciate the time you took to consider the additional results.

We also appreciate the concerns raised by Reviewer ABok. We posted a detailed response and reported additional experiments in the review thread to address those concerns. We regret that we have not yet had the opportunity to discuss our response and the new results directly with Reviewer ABok. We hope the additional results can serve as a starting point for further discussion, and we would be glad to address any remaining questions.

Thank you again for your time and thoughtful consideration of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.