acceptodds
Under review as a conference paper at ICLR 2027

Beyond the Final Iterate: Learning from Solver Trajectories for Inverse Medium Scattering

Abstract

Inverse medium scattering concerns the reconstruction of spatially varying media from limited and noisy wave measurements. We investigate whether the trajectory generated by a classical solver can itself serve as a useful representation for learned inverse reconstruction. We propose a Transformer-based neural operator that learns reconstruction corrections from function-space trajectories generated by Gauss–Newton (GN) iterations with total variation (TV) regularization. An attention mechanism with convolutional query, key, and value projections aggregates information across the trajectory while preserving spatial structure. For Helmholtz inverse medium reconstruction with nominal observation noise, our method achieves a mean relative reconstruction error of , compared with for GN–TV and for a comparably sized Fourier Neural Operator (FNO) baseline that corrects the final GN–TV iterate. Through extensive comparative experiments and ablation studies, we demonstrate the effectiveness of the proposed model. Without retraining, it achieves lower mean medium reconstruction errors than GN–TV across the tested noise levels and spatial resolutions, supporting its robustness to observation noise and discretization changes. These findings highlight solver trajectories as an effective interface between model-based inversion and operator learning.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operators, Inverse Medium Scattering, Partial Differential Equations, Transformer

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Y2N1OpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes a Transformer that corrects a Gauss–Newton–TV reconstruction using intermediate medium estimates and adjoint residual fields. On 144 synthetic test trajectories, its default model reduces relative medium error from 13.96% to 9.26%, with additional comparisons covering neural baselines, input ablations, noise, grid resolution, and runtime.

Critical strengths
  • Substantial reconstruction gains with supporting physical validation. Transformer–Local improves medium error over GN–TV3 and the complete-trajectory U-Net baseline: 9.262% versus 13.964% and 10.268%, respectively. Clean unobserved-direction scattering error also improves from 3.891% to 2.416% over GN–TV3, supporting meaningful recovery.

  • Useful controls and practical cost reporting. Retrained input ablations, complete-input concatenation baselines, and full deployment timing strengthen the evaluation. The default model adds a moderate runtime cost over GN–TV3: 10.178 versus 8.593 seconds.

Critical weaknesses
  • The central benefit of solver history is small and uncertain. Repeating only the final medium–adjoint token nearly matches the complete trajectory: 9.386% versus 9.262% error for Transformer–Local; Transformer–FNO actually favors repetition (Table 2). Appendix G shows that repetition also eliminates attention’s influence, leaving only 38,073 effective parameters versus 152,841. These results suggest that most gains come from final-state correction and adjoint information. With one training seed and no numerical bootstrap intervals reported, the incremental contribution of history—the paper’s central premise—is not established convincingly.

  • Grid-transfer evidence is difficult to interpret reproducibly. Appendix B identifies the grid dependence of Euclidean adjoints, but Appendix H omits explicit cross-grid feature scaling, regularization scaling, and common-grid resampling details. Moreover, at 64^2 Local worsens visible scattering error from 6.495% to 9.101% relative to GN–TV3 . Although acknowledged, this limits the robustness conclusion.

Questions for authors
  • Assessment-critical validation: Does the history advantage persist against a model of comparable capacity using only the final medium and adjoints, across multiple training seeds? Please provide the numerical paired confidence intervals described in Section 5.1.

  • Clarification: How are adjoint inputs, solver regularization, and medium resampling handled across grids? These details would materially affect interpretation of the transfer results.

8 comments
AuthorsOpenReview ↗

Response (Part 1/2)

We sincerely thank you for those insightful comments. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.

  • Core history benefit, effective capacity, and multi-seed validation (W1 & Q1)

    • Controlled capacity-matched comparisons and 20-seed evaluations confirm that the performance advantages of solver trajectories are statistically significant and robust. To address concerns regarding the margin between complete trajectories and repeated final states, effective parameter differences, and single-seed variance, we conducted supplementary evaluations across 20 independent random seeds. In addition to comparing Complete (A1) against Repeated last, we also evaluated a capacity-matched single-state baseline (B1, widening the value projection on the Local scaffold using only q3q_3 to reach 152,170 effective parameters vs. 152,841 for A1).

      EpochsComplete (A1), EfieldE_{\mathrm{field}} (%)Repeated last, EfieldE_{\mathrm{field}} (%)q3q_3 only (B1), EfieldE_{\mathrm{field}} (%)Repeated last − A1 (pp), mean [95% paired tt CI]B1 − A1 (pp), mean [95% paired tt CI]
      1009.31459.440010.3142+0.1256 [0.1064, 0.1447]+0.9997 [0.9633, 1.0362]
      2009.31899.440710.3191+0.1218 [0.1057, 0.1378]+1.0002 [0.9618, 1.0386]

      For each training seed, we compute the mean paired error difference on the fixed test set. Across 20 seeds, the 95% confidence interval is

      \bar{\Delta}\pm t_{19,\,0.975}\frac{s_\Delta}{\sqrt{20}} \end{xalignat*}$$ where $\bar{\Delta}$ and $s_\Delta$ are the sample mean and standard deviation of the seed-wise paired differences. All 95% paired $t$ confidence intervals strictly exclude zero under both 100-epoch and 200-epoch budgets, proving that the advantage of incorporating full solver history is statistically significant and not an artifact of random seed selection or parameter count.
    • Historical trajectory information yields substantial reconstruction improvements when paired with our trajectory-aware attention mechanism. While Transformer-FNO slightly favors repetition (Table 2), both Transformer-Local and Transformer-UNet consistently benefit from complete solver histories. Furthermore, comparing trajectory-fed models against single-frame or direct-concatenation baselines (Table 1) demonstrates that leveraging intermediate iterates substantially lowers reconstruction error when paired with an architecture capable of modeling iteration dynamics (e.g., reducing full-field error from 10.268% in concatenated vanilla U-Net and 12.299% in concatenated vanilla FNO down to 9.262% in Transformer-Local).

AuthorsOpenReview ↗

Response (Part 2/2)

  • Cross-grid transfer, regularization scaling, resampling, and 64264^2 scattering behavior (W2 & Clarification)

    • Rigorously scaling regularization parameters and applying area-overlap resampling ensure mathematical consistency and reproducibility across different grid resolutions. We summarize the exact scaling rules and resampling procedures below and have added them explicitly to Appendix H to ensure full reproducibility. Let NN denote the number of grid cells per side, with 96296^2 serving as the reference grid. The regularization and damping coefficients, together with the adjoint inputs, follow the scaling laws:

      \alpha_N = \alpha_{96}(96/N)^2,\ \mu_N = \mu_{96}(96/N)^2,\ \lambda_N = \lambda_{96}(96/N),\ \widetilde a_N = (N/96)^2a_N \end{xalignat*}$$ where $\alpha$, $\mu$, and $\lambda$ denote the Born regularization, Gauss–Newton damping, and TV coefficients, respectively. After scaling, the cached adjoint fields $\widetilde{a}_N$ are normalized by the fixed training RMS values. | Native grid size $N$ | Born/GN coefficient multiplier | TV coefficient multiplier | Adjoint multiplier before normalization | | :------------------: | :----------------------------: | :-----------------------: | :-------------------------------------: | | $64$ | $2.25$ | $1.5$ | $4/9$ | | $96$ | $1$ | $1$ | $1$ | | $128$ | $0.5625$ | $0.75$ | $16/9$ |
    • Area-overlap averaging projection maps native predictions back to the reference grid for fair evaluation. The neural network operates directly on the native N2N^2 grid with frozen weights. Ground truth media are constructed from the same analytic medium by averaging values at 4×44 \times 4 subcells within each cell. For common-grid medium-error evaluation, predictions on N2N^2 are resampled onto the reference 96296^2 grid using cell-overlap area fractions as weights (area-overlap averaging projection). Scattering data are computed directly on the native grid and compared against high-resolution reference measurements.

    • Discrepancy between medium error and scattering consistency stems from objective divergence and forward-operator sensitivity. At 64264^2, Transformer-Local still improves native-grid medium reconstruction over GN–TV3 (12.213% vs. 13.048% in Appendix Table 7) because the network is supervised to minimize medium-field error rather than data misfit. However, because the forward wave operator exhibits heightened sensitivity to specific spatial error modes—and zero-shot evaluation across grids introduces layerwise discretization mismatch [1] and frequency shifts [2]—improved medium accuracy does not automatically translate into lower scattering misfit on coarser grids. We have expanded Section 5.4 and Appendix H to clarify this distinction.

References:

  • [1] Wenhan Gao, Ruichen Xu, Yuefan Deng, Yi Liu. Discretization-invariance? On the Discretization Mismatch Errors in Neural Operators. In International Conference on Learning Representations (ICLR), 2025.

  • [2] Mansi Sakarvadia et al. The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators. In International Conference on Learning Representations (ICLR), 2026.

AuthorsOpenReview ↗

Follow-up on Our Response and Additional Results

Dear Reviewer,

Thank you again for your time and thoughtful feedback. We would like to follow up on our response and ask whether you have any remaining questions or concerns. We welcome further discussion and would be glad to provide any additional clarification.

If our response and additional results have addressed your concerns, we would be grateful if you could consider a more favorable score in light of this evidence.

Thank you for your consideration.

Reviewer_UBtfOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes a Trajectory Transformer for inverse medium scattering, which aggregates intermediate medium estimates and adjoint residual fields generated by a TV-regularized Gauss–Newton solver. The model learns a bounded correction to the final iterate without differentiating through the physical solver. Experiments on synthetic Helmholtz inverse scattering demonstrate improved reconstruction accuracy over GN–TV and neural operator baselines, with additional ablations on trajectory information, noise robustness, and grid resolution.

Critical strengths
  1. Leveraging intermediate solver states and adjoint residual fields provides a practical way to incorporate physics-based information into learned reconstruction without expensive end-to-end differentiation.

  2. The paper compares multiple neural architectures and classical baselines, investigates trajectory and adjoint information through ablations, and reports reconstruction accuracy, noise robustness, discretization transfer, and computational cost.

Critical weaknesses
  1. The improvement from complete trajectories over repeated final states is relatively small (9.262% vs. 9.386% for Transformer–Local), and some variants perform better with repeated states. Moreover, the repeated-state intervention changes the effective model capacity, making it difficult to isolate the benefits of trajectory information. The improvements may primarily arise from adjoint features or architectural differences rather than the trajectory itself.

  2. The experiments rely on relatively small synthetic datasets with restricted geometric families. Robustness under substantially different medium distributions, acquisition configurations, and inverse solvers remains unclear. The architecture also largely combines established convolutional attention and neural operator components, without sufficiently demonstrating why this particular trajectory aggregation mechanism is preferable to simpler alternatives.

Questions for authors
  1. Can the authors provide more controlled, capacity-matched ablations to isolate the contribution of historical iterates from adjoint information and architectural differences? Are the relatively small improvements over repeated-final-state inputs statistically significant?

  2. How does the method generalize to more complex or out-of-distribution media, different acquisition geometries, and alternative inverse solvers? In particular, what explains the deterioration in scattering-data consistency at the 64×64 resolution?

  3. Since the attention weights are spatially shared for each iteration, what evidence demonstrates that the proposed attention mechanism captures meaningful optimization dynamics beyond simple learned trajectory weighting or concatenation?

6 comments
AuthorsOpenReview ↗

Response (Part 1/3)

We sincerely thank you for those insightful comments. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.

  • Capacity-matching, input interventions, and statistical significance (W1 & Q1)

    • Controlled capacity-matched comparisons and 20-seed evaluations confirm that the performance advantages of solver trajectories are statistically significant and robust. To rigorously isolate the contribution of trajectory history from capacity differences and random seed variance, we conducted supplementary experiments across 20 independent random seeds and added a parameter-matched baseline (B1), which widens the value projection on the Local scaffold using only q3q_3 to reach 152170 effective parameters.

      EpochsComplete (A1), EfieldE_{\mathrm{field}} (%)Repeated last, EfieldE_{\mathrm{field}} (%)q3q_3 only (B1), EfieldE_{\mathrm{field}} (%)Repeated last − A1 (pp), mean [95% paired tt CI]B1 − A1 (pp), mean [95% paired tt CI]
      1009.31459.440010.3142+0.1256 [0.1064, 0.1447]+0.9997 [0.9633, 1.0362]
      2009.31899.440710.3191+0.1218 [0.1057, 0.1378]+1.0002 [0.9618, 1.0386]

      Across 20 seeds, the 95% confidence interval is

      \bar{\Delta}\pm t_{19,\,0.975}\frac{s_\Delta}{\sqrt{20}} \end{xalignat*}$$ where $\bar{\Delta}$ and $s_\Delta$ are the sample mean and standard deviation of the seed-wise paired differences. As shown in the table, all 95% paired $t$ confidence intervals strictly exclude zero, confirming that incorporating historical iterates yields statistically robust gains over both repeated last and parameter-matched $q_3$ only controls.
    • Historical trajectory information yields substantial reconstruction improvements when paired with our trajectory-aware attention mechanism. As shown in Table 1 and Table 2 of our paper, vanilla U-Net and FNO models that directly concatenate the complete medium-adjoint trajectory achieve higher full-field errors (10.268% and 12.299%, respectively) compared to our trajectory Transformer (9.262%), even when supplied with the exact same sequence of adjoint and medium states. This demonstrates that the improvements stem from the synergy between our convolutional attention mechanism and the sequential solver trajectory, rather than from adjoint features or general network capacity alone.

AuthorsOpenReview ↗

Response (Part 2/3)

Generalization, coarse-grid scattering behavior, and comparison with simpler trajectory aggregation (W2 & Q2 & Q3)

  • The frozen Transformer–Local generalizes robustly to unseen geometric families and limited-aperture acquisition layouts. To directly evaluate out-of-distribution robustness, we tested the frozen model (trained solely on the primary 96296^2 GN–TV benchmark) across three controlled single-factor variations: (1) two unseen geometric families (multiple ellipses and multiple rounded polygons), (2) a limited-aperture receiver layout that preserves the total number of measurements per frequency, and (3) trajectories generated by three steps of TV-regularized proximal gradient instead of GN–TV.

    SettingClassical solver EfieldE_{\mathrm{field}} (%)Transformer-Local EfieldE_{\mathrm{field}} (%)Classic - Local (pp) [95% CI]
    Unseen ellipses and rounded polygons12.8859.7363.149 [2.807, 3.490]
    Limited-aperture acquisition14.37610.3454.030 [3.812, 4.238]
    Three-step proximal gradient38.68747.015-8.328 [-9.148, -7.467]

    Here, the classical solver baseline denotes the third iterate q3q_3 of GN–TV in the first two settings and of TV-regularized proximal gradient in the third. Without any retraining, Transformer–Local reduces full-field relative L2L^2 error by 3.149 percentage points on unseen medium shapes and by 4.030 percentage points under the limited-aperture configuration, with all 95% confidence intervals strictly below zero. Conversely, the error increase under three-step proximal gradient reflects the substantial mismatch in convergence speed and trajectory scale between slow first-order proximal steps (q3q_3 error of 38.687%) and the second-order Gauss–Newton dynamics used during training, clarifying that zero-shot transfer requires compatible solver trajectory scales. We will include these out-of-distribution results in the revised appendix.

  • Detailed analysis of scattering-data consistency deterioration at 64264^2:

    • Our neural operator and the classical solver optimize distinct training objectives, so superior medium recovery does not automatically imply lower data misfit. Our neural operator is directly supervised to minimize the spatial medium reconstruction error (EfieldE_{\mathrm{field}}), whereas the classical GN–TV solver minimizes a scattering-data misfit against noisy visible observations. These two objectives emphasize different aspects of reconstruction quality. As reported in Appendix Table 7, Transformer-Local still reduces the native-grid full-field relative L2L^2 error at 64264^2 from 13.048% (GN–TV3) down to 12.213% without retraining.
    • The forward wave operator exhibits anisotropic sensitivity, meaning smaller L2L^2 medium errors can still trigger larger scattering discrepancies. Scattering errors depend on the spatial structure and direction of the remaining medium errors, not merely their overall L2L^2 magnitude. A prediction with a smaller medium-field error may still retain residual error components in regions or frequencies to which the forward scattering operator is exceptionally sensitive, resulting in a larger scattering discrepancy on coarse grids.
    • Zero-shot evaluation across grid resolutions inherently induces discretization mismatch, which is an orthogonal challenge in operator learning. Evaluating zero-shot models across non-training grid resolutions is a recognized theoretical challenge in operator learning, inducing layerwise discretization mismatch errors [1] and frequency shifts [2]. Developing specialized multi-resolution training regimes is an orthogonal problem to our focus on solver trajectory exploitation. We have expanded Section 5.4 and Appendix H in the revised manuscript to thoroughly analyze this coarse-grid behavior.
AuthorsOpenReview ↗

Response (Part 3/3)

  • Comparison with simpler trajectory aggregation mechanisms and rationale for spatially shared attention:

    • Adaptive attention yields statistically lower reconstruction errors than uniform averaging and learned static aggregation. To verify that our spatially shared attention mechanism captures meaningful optimization dynamics rather than trivial weighting, we evaluated two matched aggregation controls across 20 random seeds: (A3) uniform averaging across all trajectory states and (A4) learned static linear combination weights trained directly on the dataset, compared against our adaptive Transformer-Local (A1).

      EpochsA1, EfieldE_{\mathrm{field}} (%)A3, EfieldE_{\mathrm{field}} (%)A4, EfieldE_{\mathrm{field}} (%)A3 − A1 (pp), mean [95% paired tt CI]A4 − A1 (pp), mean [95% paired tt CI]
      1009.31459.51499.4634+0.2004 [0.1733, 0.2275]+0.1489 [0.1243, 0.1736]
      2009.31899.53109.4691+0.2121 [0.1849, 0.2393]+0.1502 [0.1288, 0.1715]

      Adaptive attention reduces mean full-field relative L2L^2 error by 0.2004 percentage points over uniform averaging and 0.1489 percentage points over learned static aggregation, with all 95% confidence intervals strictly above zero.

    • Spatially shared scalar weights preserve wave-field coherence while dynamically weighting optimization progress across solver iterations. Keeping the attention weight wtaw_t^a spatially uniform per iteration token (Equation 6) is deliberate: it preserves the spatial coherence of the wave fields and gradients during aggregation while allowing the network to dynamically modulate which solver steps are most informative for a given sample (e.g., balancing early coarse-structure recovery against late boundary sharpening), leaving spatial feature extraction to the convolutional projections and decoder.

References:

  • [1] Wenhan Gao, Ruichen Xu, Yuefan Deng, Yi Liu. Discretization-invariance? On the Discretization Mismatch Errors in Neural Operators. In International Conference on Learning Representations (ICLR), 2025.

  • [2] Mansi Sakarvadia et al. The False Promise of Zero-Shot Super-Resolution in Machine-Learned Operators. In International Conference on Learning Representations (ICLR), 2026.

Reviewer_97qcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies learned post-processing for 2D Helmholtz inverse medium scattering. The method in Sections 3–4 combines a Born estimate with three Gauss–Newton-TV updates to produce four states, each containing a medium estimate and frequency-specific adjoint residual fields. A convolutional attention network predicts a bounded correction to the final iterate without differentiating through the PDE solver. Table 1 shows that, at 96296^2 resolution with 3% noise, Transformer-Local reduces mean relative medium error from 13.964% to 9.262% and improves scattering predictions on withheld measurements.

Critical strengths
  1. The pipeline is practical: trajectories and adjoint fields are precomputed, so training needs no PDE solves, and the projection keeps the output within the contrast bounds.
  2. Transformer-Local substantially reduces both medium and scattering errors at the training resolution and beats the concatenated baselines, while extra classical iterations barely help and the added runtime is modest.
  3. The paper is well motivated and clearly written.
Critical weaknesses
  1. The benefit of solver history is small and backbone-dependent: for Transformer-Local, the full trajectory beats repeating the last state (which disables attention) by only 0.12 points, and history worsens full-field error for both FNO variants.
  2. The value of attention over a matched simpler aggregation, such as uniform averaging with the same network, is not tested.
  3. All results come from a single training seed with no numerical confidence intervals, so differences this small cannot be assessed.
  4. At 64², all Transformer variants have worse scattering errors than GN-TV3, which the paper acknowledges only briefly and does not explain.
  5. The handling of adjoint scaling, regularization weights, and interpolation across grids is unclear, making the grid-transfer results hard to interpret and reproduce.
Questions for authors
  1. Can you report confidence intervals and seed variability for Complete versus Repeated last, and would longer training change the comparison for runs that peak at the final epoch?
  2. How does attention compare with uniform averaging using the same network, and how does a matched model perform with only q3 and no adjoint fields?
  3. How often does the line search stop early and leave repeated states in the trajectory, and does history help more when all four states are distinct?
  4. How are adjoint inputs, regularization weights, and interpolation handled across grids, and what causes the worse scattering errors at 64²?
7 comments
AuthorsOpenReview ↗

Response (Part 1/4)

We sincerely thank you for those insightful comments. Many of your comments will help us produce a more readable and self-contained version of the paper. Below, we address each of your specific concerns in turn.

  • Magnitude and backbone dependence of solver-history gains, extending training (W1 & W3 & Q1)

    We conducted additional experiments across a wider range of network configurations (using 20 random seeds for each setup). We will include the relevant ablation study configurations in the response below, and the full results of the ablation studies will be provided in the paper's appendix.

    EpochsComplete EfieldE_{\mathrm{field}} (%)Repeated last EfieldE_{\mathrm{field}} (%)Repeated last − Complete (pp), mean [95% paired tt CI]
    1009.31459.4400+0.1256 [0.1064, 0.1447]
    2009.31899.4407+0.1218 [0.1057, 0.1378]

    For each training seed, we compute the mean paired error difference on the fixed test set. Across 20 seeds, the 95% confidence interval is

\bar{\Delta}\pm t_{19,\,0.975}\frac{s_\Delta}{\sqrt{20}} \end{xalignat*}$$ where $\bar{\Delta}$ and $s_\Delta$ are the sample mean and standard deviation of the seed-wise paired differences. + **Inverse medium scattering is a challenging nonlinear, ill-posed problem** [1]. Although the incremental gain associated with solver trajectories is modest, it remains encouraging and complements the contribution of the proposed network architecture. + **Incorporating solver trajectories yields consistent, statistically significant improvements in inversion accuracy.** The 95% bootstrap confidence intervals for the paired error reduction exclude zero under both training budgets, with lower bounds of 0.0651 and 0.0602 percentage points for the 100 and 200 epochs, respectively, which demonstrates that trajectory history provides stable, significant gains in medium reconstruction precision across the evaluation suite. + **Unlocking the full value of solver trajectories requires an iteration aware aggregation architecture rather than simple plug-and-play channel concatenation.** As demonstrated by these comparative results, solver iterates do not function as generic spatial features for standard operators. Instead, effectively exploiting their sequential dynamics necessitates a dedicated mechanism capable of modeling iterative evolution, as instantiated in our trajectory transformer. + **The performance degradation observed in FNO indicates that exploiting solver history is inherently architecture-dependent.** Solver iterates contain distinct sequential dependencies that direct channel concatenation in standard FNO fails to exploit, whereas our convolutional attention mechanism is specifically designed to aggregate function-valued states across solver steps. + **Extending training to 200 epochs preserved the comparison for the one run that peaked at epoch 100.** As shown above and in the appendix, the Repeated last run selected at epoch 100 achieved almost the same full-field test error when training was extended to 200 epochs.
AuthorsOpenReview ↗

Response (Part 2/4)

  • Attention versus matched other aggregation (W2 & Q2).

    • Attention yielded lower mean full-field error than simple aggregation methods. In the following, (A1) stands by our Transformer-Local network. We replaced the attention module with (A3) uniform averaging across all trajectory states and (A4) learned static linear combination weights trained directly on the dataset.

      EpochsA1, EfieldE_{\mathrm{field}} (%)A3, EfieldE_{\mathrm{field}} (%)A4, EfieldE_{\mathrm{field}} (%)A3 − A1 (pp), mean [95% paired tt CI]A4 − A1 (pp), mean [95% paired tt CI]
      1009.31459.51499.4634+0.2004 [0.1733, 0.2275]+0.1489 [0.1243, 0.1736]
      2009.31899.53109.4691+0.2121 [0.1849, 0.2393]+0.1502 [0.1288, 0.1715]

      Evaluated over 20 random seeds with confidence intervals, our Transformer-Local reduced mean full-field relative L2L^2 error by 0.2004 percentage points relative to uniform averaging and 0.1489 percentage points relative to learned static aggregation, with all 95% paired tt confidence intervals above zero. These results support the advantage of adaptive attention over uniform and learned static aggregation in exploiting solver trajectories information.

    • A parameter matched q3q_3 only model performs worse than our model trained with trajectories and adjoint inputs. In Table 1, the q3q_3-only vanilla FNO and vanilla U-Net baselines achieved full-field relative L2L^2 errors of 12.700% and 11.150%, respectively. To match Transformer–Local (Complete, A1) more closely, we trained B1 on the Local scaffold using only the final medium q3q_3 and widened the value projection to reach 152170 effective parameters, compared with 152841 for A1.

      EpochsComplete (A1), EfieldE_{\mathrm{field}} (%)q3q_3 only (B1), EfieldE_{\mathrm{field}} (%)B1 − A1 (pp), mean [95% paired tt CI]
      1009.314510.3142+0.9997 [0.9633, 1.0362]
      2009.318910.3191+1.0002 [0.9618, 1.0386]

      B1 had a mean full-field error of 10.3142% versus 9.3145% for A1 under the 100 epoch budget, a paired gap of 0.9997 percentage points. The results indicate that using the final state q3q_3 in an end-to-end fashion only yields lower accuracy.

AuthorsOpenReview ↗

Response (Part 3/4)

Scattering error deterioration at 64² resolution (W4 & W5 & Q4).

  • Scattering error deterioration at 64² resolution

    • Our network retains a medium reconstruction advantage at 64264^2, consistent with its medium focused training objective. Our neural operator minimizes supervised medium field reconstruction error, whereas GN–TV minimizes a scattering-data misfit against noisy visible observations. These objectives emphasize different aspects of reconstruction quality. As reported in Appendix Table 7, Transformer–Local reduces the native-grid full-field relative L2L^2 error from 13.048% for GN–TV3 to 12.213% without retraining the model.
    • The forward operator’s sensitivity to different error patterns means that improved medium accuracy does not necessarily imply improved scattering consistency. Scattering errors depend on the spatial structure and direction of the remaining medium errors, not merely their overall magnitude. A prediction with a smaller medium-field error may still contain error components to which the scattering operator is more sensitive, resulting in a larger scattering discrepancy.
    • Performance degradation during zero-shot transfer across grid resolutions is a recognized theoretical challenge in operator learning [2,3]. Recent studies demonstrate that evaluating models on non-training grids induces layerwise discretization mismatch errors [2] and frequency shifts [3], both of which typically require dedicated multi-resolution training to mitigate. Developing specialized architectures or training regimes for cross-resolution invariance is an orthogonal problem and was therefore not elaborated in the original manuscript.
    • We added a dedicated technical discussion to the revised manuscript to thoroughly analyze this coarse-grid behavior. Specifically, we will expand Section 5.4 and Appendix H to contrast direct solver misfit minimization with neural discretization mismatch, contextualize our results with relevant operator learning literature, and appropriately scope our claims regarding resolution robustness.
  • Cross-grid adjoint scaling, regularization, and resampling.

    We summarize the implementation details below and will include them in Appendix H to facilitate interpretation and reproducibility.

    Let NN denote the number of grid cells, with 96296^2 as the reference grid. The regularization and damping coefficients, together with the adjoint inputs, follow

    \alpha_N = \alpha_{96}(96/N)^2,\ \mu_N = \mu_{96}(96/N)^2,\ \lambda_N = \lambda_{96}(96/N),\ \widetilde a_N = (N/96)^2a_N \end{xalignat*}$$ | Native grid size $N$ | Born/GN coefficient multiplier | TV coefficient multiplier | Adjoint multiplier before normalization | | -------------------- | ------------------------------ | ------------------------- | --------------------------------------- | | $64$ | $2.25$ | $1.5$ | $4/9$ | | $96$ | $1$ | $1$ | $1$ | | $128$ | $0.5625$ | $0.75$ | $16/9$ | Here, $\alpha$, $\mu$, and $\lambda$ denote the Born regularization, Gauss–Newton damping, and TV coefficients, respectively. The network operates directly on the native $N^2$ grid. Ground truth is constructed from the same analytic medium by averaging its values at $4^2$ oints within each cell. For medium-error evaluation, predictions are resampled onto the reference $96^2$ grid using cell-overlap area fractions as weights—an area-overlap averaging projection. Scattering data are computed on the native grid and compared against the high-resolution reference data.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.