acceptodds
Under review as a conference paper at ICLR 2027

Beyond Simulation: Retain-and-Repair Neural Operators for Real-World Adaptation

Abstract

Neural operators increasingly benefit from pretraining on numerical simulations, yet adapting them for real-world prediction remains challenging. We introduce the Retain-and-Repair Neural Operator (), a framework for adapting simulation-pretrained operators to real-world data while retaining useful pretrained structure. The pretrained operator is first finetuned on real data and then frozen to provide a source prediction, and a shared repair module learns a sequence of refinements from the same observations. Using orthogonal Fourier projections, a spectral ensemble fits a small ridge regression within each cell of the Fourier domain and combines the refinements by weights fitted on a held-out split of the real data. The cells are defined jointly by radial ranges, angular sectors, and measured channels, allowing refinement depth to vary with frequency magnitude, with orientation, and across channels. Including the source prediction as a candidate makes retention available in every cell, and independently trained repair modules enter the same combination as additional candidates. On all RealPDEBench systems and six backbones, consistently outperforms full finetuning and iterative refinement. The framework treats adaptation depth as a cell-specific choice learned from real data.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
neural operators, sim2real

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_oaVgOpenReview ↗
Rating
3
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposed a framework named R2NO for adapting simulation pre-trained neural operators to real-world data.

The key idea is that different spectral regions might be benefit from the refinement depths, which makes the single refinement depth suboptimal.

The proposed method R2NO first fine-tunes and freezes a pre-trained operator, then trains a shared residual repair module to generate multiple refinement iterates

In this paper, R2NO showed consistent improvements over baselines (full fine-tuning and iterative refinement) on on five RealPDEBench datasets and six neural operator backbones.

Critical strengths

This paper in general is well-motivated. The core method is motivated by the assumption that different spectral regions can favor different refinement depths. The proposed method directly fixes this issue, and the orthogonality of Fourier projections provides some decompositions. The method is model-agnostic.

The experiments cover multiple systems and neural operator architectures. The results are consistent.

Critical weaknesses

The core weakness of the paper is that we lacks of a baseline that does region-wise best-depth selection. Because the core motivation is that different Fourier regions need different refinement depths. The R2NO effectively uses Ridge regression to build more depths. So If we choose a "optimal" depth for each Fourier region, can we get the same or better results? If so, we perhaps do not need to have the same complexity as R2NO.

The paper also lacks of the robustness related experiments. The results in this paper do not have conclusive evidences to confirm whether the improvements remain stable with less observations or across different data splits.

Questions for authors

How does R2NO perform when the number of real observations available is reduced?

Have the authors consider selecting the "best" refinement iterate independently for each Fourier cell.

What is the computational costs of training and running the inference of R2NO with one repair, two repair, or 3 repair modules, etc?

4 comments

Thank you for suggesting the per-cell depth-selection baseline and asking about robustness and cost. Appendix I compares depth selection with the ridge readout on the same candidates, Appendices M and J examine a different data split and fewer fitting windows, and Appendix L reports training and inference costs for one to three repair modules.

Below, IRNO (ℓ⋆\ell^{\star}) denotes IRNO at the depth selected on the validation split and IRNO (ℓ=12\ell=12) denotes IRNO at its final depth. For aggregate comparisons, we report percentage changes based on geometric-mean error ratios across backbone–system pairs relative to the stated baseline; negative values indicate lower error.

 

Q1 (W1 / Q2). A baseline that selects the best refinement depth per Fourier region is missing; if it matches R2NO\mathrm{R}^{2}\mathrm{NO}, the ridge regression is unnecessary complexity.

A. We added this baseline for all six backbones and five systems (Appendix I). In every Fourier cell it uses the single iterate among h0,…,h12h_0,\dots,h_{12} with the lowest error on the fitting split. It shares the cells and the candidates of R2NO\mathrm{R}^{2}\mathrm{NO}, so the only difference is that R2NO\mathrm{R}^{2}\mathrm{NO} combines the candidates with fitted weights instead of choosing one.

Error relative to IRNO (ℓ⋆\ell^{\star}) over all 30 pairs. Best in bold.

MethodRMSERel. L2fRMSE
Best depth per cell (selection)−0.5%−1.1%−0.6%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)−8.5%−2.6%−11.5%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)−13.4%−5.3%−18.1%

With one repair module, per-cell depth selection and the ridge readout of R2NO\mathrm{R}^{2}\mathrm{NO} use the same Fourier cells, the same 13 candidate predictions, and the same 12-step rollout. Relative to IRNO (ℓ⋆\ell^{\star}), depth selection reduces RMSE by 0.5%, while the ridge readout reduces it by 8.5%. The ridge readout adds a small closed-form fitting cost and requires no gradient updates.

 

Q2 (W2 / Q1). Robustness is not shown: fewer real observations, different data splits.

A. We repeated the comparison on one additional random split of the real trajectories into training, validation, and test sets (Table 15, Appendix M). R2NO\mathrm{R}^{2}\mathrm{NO} (M=1) improves on IRNO (ℓ⋆\ell^{\star}) in all ten pairs of FNO and DPOT-S.

Test RMSE on the additional random split, FNO and DPOT-S on all five systems. Best in bold.

SystemBackboneFull finetuningIRNO (ℓ⋆\ell^{\star})R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)
CylinderFNO0.062690.057940.04562
CylinderDPOT-S0.058170.053860.03517
Controlled Cyl.FNO0.009490.008260.00821
Controlled Cyl.DPOT-S0.008590.008290.00823
FSIFNO0.015030.011910.01157
FSIDPOT-S0.011360.009960.00989
FoilFNO0.013260.012620.01189
FoilDPOT-S0.011330.010950.01075
CombustionFNO0.021970.021880.02158
CombustionDPOT-S0.020500.020380.02017

On the original split, we also kept the trained repair modules fixed and varied the number of validation windows used to fit the readout (Appendix J).

R2NO\mathrm{R}^{2}\mathrm{NO} (M=3) relative to IRNO (ℓ⋆\ell^{\star}), FNO and DPOT-S on all five systems. IRNO (ℓ⋆\ell^{\star}) selects its depth on the full validation split in every row.

Validation windows used for the fitRMSEfRMSE
10%−8.6%−10.1%
25%−9.5%−12.0%
50%−9.8%−12.6%
100% (Table 1)−9.9%−12.7%

With 10% of the fitting windows, R2NO\mathrm{R}^{2}\mathrm{NO} reduces RMSE by 8.6% relative to IRNO (ℓ⋆\ell^{\star}), compared with 9.9% when using the full fitting split. Appendix H additionally reports the mean and standard deviation across three independently trained repair modules.

Q3 (Q3). Computational cost of training and inference with one, two, or three repair modules.

A. Appendix L reports the measured costs on one NVIDIA RTX A6000.

Cost and accuracy with one, two, and three repair modules. Inference is measured on the Cylinder grid at batch size 8; the last row is the RMSE relative to IRNO (ℓ⋆\ell^{\star}) over the 8 pairs of FNO and DPOT-S outside Cylinder.

MeasureIRNO (ℓ=12\ell=12)R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)R2NO\mathrm{R}^{2}\mathrm{NO} (M=2)R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)
Repair modules trained1123
Evaluations of Φ\Phi at inference12122436
Inference time per batch (ms)110125235345
RMSE relative to IRNO (ℓ⋆\ell^{\star})+0.6%−1.5%−2.2%−2.7%

Training one repair module takes 0.6–3.2 hours depending on the system. The one-, two-, and three-module configurations require one, two, and three training runs, respectively. On the Cylinder grid at batch size 8, IRNO (ℓ=12\ell=12) takes 110 ms per batch, while R2NO\mathrm{R}^{2}\mathrm{NO} takes 125, 235, and 345 ms, respectively. With one module, R2NO\mathrm{R}^{2}\mathrm{NO} matches the 12 repair evaluations of IRNO (ℓ=12\ell=12) and adds about 15 ms per batch for the spectral readout.

We hope these additions address the reviewer's concerns, and we are glad to provide further analyses during the discussion period.

Thanks a lot for providing additional ablation study.

What is the meaning of “best” in the Best depth per cell (selection)? Is it select some highest score or something else? Is it to select the best depth according to the Fourier region as suggested by me?

For computation cost experiments, did you use the NVIDIA RTX A6000 GPU with 48GB RAM? What is the batch size? What is the config? Did you use bfloat16/float32/int8 or something else? What is the main computational bottleneck if u run the Cudnn profile.

When would the improvement stops? Look like u can get better results when increasing M?

Thank you for the follow-up questions. We answer them in order below.  

Q1. What does "best" mean in "Best depth per cell (selection)"?

Yes, this is the region-wise depth selection you suggested. For each Fourier cell, the baseline picks the single candidate among h0,…,h12h_0,\ldots,h_{12} with the lowest squared error within that cell on the validation split (the region-wise selection of Eq. 5). "Best" therefore means lowest error rather than a separate score. The choice is made once on the validation split and then applied unchanged to the test split.

This baseline shares the Fourier cells, the 13 candidates, and the 12-step rollout with R$^2$NO (M=1M=1). The only difference is that R$^2$NO combines the candidates with fitted weights instead of choosing one. Over all 30 backbone–system pairs (Table 8), selection reduces RMSE by 0.5% relative to IRNO (ℓ⋆\ell^\star), i.e., IRNO at its validation-selected depth, while R$^2$NO (M=1M=1) reduces it by 8.5%.

 

Q2. What configuration and precision were used, and what is the computational bottleneck?

Yes. The cost measurements in Table 14 were run on one NVIDIA RTX A6000 (48 GB) with PyTorch 2.7.0 and CUDA 12.6.

  • Batch size. We timed a batch of 8 test windows.
  • Configuration.
    • Data: the Cylinder grid (20 input and 20 output frames, 64×12864\times128 resolution, 3 channels).
    • Repair module Φ\Phi: the U-Net used in the paper (base width 32, about 8.7M parameters).
    • Pipeline: an L=12L=12 rollout followed by the spectral combination (M=1M=1, 13 columns), in evaluation mode without gradients.
    • Timing: the median over repeated runs after warm-up.
  • Precision. All networks run in single precision (float32) with PyTorch's default settings. We did not use bfloat16, mixed precision, or int8. The only exception is the one-time ridge fit of the cell weights on the validation split, which is computed in float64 and is not part of inference.
  • Bottleneck. The repair-module evaluations dominate: the 12-step rollout takes about 110 ms per batch, and the spectral combination adds about 15 ms (one forward and one inverse FFT per window). This is based on pipeline-level timing; we did not run a separate cuDNN-level profile.

 

Q3. Does increasing MM keep improving performance?

MM is the number of repair modules trained with different random seeds. Within the range we tested (M≤3M \le 3), the error keeps decreasing, but each added module helps less. Over the eight non-Cylinder pairs of FNO and DPOT-S (Table 14), the RMSE reduction relative to IRNO (ℓ⋆\ell^\star) is 1.5%, 2.2%, and 2.7% for M=1,2,3M=1,2,3, while the inference time grows roughly linearly (125, 235, and 345 ms per batch).

We use M=3M=3 as a practical accuracy–cost tradeoff. We will evaluate larger values of MM and add an extended accuracy–cost analysis to the camera-ready version to characterize where the gains saturate.

 

We hope this clarifies the points above.

Reviewer_f6TxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

R2NO (Retain-and-Repair Neural Operator) is a model agnostic framework for adapting neural operators pretrained on simulations, to real measurements.

This is achieved by two phases:

(1) Finetuning the pretrained operator Ψ on real observations, training data.

(2) Freezing the neural operator and applying a repair module Φ L times, to obtain trajectories H of candidate predictions. A spectral ensemble combines the candidates into Fourier cells, including the original prediction h0. The cell weights are fitted on real validation data, such that each cell may choose to transfer simulation learned components or repair them to fit the real measurements.

e.g. R2NO does not rely on a single number of refinement steps, but rather lets each Fourier cell choose how much to leverage the real data. The motivation behind this pipeline is given in Figure 1: with a trained repair loop, different spectral regions (bands) may keep improving with more repair steps, or get worse, but no single "depth" (number of repair steps) fits all regions.

The repair loop Φ training objective includes a spatial loss term (MSE one i.e. pixels), a spectral residual term (matching the magnitude of frequencies, weighted higher as frequencies increase), and a fixed point penalty (penalizing overcorrections by feeding the real target as input).

Critical strengths
  • R2NO is model agnostic: it does not depend on the neural operator architecture and can be applied across various backbones.

  • On RealPDEBench, Table 1 shows R2NO consistently outperforms IRNO and full finetuning across 6 different backbones. This claim also scales when multiple repair modules of different seeds are employed (M=3 variant).

Critical weaknesses
  • Among stronger backbones (i.e. UNet, CNO), gains are smaller on non-Cylinder systems (CC, FSI, Combustion) against IRNO. It is difficult to judge whether this is training variance or actual gains. Adding error bars to prove this isn't measurement noise would have helped.

  • Does the Full Fine-Tuning baseline leverage the validation data? In that case, does R2NO use additional data that the other baseline does not? Arguably, R2NO fits additional weights to the validation data (rather than using it to select checkpoint and depth as IRNO), which means it learns from more labeled data. What would an equal data comparison look like, if i.e. IRNO also fit weights to validation data, or R2NO reserves some of the training data as fitting split?

Questions for authors
  • Is there a reason not to also condition the repair loop on repair depth l? The repair loop resembles the velocity update rule from Flow-Matching (Lipman et al. 2022, Liu et al. 2022), the current state of the art in generative image and video models. One difference is that FM takes the "time" as input (parallel to "depth" in this submission). Arguably, the IRNO framework's repair loop could similarly feed the "current depth" per step, as it is informative.

  • Depth is capped at L=12, without providing an ablation. Is there a depth limit where R2NO provides diminishing returns?

4 comments

Thank you for raising the question of how each method uses the validation data. In Appendix J, we clarify the protocol and compare global and per-cell readouts fitted on the same validation windows. Appendix H reports comparisons across repair-module seeds, and Appendix L examines how performance changes with rollout length.

Below, IRNO (ℓ⋆\ell^{\star}) denotes IRNO at the depth selected on the validation split and IRNO (ℓ=12\ell=12) denotes IRNO at its final depth. For aggregate comparisons, we report percentage changes based on geometric-mean error ratios across backbone–system pairs relative to the stated baseline; negative values indicate lower error.

 

Q1 (W1). Among stronger backbones the gains on non-Cylinder systems are small; without error bars it is difficult to separate real gains from training variance.

A. With the source checkpoint held fixed, we compared both readouts on the same repair trajectory for each of three independently trained seeds. R2NO\mathrm{R}^{2}\mathrm{NO} (M=1) achieved lower RMSE and fRMSE than IRNO (ℓ⋆\ell^{\star}) in all 90 matched comparisons. The table below shows two strong backbones, CNO and DPOT-S; Table 7 (Appendix H) reports all six.

Test RMSE, mean ± std over three repair-module seeds (full finetuning has a single checkpoint). Best in bold.

SystemBackboneFull finetuningIRNO (ℓ=12\ell=12)IRNO (ℓ⋆\ell^{\star})R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)
CylinderCNO0.035240.03277 ± 0.000490.03277 ± 0.000490.02901 ± 0.00111
CylinderDPOT-S0.044040.04233 ± 0.000970.04211 ± 0.000620.03289 ± 0.00073
Controlled Cyl.CNO0.007980.00784 ± 0.000010.00783 ± 0.000010.00782 ± 0.00001
Controlled Cyl.DPOT-S0.008470.00799 ± 0.000010.00798 ± 0.000020.00794 ± 0.00001
FSICNO0.009600.00857 ± 0.000020.00857 ± 0.000020.00854 ± 0.00002
FSIDPOT-S0.009900.00871 ± 0.000030.00871 ± 0.000030.00866 ± 0.00001
FoilCNO0.011450.01093 ± 0.000390.01044 ± 0.000040.01002 ± 0.00006
FoilDPOT-S0.010930.01058 ± 0.000030.01056 ± 0.000020.01034 ± 0.00005
CombustionCNO0.023700.02230 ± 0.000270.02229 ± 0.000260.02211 ± 0.00024
CombustionDPOT-S0.021060.02111 ± 0.000070.02081 ± 0.000000.02062 ± 0.00001

The gains are smaller outside Cylinder, but they persist across the remaining 24 pairs, as shown below.

Error relative to IRNO (ℓ⋆\ell^{\star}) over the 24 pairs outside Cylinder.

MethodRMSERel. L2fRMSE
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)−3.5%−3.9%−5.3%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)−5.6%−5.8%−8.6%

Q2 (W2). Does full finetuning use the validation data? R2NO\mathrm{R}^{2}\mathrm{NO} fits weights to the validation split, so does it learn from more labeled data than the baselines? What would an equal-data comparison look like?

A. In the protocol of Table 1, the validation split is used by full finetuning to select its checkpoint, by IRNO to select its checkpoint and its depth ℓ⋆\ell^{\star}, and by R2NO\mathrm{R}^{2}\mathrm{NO} to select the checkpoint and to fit the cell weights. All three methods therefore see the same training and validation windows and differ in what they take from the validation split. Fitting the cell weights on held-out predictions is a second-stage learning step, as in stacking (Appendix A), so R2NO\mathrm{R}^{2}\mathrm{NO} does take more from that split than a depth selection does. Appendix J examines whether this accounts for the gain.

IRNO with weights fitted on the validation split. This comparison, which the reviewer suggests, is the global combination of Table 2: the iterates of the repair loop are combined with weights fitted on the validation split by the same ridge regression, but with one set of weights for the whole spectrum instead of one per Fourier cell. The last two columns below therefore fit their weights on the same validation windows and differ only in the Fourier cells.

Test RMSE when the weights are fitted on the validation split without and with Fourier cells, using the same three repair modules. FNO and DPOT-S on all five systems. Best in bold.

SystemBackboneIRNO (ℓ⋆\ell^{\star})IRNO with fitted weights (no cells)R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)
CylinderFNO0.049130.044630.03432
CylinderDPOT-S0.041510.038290.02617
Controlled Cyl.FNO0.008180.008080.00804
Controlled Cyl.DPOT-S0.007990.007930.00789
FSIFNO0.010370.010100.00985
FSIDPOT-S0.008690.008640.00860
FoilFNO0.011300.011200.01061
FoilDPOT-S0.010570.010480.01016
CombustionFNO0.022390.022350.02201
CombustionDPOT-S0.020820.020710.02058

Global fitting improves on IRNO, and per-cell fitting further reduces RMSE in every setting shown. On Cylinder with FNO, for example, the RMSE is 0.0491 for IRNO (ℓ⋆\ell^{\star}), 0.0446 with global weights, and 0.0343 with per-cell weights. Table 2 reports the same comparison for U-Net and for the fRMSE. The difference is therefore associated with fitting per Fourier cell rather than with fitting on the validation split as such.

Fitting the validation split without the repair loop. Conversely, fitting the validation split per Fourier cell without any repair iterate, so that each cell only rescales the finetuned prediction (J=1, Appendix I), is 24.7% worse than IRNO (ℓ⋆\ell^{\star}). Access to the validation labels alone does not produce the gain.

Using fewer validation windows. We also reduced the validation windows used for the fit to 50%, 25%, and 10%, while IRNO (ℓ⋆\ell^{\star}) keeps the full split for its depth selection.

R2NO\mathrm{R}^{2}\mathrm{NO} (M=3) relative to IRNO (ℓ⋆\ell^{\star}), FNO and DPOT-S on all five systems.

Validation windows used for the fitRMSEfRMSE
10%−8.6%−10.1%
25%−9.5%−12.0%
50%−9.8%−12.6%
100% (Table 1)−9.9%−12.7%

With 10% of the fitting windows, R2NO\mathrm{R}^{2}\mathrm{NO} reduces RMSE by 8.6% relative to IRNO (ℓ⋆\ell^{\star}), compared with 9.9% with the full split.

 

Q3 (Q1). Why not condition the repair loop on the depth ℓ\ell, as flow matching conditions the velocity on time?

A. We keep the repair update independent of depth to match IRNO and isolate the effect of the readout. Appendix B.1 analyzes repeated application of a single update map. A depth-conditioned repair module remains compatible with our readout, which takes a sequence of candidate predictions as input, and would require extending that analysis.

Q4 (Q2). Depth is capped at L=12 without ablation; is there a depth beyond which R2NO\mathrm{R}^{2}\mathrm{NO} gives diminishing returns?

A. We shortened the loop of the trained modules to L=4 and extended it to L=24, refitting R2NO\mathrm{R}^{2}\mathrm{NO} on the depths up to L in each case (Appendix L).

Error relative to IRNO (ℓ⋆\ell^{\star}) over the 8 pairs of FNO and DPOT-S outside Cylinder. Best in bold.

MethodLRMSERel. L2fRMSE
IRNO (final iterate)12+0.6%+2.6%+0.9%
IRNO (final iterate)24+4.3%+11.5%+7.0%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)4−1.3%−1.5%−1.5%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)12−1.5%−1.7%−2.0%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)24−1.5%−1.8%−2.2%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)4−2.4%−2.7%−3.3%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)12−2.7%−2.9%−3.9%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)24−2.8%−3.0%−4.3%

Across the settings in Table 13, extending the rollout from L=12 to L=24 changes the aggregate RMSE improvement by at most 0.1 percentage points at the reported precision after refitting the readout. This suggests diminishing returns over the tested range. With M=3 and L=4, R2NO\mathrm{R}^{2}\mathrm{NO} uses 12 repair evaluations, matching IRNO (ℓ=12\ell=12), and achieves lower aggregate RMSE than IRNO (ℓ⋆\ell^{\star}) in the settings shown.

We hope these additions resolve the reviewer's concerns, and we would be happy to run further analyses during the discussion period.

Thank you for adding the error bars and clarifying on the usage of the validation set. The ablation on depth also proves your point of diminishing returns as depth increases. My concerns are addressed and I am inclined to keep my positive score.

Reviewer_jjrROpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The work proposes Retain-and-Repair Neural Operator (R2NO), a framework for adapting simulation-pretrained neural operators to real data. R2NO builds upon IRNO, a refinement loop applied on top of the frozen finetuned operator, and the authors notice that the optimal number of refinement steps depends on frequency. By using orthogonal Fourier projections, R2NO can treat different frequency cells separately. For each cell it fits (on the validation split) a linear combination of the source predictions and all refinement steps, retaining the source where it is already good and refining where further processing is needed. The method shows improvements over full-finetuning and IRNO on all five RealPDEBench systems and six backbones, with the largest gains on Cylinder.

Critical strengths
  • The observation that different Fourier regions favor different refinement depths is interesting, and the use of orthogonal Fourier projections to decompose the error per cell is clean and well motivated
  • The paper is easy to follow, and presents the idea clearly
Critical weaknesses
  1. The majority of the gains come from the Cylinder task, while others seem to be quite close to baselines, especially for the strongest backbones (U-Net, CNO, DPOT-S) on Controlled Cylinder and FSI. The summed improvement reported in the text (14.5% RMSE, 20.6% fRMSE) add raw errors across systems and are therefore dominated by Cylinder. Without standard deviations from seed repetitions it is not clear how significant are the differences for these tasks.
  2. The central claim of "refinement should vary per frequency component" is not isolated from the effect of per-cell weighting. R2NO has two ingredients, a set of weights per Fourier cell and a diverse set of refinement depths per cell. The paper attributes its gains to the second one but the ablations only vary each ingredient separately (Table 2 single cell, all depths; Fig 3 all cells, varying depths). Specifically Fig 3 suggests that per-cell weighting does most of the work: using all depths instead of one lowers the error by only 5.2% (RMSE) and 9.6% (fRMSE), considerably less that the single-cell vs full method (Table 2). The experiment with J=1, i.e. the base column only (a per-cell recalibration of the source prediction without refinement iterations), is not run, so it is not clear to whom the gain should be attributed to: it might come from fitting a per-frequency linear correction on the validation split, rather than from depth diversity or from the repair loop
Questions for authors
  1. Could the authors report results for J=1 (per-cell recalibration of the source prediction without refinement iterates) and J=2 (source and final iterate), for all backbones as in Table 2?
  2. Could the authors show the per-window or per-operating-condition error distribution on Cylinder, before and after R2NO, and indicate which Fourier cells account for the improvement?
  3. Can the authors report mean and standard deviations over the three trained repair modules, for IRNO and R2NO (M=1)?
3 comments

Thank you for the careful review and for pointing out how strongly Cylinder influences the aggregate results. Appendix H reports the results with Cylinder excluded and compares the methods across three matched repair-module seeds. Appendix I presents the requested ablations, and Appendix K breaks down the Cylinder results by trajectory and frequency band.

Below, IRNO (ℓ⋆\ell^{\star}) denotes IRNO at the depth selected on the validation split and IRNO (ℓ=12\ell=12) denotes IRNO at its final depth. For aggregate comparisons, we report percentage changes based on geometric-mean error ratios across backbone–system pairs relative to the stated baseline; negative values indicate lower error.

 

Q1 (W1). Most of the gain comes from Cylinder; the summed percentages are dominated by it; without seed standard deviations it is unclear whether the small differences on Controlled Cylinder and FSI are significant.

A. Cylinder shows the largest gains, and summing raw errors gives it disproportionate weight. We now give each backbone–system pair equal weight and also report the results with Cylinder excluded. As shown below, the gains persist across the remaining systems.

Error relative to IRNO (ℓ⋆\ell^{\star}) over the 24 pairs outside Cylinder.

MethodRMSERel. L2fRMSE
R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)−3.5%−3.9%−5.3%
R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)−5.6%−5.8%−8.6%

To assess seed variance, we kept the source checkpoint fixed and evaluated IRNO (ℓ=12\ell=12), IRNO (ℓ⋆\ell^{\star}), and R2NO\mathrm{R}^{2}\mathrm{NO} (M=1) with each of the three independently trained repair modules of the M=3 setting. R2NO\mathrm{R}^{2}\mathrm{NO} (M=1) achieved lower RMSE and fRMSE than IRNO (ℓ⋆\ell^{\star}) from the same module in all 90 matched comparisons. The table below shows two strong backbones, CNO and DPOT-S; Table 7 (Appendix H) reports all six.

Test RMSE, mean ± std over three repair-module seeds (full finetuning has a single checkpoint). Best in bold.

SystemBackboneFull finetuningIRNO (ℓ=12\ell=12)IRNO (ℓ⋆\ell^{\star})R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)
CylinderCNO0.035240.03277 ± 0.000490.03277 ± 0.000490.02901 ± 0.00111
CylinderDPOT-S0.044040.04233 ± 0.000970.04211 ± 0.000620.03289 ± 0.00073
Controlled Cyl.CNO0.007980.00784 ± 0.000010.00783 ± 0.000010.00782 ± 0.00001
Controlled Cyl.DPOT-S0.008470.00799 ± 0.000010.00798 ± 0.000020.00794 ± 0.00001
FSICNO0.009600.00857 ± 0.000020.00857 ± 0.000020.00854 ± 0.00002
FSIDPOT-S0.009900.00871 ± 0.000030.00871 ± 0.000030.00866 ± 0.00001
FoilCNO0.011450.01093 ± 0.000390.01044 ± 0.000040.01002 ± 0.00006
FoilDPOT-S0.010930.01058 ± 0.000030.01056 ± 0.000020.01034 ± 0.00005
CombustionCNO0.023700.02230 ± 0.000270.02229 ± 0.000260.02211 ± 0.00024
CombustionDPOT-S0.021060.02111 ± 0.000070.02081 ± 0.000000.02062 ± 0.00001

The gains are small for several strong backbones on Controlled Cylinder and FSI, but their direction is consistent across the three repair-module seeds.

Q2 (W2 / Q1). The two ingredients, per-cell weights and depth diversity, are not isolated; Figure 3 suggests per-cell weighting does most of the work; J=1 and J=2 were not run.

A. We agree that the submission did not isolate the two ingredients. We ran both for all six backbones and five systems, with the partition and the ridge selection of Table 1 (Appendix I). J=1 keeps only the base column: each Fourier cell rescales the finetuned prediction by a constant fitted on the validation split, and no repair iterate is used. J=2 adds the final iterate, so each cell combines h_0 and h_12.

J=13 and J=37 are R2NO\mathrm{R}^{2}\mathrm{NO} with M=1 and M=3. The table below reports the RMSE by backbone; Table 8 gives the aggregate for all three metrics, and Table 9 lists every backbone–system pair.

RMSE relative to IRNO (ℓ⋆\ell^{\star}) by backbone, over the five systems. Best in bold.

BackboneFull finetuningIRNO (ℓ=12\ell=12)J=1J=2J=13 (R2NO\mathrm{R}^{2}\mathrm{NO}, M=1)J=37 (R2NO\mathrm{R}^{2}\mathrm{NO}, M=3)
U-Net+1.7%+1.7%+0.9%−1.9%−10.5%−14.7%
DeepONet+87.8%+0.0%+84.9%−6.9%−12.1%−18.8%
FNO+10.7%+0.7%+9.1%−3.3%−6.0%−9.7%
CNO+8.0%+0.5%+5.8%−1.6%−3.6%−6.6%
Transolver+78.4%+0.4%+69.1%−6.2%−12.8%−19.8%
DPOT-S+6.0%+0.2%+3.0%−4.4%−5.6%−10.2%
All 30 pairs+27.6%+0.6%+24.7%−4.1%−8.5%−13.4%

Fitting the source alone (J=1) gives only a modest improvement over full finetuning and remains well above IRNO (ℓ⋆\ell^{\star}) in the aggregate over all 30 pairs, so a per-cell linear correction fitted on the validation split does not account for the gain. Adding the final iterate (J=2) brings the error below IRNO (ℓ⋆\ell^{\star}), and including the intermediate iterates (J=13) roughly doubles that reduction with the same 12-step rollout. Outside Cylinder, the corresponding reductions are 2.5% and 3.5%.

In Figure 3, J=4 already includes the source and the final iterate of each of the three repair modules. The figure therefore measures what the intermediate iterates add once those candidates are available.

 

Q3 (Q2). Per-window or per-operating-condition error distribution on Cylinder, and which Fourier cells account for the improvement.

A. Appendix K reports the breakdown for DPOT-S, the backbone of Figures 5–9. The Cylinder test split contains 92 trajectories, one per Reynolds number between 1,800 and 12,000, and we aggregate the test windows by trajectory. As shown in Figure 10 of the revised manuscript, the error is not uniform across operating conditions: for 91 trajectories the error of every method lies between 0.003 and 0.03 and rises smoothly with the Reynolds number, while one trajectory (Re=3656\mathrm{Re}=3656) has a much larger error.

Cylinder test RMSE of DPOT-S by trajectory. Best in bold.

Test subsetFull finetuningR2NO\mathrm{R}^{2}\mathrm{NO} (M=1)R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)
All test windows0.04400.03220.0262
Trajectory with the largest error0.3140.2200.171
Remaining 91 trajectories0.01360.01340.0129

For DPOT-S, R2NO\mathrm{R}^{2}\mathrm{NO} with M=3 reduces RMSE from 0.314 to 0.171 on the trajectory with the largest source error. Across the remaining 91 trajectories, it reduces RMSE from 0.0136 to 0.0129 relative to full finetuning.

For the second part of the question, we resolve the error by radial wavenumber and report the change in error energy relative to full finetuning.

Change in error energy relative to full finetuning by band of ∣k∣/∣k∣max⁡\left|k\right|/\left|k\right|_{\max}, DPOT-S, all Cylinder test windows. Best in bold.

BandIRNO (ℓ⋆\ell^{\star})R2NO\mathrm{R}^{2}\mathrm{NO} (M=1)R2NO\mathrm{R}^{2}\mathrm{NO} (M=3)
Low (<0.1)−9.3%−44.2%−69.3%
Mid (0.1–0.3)−14.4%−22.8%−31.1%
High (>0.3)−15.0%−82.7%−83.9%

For DPOT-S, R2NO\mathrm{R}^{2}\mathrm{NO} reduces residual energy more than IRNO (ℓ⋆\ell^{\star}) in all three radial bands. The largest relative reductions occur in the low- and high-wavenumber bands. Appendix K relates these reductions to the trajectory-level breakdown.

 

Q4 (Q3). Mean and standard deviation over the three repair modules for IRNO and R2NO\mathrm{R}^{2}\mathrm{NO} (M=1).

A. Table 7 (Appendix H) reports the requested mean and standard deviation across the three repair-module seeds for IRNO at both depths and for R2NO\mathrm{R}^{2}\mathrm{NO} (M=1), for all thirty backbone–system pairs. The table in our response to Q1 shows CNO and DPOT-S.

We hope the new results address the points raised above, and we are glad to provide any further analysis during the discussion period.

Thank you for the rebuttal, which positively addressed my concerns. Based on this I will raise my score to 3

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.