- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposed a framework named R2NO for adapting simulation pre-trained neural operators to real-world data.
The key idea is that different spectral regions might be benefit from the refinement depths, which makes the single refinement depth suboptimal.
The proposed method R2NO first fine-tunes and freezes a pre-trained operator, then trains a shared residual repair module to generate multiple refinement iterates
In this paper, R2NO showed consistent improvements over baselines (full fine-tuning and iterative refinement) on on five RealPDEBench datasets and six neural operator backbones.
Critical strengths
This paper in general is well-motivated. The core method is motivated by the assumption that different spectral regions can favor different refinement depths. The proposed method directly fixes this issue, and the orthogonality of Fourier projections provides some decompositions. The method is model-agnostic.
The experiments cover multiple systems and neural operator architectures. The results are consistent.
Critical weaknesses
The core weakness of the paper is that we lacks of a baseline that does region-wise best-depth selection. Because the core motivation is that different Fourier regions need different refinement depths. The R2NO effectively uses Ridge regression to build more depths. So If we choose a "optimal" depth for each Fourier region, can we get the same or better results? If so, we perhaps do not need to have the same complexity as R2NO.
The paper also lacks of the robustness related experiments. The results in this paper do not have conclusive evidences to confirm whether the improvements remain stable with less observations or across different data splits.
Questions for authors
How does R2NO perform when the number of real observations available is reduced?
Have the authors consider selecting the "best" refinement iterate independently for each Fourier cell.
What is the computational costs of training and running the inference of R2NO with one repair, two repair, or 3 repair modules, etc?
4 comments
Q3 (Q3). Computational cost of training and inference with one, two, or three repair modules.
A. Appendix L reports the measured costs on one NVIDIA RTX A6000.
Cost and accuracy with one, two, and three repair modules. Inference is measured on the Cylinder grid at batch size 8; the last row is the RMSE relative to IRNO () over the 8 pairs of FNO and DPOT-S outside Cylinder.
| Measure | IRNO () | (M=1) | (M=2) | (M=3) |
|---|---|---|---|---|
| Repair modules trained | 1 | 1 | 2 | 3 |
| Evaluations of at inference | 12 | 12 | 24 | 36 |
| Inference time per batch (ms) | 110 | 125 | 235 | 345 |
| RMSE relative to IRNO () | +0.6% | −1.5% | −2.2% | −2.7% |
Training one repair module takes 0.6–3.2 hours depending on the system. The one-, two-, and three-module configurations require one, two, and three training runs, respectively. On the Cylinder grid at batch size 8, IRNO () takes 110 ms per batch, while takes 125, 235, and 345 ms, respectively. With one module, matches the 12 repair evaluations of IRNO () and adds about 15 ms per batch for the spectral readout.
We hope these additions address the reviewer's concerns, and we are glad to provide further analyses during the discussion period.
Thanks a lot for providing additional ablation study.
What is the meaning of “best” in the Best depth per cell (selection)? Is it select some highest score or something else? Is it to select the best depth according to the Fourier region as suggested by me?
For computation cost experiments, did you use the NVIDIA RTX A6000 GPU with 48GB RAM? What is the batch size? What is the config? Did you use bfloat16/float32/int8 or something else? What is the main computational bottleneck if u run the Cudnn profile.
When would the improvement stops? Look like u can get better results when increasing M?
Thank you for the follow-up questions. We answer them in order below.
Q1. What does "best" mean in "Best depth per cell (selection)"?
Yes, this is the region-wise depth selection you suggested. For each Fourier cell, the baseline picks the single candidate among with the lowest squared error within that cell on the validation split (the region-wise selection of Eq. 5). "Best" therefore means lowest error rather than a separate score. The choice is made once on the validation split and then applied unchanged to the test split.
This baseline shares the Fourier cells, the 13 candidates, and the 12-step rollout with R$^2$NO (). The only difference is that R$^2$NO combines the candidates with fitted weights instead of choosing one. Over all 30 backbone–system pairs (Table 8), selection reduces RMSE by 0.5% relative to IRNO (), i.e., IRNO at its validation-selected depth, while R$^2$NO () reduces it by 8.5%.
Q2. What configuration and precision were used, and what is the computational bottleneck?
Yes. The cost measurements in Table 14 were run on one NVIDIA RTX A6000 (48 GB) with PyTorch 2.7.0 and CUDA 12.6.
- Batch size. We timed a batch of 8 test windows.
- Configuration.
- Data: the Cylinder grid (20 input and 20 output frames, resolution, 3 channels).
- Repair module : the U-Net used in the paper (base width 32, about 8.7M parameters).
- Pipeline: an rollout followed by the spectral combination (, 13 columns), in evaluation mode without gradients.
- Timing: the median over repeated runs after warm-up.
- Precision. All networks run in single precision (float32) with PyTorch's default settings. We did not use bfloat16, mixed precision, or int8. The only exception is the one-time ridge fit of the cell weights on the validation split, which is computed in float64 and is not part of inference.
- Bottleneck. The repair-module evaluations dominate: the 12-step rollout takes about 110 ms per batch, and the spectral combination adds about 15 ms (one forward and one inverse FFT per window). This is based on pipeline-level timing; we did not run a separate cuDNN-level profile.
Q3. Does increasing keep improving performance?
is the number of repair modules trained with different random seeds. Within the range we tested (), the error keeps decreasing, but each added module helps less. Over the eight non-Cylinder pairs of FNO and DPOT-S (Table 14), the RMSE reduction relative to IRNO () is 1.5%, 2.2%, and 2.7% for , while the inference time grows roughly linearly (125, 235, and 345 ms per batch).
We use as a practical accuracy–cost tradeoff. We will evaluate larger values of and add an extended accuracy–cost analysis to the camera-ready version to characterize where the gains saturate.
We hope this clarifies the points above.
Thank you for suggesting the per-cell depth-selection baseline and asking about robustness and cost. Appendix I compares depth selection with the ridge readout on the same candidates, Appendices M and J examine a different data split and fewer fitting windows, and Appendix L reports training and inference costs for one to three repair modules.
Below, IRNO (ℓ⋆) denotes IRNO at the depth selected on the validation split and IRNO (ℓ=12) denotes IRNO at its final depth. For aggregate comparisons, we report percentage changes based on geometric-mean error ratios across backbone–system pairs relative to the stated baseline; negative values indicate lower error.
Q1 (W1 / Q2). A baseline that selects the best refinement depth per Fourier region is missing; if it matches R2NO, the ridge regression is unnecessary complexity.
A. We added this baseline for all six backbones and five systems (Appendix I). In every Fourier cell it uses the single iterate among h0,…,h12 with the lowest error on the fitting split. It shares the cells and the candidates of R2NO, so the only difference is that R2NO combines the candidates with fitted weights instead of choosing one.
Error relative to IRNO (ℓ⋆) over all 30 pairs. Best in bold.
With one repair module, per-cell depth selection and the ridge readout of R2NO use the same Fourier cells, the same 13 candidate predictions, and the same 12-step rollout. Relative to IRNO (ℓ⋆), depth selection reduces RMSE by 0.5%, while the ridge readout reduces it by 8.5%. The ridge readout adds a small closed-form fitting cost and requires no gradient updates.
Q2 (W2 / Q1). Robustness is not shown: fewer real observations, different data splits.
A. We repeated the comparison on one additional random split of the real trajectories into training, validation, and test sets (Table 15, Appendix M). R2NO (M=1) improves on IRNO (ℓ⋆) in all ten pairs of FNO and DPOT-S.
Test RMSE on the additional random split, FNO and DPOT-S on all five systems. Best in bold.
On the original split, we also kept the trained repair modules fixed and varied the number of validation windows used to fit the readout (Appendix J).
R2NO (M=3) relative to IRNO (ℓ⋆), FNO and DPOT-S on all five systems. IRNO (ℓ⋆) selects its depth on the full validation split in every row.
With 10% of the fitting windows, R2NO reduces RMSE by 8.6% relative to IRNO (ℓ⋆), compared with 9.9% when using the full fitting split. Appendix H additionally reports the mean and standard deviation across three independently trained repair modules.