acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Scaling for PDE Surrogates via Solution-Space Refinement Flows

Abstract

Neural PDE surrogates have emerged as powerful tools for accelerating scientific and engineering simulations, but are typically deployed as one-step predictors and remain largely decoupled from the test-time scaling paradigm that has driven recent advances in generative models and large language models. Inspired by flow-based learning and on-policy training, we introduce Flow Corrector (FOCO), a computationally-efficient general framework that improves frozen PDE surrogates through iterative test-time refinement. FOCO learns a shared corrector from self-generated on-policy states in solution space and stochastic detached rollouts, avoiding backpropagation through long correction trajectories while substantially reducing training computation and activation-memory requirements. Across global weather super-resolution and next-state prediction for 2D and 3D spatiotemporal PDE systems, FOCO consistently improves frozen surrogates and outperforms matched baselines, reducing normalized MSE by \(78.6%\) on 3D magnetohydrodynamic turbulence and by \(84.8%\) over a 20-step rollout on the Wave Layer benchmark. The learned corrector further transfers zero-shot across neural-operator architectures without retraining.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
Neural Operator, Scientific Machine Learning, PDE Surrogates

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_1nxZOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces FOCO, a post-hoc method that trains a small U-Net to iteratively correct the predictions of a frozen surrogate (FNO, TFNO or Transolver), following a relaxed Picard iteration whose number of steps sets the test-time budget. The main contribution is the training procedure: the corrector learns from its own detached intermediate states, which requires only three corrector evaluations per example and avoids backpropagating through the refinement trajectory. The authors derive error bounds under an assumed contraction and provide empirical results on ERA5 super-resolution and four 2D/3D dynamical systems.

Critical strengths
  • Simple and efficient training scheme. The objective has no spectral, fixed-point or PDE-residual terms, and the staged backward pass keeps memory independent of the refinement depth. The efficiency comparison with IRNO is carefully controlled.

  • Broad evaluation. Five benchmarks, including a 3D MHD problem and global weather super-resolution,

  • This paper has a clear value: it presents a method capable of improving PDE surrogate solutions at a low cost, with a simple yet unexplored method.

  • The experimental section provides a clear proof that this method is able to improve predictions for 2D or 3D benchmarks. The rollout analysis is specially interesting.

Critical weaknesses
  • I find the methods section slightly unclear. Specially on the definition of DθD_{\theta} and GθG_{\theta}, that are not explicitly presented. For instance, it is confusing that section 3.1 only mentions DθD_{\theta} and then section 3.2 starts by saying that GθG_{\theta} is frozen, which did not appear during section 3.1.

  • It would be better if the authors proposed a method to automatically decide the hyper-parameter K (number of refinement steps) rather than hard coding it, since there is clearly an optimal value of K according to their experiments. The “test-time scaling” claim is not well supported.

  • Missing ablations and related work on the training objective. Training on detached model-generated states is closely related to the push-forward trick of, which unrolls a model without gradients and trains on the resulting states to reduce distribution shift in autoregressive solvers. This should be discussed.

  • The evidence suggests the corrector may act as a second predictor rather than a refiner of the surrogate. The corrector receives the full conditioning input x in addition to the current estimate, so it can in principle predict y from x and largely ignore u. The corrector has some times more parameters than the base model.

Questions for authors
  • What does the corrector achieve when trained and evaluated from a trivial initialization instead of the surrogate prediction? How sensitive is its output to u, for example measured by perturbing u_0?

  • Can you do ablations on K?

  • How does FOCO compare with the pushforward trick and with a standard two-step unroll with full backpropagation?

  • What are the results of ablating each of the three loss branches?

4 comments
AuthorsOpenReview ↗

Official Comment by Authors Part I

We thank the reviewer for recognizing FOCO’s practical value, efficient training procedure, controlled comparisons, and broad evaluation. We address the methodological clarifications and requested controls below.

W1. I find the methods section slightly unclear.

Equation (1) defines Gθ(x,u)G_\theta(x,u) as the residual corrector and Dθ(x,u)=u+Gθ(x,u)D_\theta(x,u)=u+G_\theta(x,u) as the corrected-solution map. Section 3.1 uses DθD_\theta to express endpoint supervision; its trainable parameters are those of GθG_\theta. The pretrained surrogate BϕB_\phi remains frozen throughout corrector training. After training, GθG_\theta is also frozen for inference. We will explicitly restate these relationships at the beginning of Sec3.1–3.2 and label the maps consistently in Figure 1.

W2. It would be better if the authors proposed a method to automatically decide the hyper-parameter K (number of refinement steps) rather than hard coding it, since there is clearly an optimal value of K according to their experiments. The “test-time scaling” claim is not well supported.

Most practical gains are obtained in one or two evaluations under the budget-independent schedule. The budget-aware experiments demonstrate effective correction along longer trajectories, with limited additional terminal-accuracy gains across budgets. We will therefore use “test-time refinement” throughout the title and manuscript, making the empirical scope explicit.

The budget-independent schedule supports anytime stopping along a shared trajectory, allowing the inference budget KK to be selected according to available computation without retraining. Automatically selecting the accuracy-optimal stopping point for each example remains future work, as noted in the conclusion. We will make the distinction between budget flexibility and adaptive stopping explicit.

W3. Missing ablations and related work on the training objective. Training on detached model-generated states is closely related to the push-forward trick of, which unrolls a model without gradients and trains on the resulting states to reduce distribution shift in autoregressive solvers. This should be discussed.

Thanks for the great suggestion. We will explicitly discuss and cite the pushforward trick (Brandstetter et al., ICLR 2022). Both approaches train on detached model-generated states to reduce deployment-state mismatch.

FOCO applies this principle to post-hoc correction of an unchanged frozen surrogate. The original pushforward formulation trains an autoregressive solver using generated histories and subsequent physical-time targets. FOCO instead holds the conditioning–target pair (x,y)(x,y) fixed and supervises initial, interpolated, and successor correction states toward the same solution. Its contribution is this specific correction-state construction and efficient endpoint-supervised training procedure, rather than detachment alone.

W4. The evidence suggests the corrector may act as a second predictor rather than a refiner of the surrogate. The corrector receives the full conditioning input x in addition to the current estimate, so it can in principle predict y from x and largely ignore u. The corrector has some times more parameters than the base model.

The corrector can be substantially smaller than the frozen surrogate. For the FNO-based experiments, Tables 7 and 10 report the following real-scalar parameter counts:

DatasetFrozen FNO (M parameters)Corrector (M parameters)Corrector / FNO
TR-2D19.0791.7058.94%
Active Matter10.7401.71715.99%
3D MHD33.0852.4227.32%

On 3D MHD, this corrector reduces the frozen FNO’s NMSE by 78.59%. Substantial improvements are therefore achieved with a correction network containing only a small fraction of the backbone’s parameters. The corrector is smaller than the FNO/TFNO backbones, although it can exceed the compact Transolver variants. Our controlled IRNO–FOCO comparisons hold corrector architecture and capacity fixed, so their performance differences cannot be attributed to a larger FOCO corrector.

Appendix F.3 further tests the residual-learning choice: with the same U-Net architecture and (x,u)(x,u) inputs, direct full-solution prediction performs worse than residual correction on TR-2D and can slightly underperform the frozen surrogate. This supports residual parameterization; directly quantifying dependence on uu requires the initialization and sensitivity controls discussed in Q1.

AuthorsOpenReview ↗

Official Comment by Authors Part II

Q1. What does the corrector achieve when trained and evaluated from a trivial initialization instead of the surrogate prediction? How sensitive is its output to uu, for example measured by perturbing u0u_0?

We added a conditioning-only U-Net control and direct perturbation tests on ERA5 and TR-2D. These experiments demonstrate better primary benchmark accuracy than standalone prediction, improved autoregressive performance, and learned state dependence beyond the identity skip. In the tables, Trivial Init. denotes a standalone U-Net receiving xx without an FNO prediction and matching the corrector’s width, depth, and normalization. It uses the standalone-surrogate training recipe, while Standard FOCO retains its correction-training recipe. This compares the two trained systems rather than isolating initialization under an identical objective.

Table 1. One-step prediction: Standard FOCO (K=1K=1) versus Trivial Init.

MethodERA5 RFNE ↓ERA5 KE ACC ↑TR-2D VRMSE ↓
Standard FOCO0.1475720.9281670.223098
Trivial Init.0.1556830.9190940.241256

Standard FOCO reduces ERA5 RFNE by 5.21% and TR-2D VRMSE by 7.53% relative to Trivial Init. For autoregressive prediction, both systems start from the same four ground-truth frames and subsequently feed back their own predictions. FOCO applies one correction after each FNO forecast. FOCO achieves lower VRMSE at every tested horizon, with a 17.60% reduction at seven steps. This supports the practical benefit of the surrogate-plus-corrector system beyond one-step prediction.

Table 2. TR-2D autoregressive VRMSE ↓ by physical forecast horizon.

Method / Horizon1234567
Standard FOCO0.2230980.2961640.3540210.4059480.4522540.4964390.535835
Trivial Init.0.2412560.3328350.4114530.4824960.5448660.6028140.650302

Transfer across different initial estimates. Our existing zero-shot cross-operator experiments provide complementary evidence (§4.6; Figure 7). A corrector trained around one frozen backbone is applied directly to another without retraining. Although FNO, TFNO, and Transolver have different initial accuracies and error structures, transferred correctors improve all 48 off-diagonal dataset–metric comparisons across ERA5, TR-2D, Active Matter, and MHD. Appendix G.8, Tables 33–36, reports the initial errors and corrected performance. These results demonstrate useful correction across naturally different surrogate estimates within the same physical task.

Perturbation robustness. We additionally hold xx fixed and perturb only the frozen surrogate prediction, without retraining:

u0(σ)=Bϕ(x)+σε,ε∼N(0,I).u_0^{(\sigma)}=B_\phi(x)+\sigma\varepsilon,\qquad \varepsilon\sim\mathcal{N}(0,I).

Noise is applied in training-standardized coordinates: 10% means a noise standard deviation equal to 0.1 times each field’s training standard deviation. We evaluate one or two full corrections using the same noise directions across amplitudes and correction counts.

Table 3. ERA5 initialization perturbations.

NoiseNoisy FNO RFNE ↓FOCO K=1K=1 RFNE ↓FOCO K=2K=2 RFNE ↓FOCO K=1K=1 KE ACC ↑FOCO K=2K=2 KE ACC ↑
0%0.25840.1475720.1469110.9281670.928700
5%0.2647630.1575680.1553110.9251960.926673
10%0.2822940.1805880.1755000.9180010.921336
20%0.3387840.2432460.2309000.8930300.902251

Table 4. TR-2D teacher-forced initialization perturbations.

NoiseNoisy FNO VRMSE ↓FOCO K=1K=1 VRMSE ↓FOCO K=2K=2 VRMSE ↓FOCO K=1K=1 NMSE ↓FOCO K=2K=2 NMSE ↓
0%0.2988980.2230980.2191240.0659210.065245
5%0.3677080.2826830.2703970.0672810.066268
10%0.4949600.3855940.3599140.0714330.069471
20%0.7936560.6257120.5692980.0889200.082899

Both correction counts improve the corresponding noisy FNO prediction at every reported noise level. At 10% noise, two corrections reduce ERA5 RFNE by 37.83% and TR-2D VRMSE by 27.28%. Errors increase with corruption, but correction remains useful throughout the tested range.

AuthorsOpenReview ↗

Official Comment by Authors Part III

We measure the change in the corrected endpoint relative to the injected perturbation:

SK(σ)=E[∥DθK(x,u0(σ))−DθK(x,u0)∥2∥u0(σ)−u0∥2].S_K(\sigma)=\mathbb{E}\left[\frac{\lVert D_\theta^K(x,u_0^{(\sigma)})-D_\theta^K(x,u_0)\rVert_2}{\lVert u_0^{(\sigma)}-u_0\rVert_2}\right].

Here DθKD_\theta^K denotes KK full corrections at fixed conditioning.

Table 5. Endpoint sensitivity.

NoiseERA5 S1S_1ERA5 S2S_2TR-2D S1S_1TR-2D S2S_2
5%0.99610.87320.74270.6647
10%0.96110.86460.74770.6658
20%0.91990.83430.76320.6723

The corrected outputs respond to initialization changes, while a second correction consistently lowers endpoint sensitivity. All reported sensitivity averages remain below one, indicating attenuation of the tested perturbations on average.

We additionally examine the net learned update UK(x,u)=DθK(x,u)−uU_K(x,u)=D_\theta^K(x,u)-u, excluding the identity contribution. At 10% noise, the learned-update change normalized by the injected-noise norm is 0.6116 on ERA5 and 0.5197 on TR-2D. These nonzero responses demonstrate that the learned update itself depends on the current estimate, beyond the explicit skip connection.

Q2. Can you do ablations on K?

Section 4.5 and Figure 6 evaluate K∈1,2,4,8,16,32K \in \\{1, 2, 4, 8, 16, 32\\} across four dynamical benchmarks using the same trained corrector without retraining. Tables 2–3 additionally compare one- and two-step refinement. Appendix G provides detailed trajectories, and Appendix G.9 reports frequency-band errors along a fixed K=16K=16 trajectory. We will make these existing experiments more prominent and clarify that the budget-aware schedule changes the discretization when KK changes.

Thank you for the new experiments. Looking at the trivial init results, it seems most of the improvement over the FNO comes from the U-Net alone (0.156 vs 0.148 on ERA5), so I think this should be shown in the main paper. The pushforward comparison and the loss ablations are still missing.

I will keep my score.

Reviewer_VFvDOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
1
Presentation
1
Soundness
3(+2)
Read review
Paper summary

This paper describes learning a correction to neural operators like the Fourier Neural Operator. The name of the proposed approach is FOCO (flow corrector). The authors claim the central idea is to learn from self-generated refinement steps without differentiating through their construction.

Critical strengths

The authors seem to have run a large number of experiments.

Critical weaknesses

The writing is confusing to read. In almost all of the neural operator literature, we learn a mapping from initial conditions or boundary conditions a(x) to a solution u(x). In this paper, they learn a mapping from u to a. I'm not sure why -- is this meaningful? They never clearly define what the variables are or what data is available. At some point a y is introduced, but never defined; I don't know how it's related to u or a.

In figure 1a, a_0 turns into u_t along an arrow without any kind of block to cause this change; it's very unclear. Going from u_0 to a_Z requires three separate applications of the learned neural operator. These steps are explained in section 3.1, but it's very unclear. Here are the steps, with slightly simplified notation and omitted stopgrad operators, where D is the pre-trained neural operator:

  1. a_0 = D(u_0)
  2. u_t = t a_0 + (1-t) u_0 -- it's not clear why I would want to do this, or why I can assume a and u even have the same domain
  3. a_t = D(u_t)
  4. v_Z = {a_0 with prob ½ and a_t with prob ½
  5. a_Z = D(v_Z) ←- now D is being applied to something in a space, whereas in all other uses it's applied to something in u space.
  6. Finally, the loss is the sum of losses on a_0, a_t, and a_Z.

None of these steps are explained or justified. I don't know where this comes from or what it accomplishes. The loss is with respect to an undefined y.

Next there is a test-time refinement step, but it's not specified. Rather, they define a sequence of values of η for a relaxation schedule. Then they write an ODE that doesn't depend on η and say there's some connection to a neural ODE. That's it. There is no explanation of what is happening here.

Theorem 3.3 measures errors in u space, whereas the nerual operator has inputs in u space and outputs in a space.

The experimental results seem to show some improvement in resolution in some subset of the domain, but in other parts of the domain it looks the same. That is, the final solution will have VERY high resolution in some areas and VERY low resolution in others. It's not clear why.

Questions for authors

None

4 comments
AuthorsOpenReview ↗

Official Comment by Authors Part I

We thank the reviewer for recognizing the breadth of our experiments. We believe, however, that several central criticisms rest on a substantial misreading of the definitions in Sec. 3, particularly the roles of the operators and the spaces of the variables. We address each concern below and will make these distinctions more immediately visible in the notation and Figure 1.

W1. The writing is quite confusing ..... At some point a y is introduced, but never defined; I don't know how it's related to u or a.

The forward prediction problem, supervised target, and variable spaces are explicitly defined in Sec 3. FOCO preserves the standard supervised task of predicting a reference solution yy from PDE input or conditioning xx, including coordinates, initial conditions, or boundary conditions as appropriate. The frozen surrogate supplies an initial prediction, and the corrector improves that prediction while retaining the original conditioning. BϕB_\phi is the frozen pretrained surrogate; Dθ(x,u)=u+Gθ(x,u)D_\theta(x,u)=u+G_\theta(x,u) is the learned solution-space correction map. The auxiliary variables a0,at,aZa_0,a_t,a_Z are corrected solution estimates, not PDE coefficient fields.

SymbolMeaningSpace or mapping
xxPDE input or conditioningX\mathcal X
yyReference target solutionU\mathcal U
BϕB_\phiFrozen pretrained PDE surrogateX→U\mathcal X\to\mathcal U
u0=Bϕ(x)u_0=B_\phi(x)Initial surrogate predictionU\mathcal U
Gθ(x,u)G_\theta(x,u)Trainable residual correctorX×U→U\mathcal X\times\mathcal U\to\mathcal U
Dθ(x,u)=u+Gθ(x,u)D_\theta(x,u)=u+G_\theta(x,u)Learned corrected-solution mapX×U→U\mathcal X\times\mathcal U\to\mathcal U
ut,uk,vZ,a0,at,aZu_t,u_k,v_Z,a_0,a_t,a_ZIntermediate or corrected solution estimatesU\mathcal U

The target y∈Uy\in\mathcal U is explicitly introduced in the opening paragraph of Sec. 3, before Eq. (1), which defines the supervised loss. Algorithm 1 receives paired training examples {(xi,yi)}\{(x_i,y_i)\}, and Appendix D.1 specifies their construction for each benchmark. The reference solution is not required at inference. For example, on TR-2D, four preceding states form a 16-channel conditioning input, and the target is the next four-channel state:

\qquad y,u_0,u_t,u_k,a_0,a_t,v_Z,a_Z \in\mathbb R^{4\times128\times384}.$$ Every interpolation, residual addition, and repeated corrector application therefore operates on compatible solution fields. The notation $a_0,a_t,a_Z$ introduces neither a separate coefficient space nor an inverse mapping from solutions to PDE parameters. **To make their role as corrected solution estimates immediately clear, we will replace $a_0,a_t,a_Z$ with $\bar{u}_0,\bar{u}_t,\bar{u}_Z$ throughout the manuscript, including Figure 1 and Algorithm 1.** > W2. In figure 1a, a_0 turns into u_t along an arrow without any kind of block to cause this change; it's very unclear. Going from u_0 to a_Z requires three separate applications of the learned neural operator. These steps are explained in section 3.1, but it's very unclear. Here are the steps, with slightly simplified notation and omitted stopgrad operators, where D is the pre-trained neural operator: > > 1. a_0 = D(u_0) > 2. u_t = t a_0 + (1-t) u_0 -- it's not clear why I would want to do this, or why I can assume a and u even have the same domain > 3. a_t = D(u_t) > 4. v_Z = {a_0 with prob ½ and a_t with prob ½ > 5. a_Z = D(v_Z) ←- now D is being applied to something in a space, whereas in all other uses it's applied to something in u space. > 6. Finally, the loss is the sum of losses on a_0, a_t, and a_Z. The transition to $u_t$ is an explicitly defined arithmetic interpolation: $$u_t=u_0+t ~\mathrm{sg}(a_0-u_0).$$ This formula appears beneath Figure 1a, in Eq. (3), and in Algorithm 1. Because $a_0=D_\theta(x,u_0)=u_0+G_\theta(x,u_0)$, its **forward value** is equivalently $$u_t=u_0+tG_\theta(x,u_0).$$ Thus, $u_t$ is a relaxed correction step with fraction $t$, exposing the model to partially corrected estimates along its own predicted update direction. Both endpoints belong to $\mathcal U$, so the interpolation is well-defined.
AuthorsOpenReview ↗

Official Comment by Authors Part II

There are indeed three corrector evaluations, with distinct supervision roles:

EvaluationState inputOutputPurpose
Initial correctionu0u_0a0=Dθ(x,u0)a_0=D_\theta(x,u_0)Correct the frozen surrogate's prediction
Intermediate correctionutu_tat=Dθ(x,ut)a_t=D_\theta(x,u_t)Correct a partially updated solution estimate
Successor correctionvZv_Z, sampled equally from sg(a0)\mathrm{sg}(a_0) or sg(at)\mathrm{sg}(a_t)aZ=Dθ(x,vZ)a_Z=D_\theta(x,v_Z)Correct the model's own outputs

The conditioning input xx is present in every corrector call. All state arguments and outputs—including vZv_Z—belong to U\mathcal U. The third evaluation therefore applies the same learned corrector to another solution estimate. These are applications of the trainable corrector DθD_\theta, not the frozen surrogate BϕB_\phi. The stop-gradient operations are essential to the optimization: three forward evaluations do not produce a three-evaluation differentiable path. We will explicitly label the interpolation and stop-gradient operations in Figure 1a.

W3. None of these steps are explained or justified. ... The loss is with respect to an undefined y.

Section 3.1 explains how the objective trains a reusable correction map on its own generated states. Equation (4) gives

LFOCO=12ℓ(a0,y)+12ℓ(at,y)+ℓ(aZ,y),\mathcal{L}_{\mathrm{FOCO}} = \frac{1}{2}\ell(a_0,y) + \frac{1}{2}\ell(a_t,y) + \ell(a_Z,y),

where all three outputs predict the same reference solution yy, defined at the start of §3. The first term teaches correction of the initial surrogate prediction. The second teaches correction from partial updates retaining the current model's characteristic errors. The third supervises correction of the model's own outputs, addressing the state-distribution shift encountered during repeated refinement. Successor supervision also motivates target fixed-point consistency. For a supervised pair, in the zero-loss limit, the outputs equal yy, and supervising their successors requires Dθ(x,y)=yD_\theta(x,y)=y. Proposition 3.1 and Appendix C.1 quantify approximate consistency under the stated Lipschitz assumption. Appendix C.7 analyzes state-distribution mismatch, and Appendix F.2 examines self-generated states versus interpolation toward the reference solution.

Detachment is central to the training procedure. Each generated state is treated as a new training input. We backpropagate each loss through its corresponding corrector evaluation, release that graph, accumulate gradients, and update θ\theta once. Each differentiable path therefore contains only one corrector application. Sampling one of two successors preserves the expected detached successor supervision while avoiding a fourth evaluation (Appendix C.5). Proposition 3.2 and Appendix C.6 establish the staged training complexity.

Table 1 quantifies the practical benefit:

DatasetBatchIRNO peak allocated memory (GiB)FOCO peak allocated memory (GiB)Memory reductionIRNO time/epoch (s)FOCO time/epoch (s)Speedup
TR-2D3220.996.4869.15%323.52161.912.00×
Active Matter3230.937.3176.36%868.89412.662.11×
MHD12.981.0166.15%2024.64943.962.14×

Within each benchmark, these comparisons use the same frozen backbone, corrector architecture, batch size, data pipeline, precision, and activation checkpointing. Measurements are medians over three complete epochs after warm-up; timing includes data loading, frozen-surrogate inference, corrector training, and optimization (Appendix E). These savings are particularly relevant to volumetric 3D PDE dynamics.

W4. Next there is a test-time refinement step, but it's not specified. ... . There is no explanation of what is happening here.

Test-time refinement is explicitly specified in Eq. (2), before the relaxation schedules are introduced: u0=Bϕ(x),uk+1=uk+ηkGθ(x,uk),k=0,…,K−1,u_0=B_\phi(x),\qquad u_{k+1}=u_k+\eta_kG_\theta(x,u_k),\qquad k=0,\ldots,K-1, returning uKu_K. Algorithm 1 gives the complete procedure, and Sec. 3.2 supplies the schedules. For temporal PDEs, kk indexes correction of the same predicted state rather than advancement of physical time. Appendix C.4 derives the continuous-relaxation interpretation: dvdτ=Dθ(x,v)−v=Gθ(x,v).\frac{dv}{d\tau}=D_\theta(x,v)-v=G_\theta(x,v). An explicit Euler step with numerical step size ηk\eta_k gives exactly Eq. (2). The numerical step size belongs to the discretization; it need not appear in the continuous vector field. The budget-aware schedule follows from the time change

τ=−log⁡(1−s),w(s)=v(−log⁡(1−s)),0≤s<1,\tau=-\log(1-s),\qquad w(s)=v(-\log(1-s)),\qquad 0\le s<1, which gives dwds=Gθ(x,w)1−s.\frac{dw}{ds}=\frac{G_\theta(x,w)}{1-s}.

Euler discretization at sk=k/Ks_k=k/K, with step size 1/K1/K, yields uk+1=uk+1/K1−k/KGθ(x,uk)=uk+1K−kGθ(x,uk),u_{k+1}=u_k+\frac{1/K}{1-k/K}G_\theta(x,u_k)=u_k+\frac{1}{K-k}G_\theta(x,u_k), recovering the schedule in Eq. (5). The discrete inference rule and its continuous-relaxation interpretation are therefore both provided.

AuthorsOpenReview ↗

Official Comment by Authors Part III

W5. Theorem 3.3 measures errors in u space, whereas the nerual operator has inputs in u space and outputs in a space.

For fixed conditioning xx, Theorem 3.3 considers the self-map Dx(u)=Dθ(x,u)D_x(u)=D_\theta(x,u). Its explicit assumption is that

Dx:Ω→ΩD_x:\Omega\to\Omega

is ρ\rho-contractive on a closed convex set Ω⊆U\Omega\subseteq\mathcal{U}, with u0,y∈Ωu_0,y\in\Omega. Every iterate and the target belong to the same solution space, so ∥uK−y∥U\lVert u_K-y\rVert_{\mathcal{U}} is the appropriate prediction error. The auxiliary aa-variables are corrected solution estimates in U\mathcal{U}. They do not define a separate coefficient space. Consequently, the stated input–output domain mismatch does not occur. The contraction and invariance assumptions are explicit. Section 3.3 and Appendices C.2–C.3 and C.8 distinguish conditional convergence from loss-controlled fixed-point consistency and state that the training objective does not itself guarantee contraction.

W6. The experimental results seem to show some improvement in resolution in some subset of the domain, but in other parts of the domain it looks the same. That is, the final solution will have VERY high resolution in some areas and VERY low resolution in others. It's not clear why.

The reference, frozen-surrogate prediction, and FOCO prediction use the same output grid in every benchmark. FOCO does not change the numerical grid between regions or refinement iterations.

If “resolution” refers to local recovery of fine-scale structure, improvements can vary spatially because the initial errors and required corrections vary across the domain. Uniform pointwise improvement is not claimed. Appendix G.10 directly visualizes the required correction, predicted correction, and remaining residual.

Our quantitative evaluation covers complete fields, rather than only the selected visual examples:

DatasetOutput gridMetricFrozen FNOFOCOCorrector evaluationsError reduction
ERA5720×1440720\times1440RFNE0.25840.1476142.88%
TR-2D128×384128\times384NMSE0.076230.06524214.41%
Active Matter256×256256\times256NMSE0.0084590.001643280.57%
MHD64×64×6464\times64\times64NMSE0.086450.01851278.59%
Wave Layer128×128128\times128NMSE0.0048490.0003650292.47%

The grids and evaluation procedures are documented in Appendices D.1 and D.3, and these prediction results appear in Tables 2–3. Appendix G.9 additionally evaluates Fourier-magnitude errors across low-, mid-, and high-frequency bands over complete fields. Table 37 reports reductions in every band across all four dynamical benchmarks for the evaluated K=16K=16 trajectory under its stated schedule.

For autoregressive prediction, Table 4 further reports an 84.77% reduction in terminal NMSE over the 20-step Wave Layer rollout. These measurements establish improvements beyond the selected visual examples. To make the formulation more immediately accessible, we will add a concise notation table, explicitly label the learned corrector and deterministic state-construction operations in Figure 1, and strengthen the cross-references to Eq. (2), Algorithm 1, and the supporting analysis.

Reviewer_VFvDOpenReview ↗

Thank you

Thank you for clarifying the notation, and I apologize for the initial misreading. These clarifications aleviate many of my concerns. I do think it would be helpful to make your notation consistent with the majority of the neural operator literature, including letting x denote spatial or channel coordinates instead of something in U space; this would make the paper more accessible. I am editing my rating.

Reviewer_N1QVOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a method that iteratively corrects the output of a frozen neural PDE surrogate.

Critical strengths

The paper focuses on a relatively new direction in the neural PDE surrogates area: test-time scaling. The proposed method is straightforward to understand. The writing is mostly fine.

Critical weaknesses

There are several major methodological and experimentation issues:

  • Methodological 1: "Test-time scaling" is used in the title and throughout the paper, to motivate the novelty, to motivate the method, etc.. Test-time scaling has to be demonstrated by showing that allocating more computational resources, at test-time, increases performance. This paper does not demonstrate test-time scaling. Section 4.5, and in particular, Figure 6, show the opposite: increasing computational resources at test-time, by increasing K from 2 or 4 to 32, leads to similar NMSEs/VRMSEs across all setting.
  • Methodological 2: In addition, the paper repeatedly discusses that the proposed method has "variable-depth inference"--this would only be useful if test-time scaling holds. Section 3.1 presents the training as a two-step strategy (a0 and at from u0 and az from a0 and at), and not coincidentally, performance saturates at K=2.
  • Experimentation 1: Section 4.2 states "On ERA5 16× super-resolution, a single FOCO update already surpasses the reproduced IRNO after six refinement steps, while recovering substantially sharper small-scale structure in all three weather variables (Figure 3)." First, as far as I can tell, Figure 3 has nothing to do with IRNO^E2. Also, Figure 3 is misleading for two reasons: 1) it does not compare FOCO with existing neural predictor-corrector methods or the related methods identified in the Introduction and Related Work sections (e.g., diffusion methods, PDE-Refiner, RecurrSim, etc.); and 2) the FNO surrogate cannot recover high-frequencies, so FOCO ( UNet) can be interpreted as projecting a blurry guess to the solution. Figures 4 and 5 are also misleading for the same reasons.
  • Experimentation 2: As far as I can tell, the papers IRNO reproduction is untuned and the numbers are significantly different from the original work, so Table 2 is unconvincing.
  • Experimentation 3: The main paper does not include in-depth comparisons with the closest alternatives, some of which are identified in the Introduction and Related Work sections. The paper also does not motivate why one might prefer iterative methods in physical space vs latent space.

There are several minor issues including making claims too strong, mismatched figures and captions, unclear notation, etc., but those could be addressed.

Questions for authors

Q1. Training uses three corrector evaluations--why 3? Why not 10? How is this different from neural predictor-corrector methods? Q2. For FOCO, please confirm how many parameters each frozen surrogate has and how many parameters its associate corrector has. In cases where the architectures do not match, e.g., FNO-UNet, why did you choose not to use the same architectures for the frozen surrogate and the corrector?

5 comments
AuthorsOpenReview ↗

Official Comment by Authors Part I

We thank the reviewer for the helpful feedback, our responses below seek to clarify all concerns and potential misunderstandings.

W1. Methodological 1: "Test-time scaling" is used in the title and throughout the paper, to motivate the novelty, to motivate the method, etc.. This paper does not demonstrate test-time scaling. …

Figure 6 evaluates whether the same corrector remains effective along trajectories generated under different integration budgets. The total budget KK determines the step-size schedule, while kk indexes progress within a trajectory. Our budget-aware schedule, ηk=1/(K−k)\eta_k=1/(K-k), follows from Euler discretization of the time-reparameterized relaxation dynamics (Sec. 3.2; Appendix C.4). As in generative ODE sampling, widely used in diffusion and flow matching models, changing KK changes the discretization: a K=32K=32 trajectory is not a K=2K=2 trajectory followed by 30 additional corrections. This endpoint-prediction perspective parallels consistency models (Song et al., 2023), which map different intermediate states along a probability-flow ODE trajectory to a common data endpoint, supporting both one-step and multistep generation. FOCO similarly supervises its correction map toward the same target from different self-generated solution estimates.

An ideal endpoint predictor, Dθ(x,u)=yD_\theta(x,u)=y, makes the distinction explicit. Under the uncapped schedule in Eq. 5,

uk=(1−kK)u0+kKy.u_k=\left(1-\frac{k}{K}\right)u_0+\frac{k}{K}y.

Every update reduces error, yet uK=yu_K=y for every KK. Thus, similar terminal accuracy across budgets is compatible with effective correction throughout each trajectory. Section 4.5 empirically examines this intermediate-state correction capability: Figure 6 shows that error generally continues decreasing well beyond k=2k=2 along longer trajectories, without retraining. This supports the interpretation of FOCO as a reusable solution-space corrector, consistent with the zero-shot cross-operator transfer directly evaluated in Figure 7. To make the scope precise, we will use “test-time refinement” throughout the title and manuscript, emphasizing repeated correction of intermediate estimates and explicitly describing the limited additional terminal-accuracy gains at larger budgets.

W2. Methodological 2: In addition, the paper repeatedly discusses that the proposed method has "variable-depth inference"--this would only be useful if test-time scaling holds. Section 3.1 presents the training as a two-step strategy (a0 and at from u0 and az from a0 and at), and not coincidentally, performance saturates at K=2.

We evaluate two complementary relaxation schedules using the same trained corrector (Section 3.2). The budget-independent schedule, used in the main one- and two-step comparisons, begins with full updates: η0=η1=1\eta_0=\eta_1=1 for K=2K=2. These results demonstrate efficient practical deployment: most improvement can be obtained with few corrector evaluations, avoiding the inference cost of unnecessarily long trajectories. The budget-aware schedule, used in Figure 6, instead sets ηk=1/(K−k)\eta_k=1/(K-k). It examines whether the corrector remains effective on intermediate states encountered during longer refinement trajectories. Error generally continues decreasing beyond k=2k=2, without retraining. Similar terminal accuracy across different budgets therefore does not imply that the correction map is effective only for two applications.

This motivation parallels consistency models (Song et al., 2023) and MeanFlow (Geng et al., 2025): learning endpoint recovery or finite-interval transport enables useful predictions from intermediate states while allowing few-step inference for efficiency. Supporting repeated refinement and favoring inexpensive deployment are complementary capabilities.

FOCO’s shallow training procedure makes this flexibility inexpensive: it uses three locally supervised corrector evaluations, each with differentiable depth one, without backpropagating through the deployment trajectory. We will clarify that variable-depth refinement describes reuse of the learned correction map across budgets and schedules, while the few-step results establish its practical efficiency.

AuthorsOpenReview ↗

Official Comment by Authors Part II

W3. Experimentation 1: Section 4.2 states "On ERA5 16× super-resolution, a single FOCO update already surpasses the reproduced IRNO after six refinement steps, ..., Figures 4 and 5 are also misleading for the same reasons.

The IRNO comparison is reported in Table 2: one-step FOCO achieves RFNE 0.1476 and ACC 0.9282, versus 0.1626 and 0.9205 for our six-step IRNO reproduction. Figure 3 visualizes improvement over the frozen FNO. We will clarify these references. Figures 4 and 5 similarly illustrate correction of the frozen surrogate’s predictions and autoregressive trajectories.

Our frozen-FNO/lightweight-U-Net setup follows IRNO (Liu et al, 2026) , motivated by smooth corrective updates, multiscale representations for spectral correction, and efficiency through shared weights. FOCO and our IRNO reproduction use the same frozen surrogate, U-Net architecture and capacity, data, and evaluation protocol, controlling the architectural factor raised here while comparing correction-training strategies. On TR-2D, the corrector has 1.705M parameters versus 19.079M real scalar parameters in the FNO (Tables 7 and 10). Appendix F.3 (ablation study) further shows that, with the same U-Net architecture and inputs, direct full-solution prediction performs substantially worse than residual correction and can slightly underperform the frozen surrogate. This supports the corrector’s role in refining the existing prediction.

As summarized in Table 5, our comparative conclusions concern post-hoc correction of pretrained, frozen PDE surrogates, for which IRNO is the directly matched baseline among the prior methods considered. RecurrSim trains a simulator with internal recurrence end to end, while PDE-Refiner trains a step-conditioned predictor with denoising supervision. Their original formulations train the surrogate itself rather than augmenting an unchanged frozen model.

W4. Experimentation 2: As far as I can tell, the papers IRNO reproduction is untuned and the numbers are significantly different from the original work, so Table 2 is unconvincing.

Our IRNO baseline is a controlled, capacity-matched reproduction, with its training procedure and metric audits already documented in Appendices D.3, D.5, and G.2. On ERA5, IRNO and FOCO use the same frozen FNO checkpoint, identical 1.699M-parameter corrector architectures, data, normalization, and evaluation protocol. Our IRNO implementation retains recurrent trajectory supervision, progressive spectral loss, and fixed-point regularization, with six-step training. The published results and our reproduction should be interpreted with their checkpoint and evaluation differences explicitly identified. For completeness, the following table includes the results from the published ICML 2026 version of IRNO.

ERA5 16× super-resolution with an FNO backbone; lower RFNE is better.

Method and sourceRFNE conventionStepsBase RFNERefined RFNEReduction
Published IRNO, Table 1SuperBench60.32470.214034.09%
Reproduced IRNO, our Table 17SuperBench60.25840.162637.05%
FOCO, our Table 17SuperBench10.25840.147642.88%

The published row uses the original authors’ checkpoints and 2.1M-parameter corrector; our rows use an independently trained FNO and matched 1.699M-parameter correctors. The published result therefore provides context, while the within-protocol IRNO–FOCO pairs constitute controlled comparisons. Our reproduced IRNO delivers substantial correction, with relative RFNE reductions comparable to the published result.

The larger apparent VRMSE discrepancy on The Well requires particular attention to the evaluation protocol. As explicitly explained in Appendix D.3, The Well’s public implementation computes a variance-normalized error separately for each physical field and then averages those scores. In contrast, IRNO’s appendix defines a joint-channel VRMSE, pooling errors and variances across channels before normalization. These operations are mathematically different and assign different weights to the physical variables; their absolute scores and relative improvements are therefore not interchangeable. Table 9 directly audits this distinction on Active Matter by evaluating the same reproduced IRNO predictions under both definitions:

SourceVRMSE conventionBase, K=0Refined, K=6Reduction
Published IRNO, ICML 2026As reported0.10170.050150.73%
Our reproductionThe Well field-wise0.19900.150824.20%
Our reproductionJoint-channel diagnostic0.075260.0342254.53%
AuthorsOpenReview ↗

Official Comment by Authors Part III

Crucially, the two reproduction rows use identical checkpoints and predictions, without any retraining or change in optimization. Changing only the aggregation convention changes the reported reduction from 24.20% to 54.53%. Under the joint-channel diagnostic, the reproduced IRNO achieves a relative improvement comparable to the published 50.73%. Because the published and reproduced checkpoints differ, we do not claim identical absolute performance; the audit demonstrates why the apparent discrepancy cannot be assessed without matching metric definitions.

Our main comparisons consistently evaluate both FOCO and IRNO using the same benchmark metric implementations. The alternative conventions are reported separately for transparency and protocol auditing. Consequently, disagreement with published absolute values does not, by itself, establish inadequate tuning or invalidate Table 2. The relevant evidence is the matched comparison, supported by the documented training procedure and the explicit evaluation-protocol checks. We will make these existing audits more prominent in the main text.

Experimentation 2: The main paper does not include in-depth comparisons with the closest alternatives, some of which are identified in the Introduction and Related Work sections. The paper also does not motivate why one might prefer iterative methods in physical space vs latent space.

Appendix A already provides an in-depth comparison with IRNO, PDE-Refiner, and RecurrSim, summarized in Table 5. Our scope is post-hoc correction of pretrained, frozen surrogates. PDE-Refiner and RecurrSim train the surrogate itself in their original formulations; IRNO provides the directly matched baseline. Tables 1–3 compare FOCO and IRNO using the same frozen checkpoints, corrector architectures, and evaluation protocols.

Solution-space refinement enables improvement through prediction-only access. Given paired training data and a black-box surrogate’s predictions, FOCO trains an external corrector without accessing the surrogate’s weights, gradients, or hidden representations. The shared physical output also enables transfer across compatible backbones without latent-feature alignment: Figure 7 shows improvement in all 48 off-diagonal dataset–metric comparisons, without retraining. These practical capabilities motivate our choice of solution space.

FOCO additionally makes corrector training substantially cheaper than matched IRNO training. Its three locally supervised evaluations use detached state construction, with one corrector evaluation per gradient path and one activation graph retained at a time. Matched IRNO training differentiates through six recurrent steps plus a fixed-point evaluation. Table 1 and Appendix E report 2.00–2.14× per-epoch speedups and 66.15–76.36% lower peak allocated memory under matched conditions. On 3D MHD, FOCO achieves a 2.14× speedup and reduces memory from 2.98 to 1.01 GiB. These savings arise from detached training; the solution-space interface provides compatibility and portability.

We will foreground these advantages and the existing methodological comparison in the main text.

Q1. Training uses three corrector evaluations--why 3? Why not 10? How is this different from neural predictor-corrector methods?

Three evaluations implement three distinct supervision roles, with only one corrector evaluation per gradient path (Sec 3.1; Algorithm 1): correction from the frozen surrogate’s prediction, an interpolated intermediate state, and a detached self-generated successor. All three outputs target the same solution yy. Sampling one of two successor branches preserves the expected detached successor supervision while avoiding a fourth evaluation (Appendix C.5). Three is the cost of this objective, not a prescribed inference horizon or a claimed universal optimum. Ten evaluations could supervise additional states at additional cost; our results demonstrate effective repeated correction with this inexpensive construction.

FOCO’s distinction within neural predictor–corrector methods is its training-state distribution and gradient structure. IRNO differentiates through recurrent refinement; PDE-Refiner uses step-conditioned denoising supervision. FOCO supervises self-generated partial states and detached successors through a common endpoint objective, with staged backward passes retaining only one corrector activation graph. Compared with matched IRNO training, Table 1 reports 2.00–2.14× per-epoch speedups and 66.15–76.36% lower peak allocated memory.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.