- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper proposes FiNO, an iterative neural PDE solver that maps the residual of the current solution to corrections. The authors state conditions under which this update is locally contractive and design a loss that encourages accurate corrections and contractive behavior, among other properties. Finally, the paper validates their method on three different PDEs.
Critical strengths
- Each term of the loss function served an important purpose, and the loss ablations justified each term.
- The method appears to have strong generalization across varying parameters and forecast horizons
- The presentation of the paper is very clear.
Critical weaknesses
- The experimental baselines do not test FiNO against other learned iterative solvers. Although the authors include IRNO, they do not empirically compare with residual-driven methods such as HINTS or the Deep Conjugate Direction Method (DCDM).
- While Theorem 1 provides a useful result, the authors do not establish whether the trained FiNO models satisfy its assumptions in practice. Figure 3 shows that the residual of the current iterate can increase with further iterations, making it especially important to report how often the learned update is contractive across test cases.
Questions for authors
- What is the intuition behind FiNO's robustness to Long horizons (Figure 5)?
3 comments
Thank you to the authors for the detailed rebuttal and additional experiments. I appreciate the effort to address my concerns.
I find the authors' explanation of why a direct comparison with HINTS/DCDM may not be appropriate convincing, given the differences in the problem settings, and consider this concern addressed. I also appreciate the explanation of FiNO's long-horizon robustness in terms of residual-driven corrections and overlapping temporal windows. Both explanations seem reasonable.
I appreciate the additional empirical analysis of contraction behavior. The results provide reassuring evidence that FiNO exhibits approximately contractive behavior in practice, which largely addresses my original concern. One aspect I would appreciate further clarification on is the observation that removing the contraction-related losses actually increases the percentage of contracting steps, despite worsening the final RMSE. The authors suggest that these losses help maintain stable contraction over longer iteration sequences rather than increasing the contraction percentage itself. Could the authors provide additional evidence supporting this explanation?
Apologies for the late response. Our experiments indicate that stable contraction typically occurs during the initial phase, a stage that can be captured through early stopping. And contraction and solution loss help maintain contraction over a longer sequence of time steps. To investigate this further, we extended the number of iterations for the ablation methods to match that of the full version; the results show a significant reduction in the contraction rate for the ablation methods. When the number of iterations for all methods is increased to 128, the contraction ratio decreases across the board.
Iteration steps aligned with the Full:
| Method | Contraction percentage | non-contracting steps mean | RMSE |
|---|---|---|---|
| Full | 91.74% | 0.057% | 0.172 |
| w/o solution contraction | 85.92% | 0.738% | 0.195 |
| w/o residual contraction | 83.91% | 0.181% | 0.214 |
| w/o both | 75.73% | 0.499% | 0.237 |
Iteration steps = 128:
| Method | Contraction percentage | non-contracting steps mean | RMSE |
|---|---|---|---|
| Full | 31.62% | 0.602% | 0.175 |
| w/o solution contraction | 31.09% | 1.089% | 0.213 |
| w/o residual contraction | 30.52% | 0.728% | 0.189 |
| w/o both | 23.31% | 0.804% | 0.338 |
It can be observed that at 128 iterations, the contraction ratio drops (91.74% to 31.62%). However, since the increase in the number of non-contraction steps is minimal (0.602%), the final RMSE shows only a marginal increase (0.172 to 0.175) compared to the early-stopping approach. This is consistent with the curve presented in Figure 3. When ablating solution or residual contraction loss, the convergence becomes unstable.
Q1 (comparison to HINTS and DCDM). HINTS and DCDM share the general residual-driven principle with FiNO. However, both are designed for solving discrete linear systems, particularly sparse linear equations, whereas FiNO targets nonlinear transient PDEs. Despite the shared design philosophy, the methods differ fundamentally in the classes of equations they address, their state-update mechanisms, and their underlying numerical solution procedures. Therefore, a direct experimental comparison would not be methodologically meaningful.
Applying HINTS or DCDM to the problems considered in this work would require substantial methodological adaptation rather than straightforward deployment, including the linearization of nonlinear PDEs, spatial and temporal discretization, and integration with an appropriate numerical solution framework. Such adaptations would introduce additional design choices and potentially alter the original methods, making a direct comparison difficult to interpret fairly. We have accordingly discussed in the Related Work section the design principles shared between FiNO and HINTS, while also clarifying their differences in target problem settings and methodological formulations.
Q2 (contraction assumption satisfaction). Please refer to Response to common questions Q1.
Q3 (intuition behind long horizon robustness). The robustness of FiNO over long prediction horizons can be attributed to two complementary factors. First, unlike autoregressive neural operators that directly learn state-to-state mappings, FiNO adopts an iterative, residual-driven correction mechanism. By explicitly incorporating PDE-based residuals into each update, the evolution is continuously guided by the governing equations rather than relying solely on the predicted state. This physics-informed closed-loop correction helps mitigate the accumulation of prediction errors over successive iterations.
Second, a sufficiently large overlap between consecutive temporal windows provides additional stability for long-horizon rollout. The overlap helps maintain continuity when transitioning from one window to the next. In our experiments, we found that overly aggressive window advancement could lead to substantial deviations in the first frame of the subsequent window, making the subsequent long-horizon rollout considerably more sensitive and fragile. Thus, an appropriate overlap is important for maintaining stable temporal propagation.