- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes HINDCAST-NOS, a support-only reranking method for Neural Operator Splitting (NOS) search. Conventional NOS selects candidates by one-step fitting error, which mismatches the autoregressive rollout used at deployment and can reverse the candidate ranking even for linear autonomous dynamics. HINDCAST-NOS instead restarts each retained candidate from multiple earlier observed states, replays it freely over the observed suffix, and ranks candidates by the resulting mean closed-loop error. Experiments demonstrate the improvements on multiple PDEs, and the authors establish a finite-horizon transfer bound from observed hindcast error to future error theoretically.
Critical strengths
- The motivation is both concrete and theoretically grounded. The paper identifies a clear objective mismatch between the one-step fitting criterion used to select operator candidates and the autoregressive rollout used at deployment.
- The proposed method is easy to implement. It requires no additional training or gradient computation.
Critical weaknesses
- My principal concern concerns the main experimental results. In Table 2, the metrics for the proposed method are very close to those of the baseline NOS. Although the differences are consistent in direction, their magnitude is small enough that the improvement may not be practically meaningful. As presented, the results make the method appear to be a refinement that is effective in a narrow sense but whose benefit in deployment is unclear.
- The experiments lack several simple test-time baselines. For example, the authors should compare HINDCAST-NOS against a random selection from the retained beam, the average prediction over the beam (a test-time ensemble), and so on. Without these controls, it is unclear whether the gain comes from the hindcast-based reranking itself or from the variance reduction that any multi-candidate averaging provides. Moreover, the maximum evaluation horizon is only H=34. Since the claimed benefit concerns accumulated autoregressive error, the authors should report results at longer horizons or explain why H=34 is sufficient.
Questions for authors
My main questions are listed in the Critical Weaknesses. Besides, the practical value of Theorem 4.1 is limited. The error radius depends on constants and , and the manuscript does not estimate their magnitudes for the systems studied. As a result, the bound cannot be used to explain the size of the gains observed in deployment, nor to predict when the hindcast-based reranking should outperform one-step selection. The author can relate the bound to the empirical improvement more clearly or provide estimates or admissible ranges for these constants.