acceptodds
Withdrawn from ICLR 2027

Event Fidelity Reorders Neural-Operator Selection at PDE Transitions

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Average field error can favor a forecast that misses the shock, phase change, or pattern onset defining a PDE transition. We introduce Critical-Point-Bench (CPB), which evaluates event fidelity before ranking models by field error within the transition band. CPB rejects the entire candidate family when no model reproduces the event. A conditional-risk bound explains why global error loses control when transition events are rare. On the final Allen-Cahn holdout, CPB selects FNO over LinearNO and reduces critical normalized error from 0.00172 to 0.000414, a 75.9% decrease, while preserving phase recovery. On trajectory-held-out PDEBench, the model with the lowest aggregate error recovers only 21.0% of post-transition events, so CPB rejects it. Event fidelity therefore changes which model wins at a transition and reveals when no candidate has learned the defining physics.

Primary area:
datasets and benchmarks
Keywords:
neural operators, PDE benchmarks, critical transitions, event fidelity, model selection, PDEBench

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jnxGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper introduces Critical-Point-Bench (CPB), a transition-aware benchmarking and model-selection framework for neural operators. The main motivation is that aggregate prediction errors may not adequately reflect a model's ability to capture important physical transitions, such as shock formation, phase selection, and pattern onset. CPB defines task-specific transition coordinates to evaluate prediction errors near critical regions, together with binary and continuous measures of event fidelity. Models must first satisfy predefined event-fidelity thresholds before being ranked by their critical-region prediction error. The framework abstains when no candidate meets the requirements. The authors evaluate CPB across several PDE systems and neural operator architectures (FNO, DeepONet, CNO, Transformer, and LinearNO). The results show that CPB can lead to different model-selection decisions, selecting FNO over LinearNO on Allen–Cahn with substantially lower critical-region error, while abstaining on an external PDEBench evaluation where no candidate meets the event requirements. Additional experiments investigate transition-specific failure patterns, model capacity, and targeted training interventions.

Critical strengths

Comprehensive and physically meaningful evaluation. The paper highlights an important limitation of aggregate prediction-error metrics in evaluating neural operators near physical transitions. By combining transition-local prediction errors with event recovery and continuous measures of physical structure, the proposed benchmark provides a more informative assessment of model performance. The experimental evaluation is extensive, covering multiple PDE systems and neural operator architectures, with independent validation-to-holdout comparisons, multiple training seeds, matched controls, and sensitivity analyses.

Critical weaknesses

Limited evidence for the added value of event-based model selection. Although the paper demonstrates that aggregate and transition-local errors can lead to different model rankings, the experiments do not clearly establish the additional benefit of the proposed event-fidelity criteria. In the Allen–Cahn experiment, where CPB selects FNO instead of LinearNO, both models already satisfy the event requirements, and the selection difference appears to arise entirely from their critical-region prediction errors. A simpler baseline that selects models solely by critical-region error would therefore likely produce the same result. On PDEBench, CPB abstains because no candidate satisfies the event requirements, but this does not demonstrate a case where event-based filtering leads to a better model selection. Including a critical-error-only selection baseline and an experiment where event fidelity meaningfully changes the selected model would help establish the benefit of the complete CPB framework.

Questions for authors

Questions:

  1. Choice of event-fidelity thresholds. How are the task-specific thresholds for event recovery and continuous event discrepancy determined? Their choice could substantially affect model eligibility and potentially the selected architecture. For example, in the Allen–Cahn experiment, LinearNO has a lower continuous event discrepancy than FNO, so a stricter threshold could potentially reverse the reported selection. Could the authors clarify the rationale behind the chosen thresholds and investigate how sensitive the selection results are to them?

  2. Extension to more complex transition structures. CPB is evaluated primarily on systems characterized by a single transition coordinate and a relatively simple event definition. How would the framework extend to physical systems with multiple phase transitions or several interacting fronts? In particular, how would the transition coordinates and event-fidelity criteria be defined in these settings?

Comments and Suggestions:

  1. Clarity of the introduction. The introduction could be more streamlined to explain the problem setup and motivation more clearly. In particular, the different physical systems used to evaluate CPB could be introduced more clearly, and the paragraphs could be better connected to provide a more coherent progression from the problem motivation to the proposed benchmark.
Reviewer_yxh7OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses the limitations of traditional global error metrics in adequately evaluating the predictive performance of neural operators near critical transitions in PDEs. To tackle this issue, the authors propose an event-aware evaluation and model selection framework called Critical-Point-Bench (CPB). The framework combines event recovery rate, continuous event discrepancy, and critical-region prediction error. It first filters models based on their ability to satisfy predefined physical event criteria and then ranks the eligible models according to their prediction errors in critical regions. The research problem is important and meaningful. However, I believe the proposed solution still has room for improvement. The main concern is that many of the physical features and corresponding thresholds used in CPB are manually specified and somewhat subjective, without sufficient justification for their selection. These choices can substantially influence the evaluation results and model rankings, raising concerns about the reliability and objectivity of the proposed framework. Furthermore, the experiments mainly focus on well-understood PDE systems, leaving the applicability and generalizability of CPB to more challenging or previously unexplored PDE systems insufficiently demonstrated.

Critical strengths

This paper highlights an important limitation of neural operators near critical PDE transitions: a model may achieve low global prediction error while failing to accurately capture the solution behavior in critical regions. This research problem is of significant scientific importance and practical relevance. The authors point out that traditional global error metrics may not adequately reflect a model's ability to reproduce key physical events. To address this issue, they propose the Critical-Point-Bench (CPB) framework, which combines event recovery rate, continuous event discrepancy, and critical-region prediction error. This framework provides a new perspective on the evaluation and selection of neural operators. The experimental evaluation is relatively systematic, covering multiple PDE systems and neural operator architectures. The statistical analyses and various controlled experiments further strengthen the credibility of the reported findings.

Critical weaknesses

1.CPB involves several manually specified thresholds, including event detection thresholds and event matching tolerances. These choices are somewhat subjective and directly affect model rankings and final conclusions. Although the authors conduct sensitivity analyses, they do not provide sufficient justification for how these thresholds are determined. 2.Different PDEs exhibit different critical transition mechanisms. Why is it appropriate to adopt a uniform relative window ratio across different PDE systems? The authors should provide stronger physical or theoretical justification for this design choice. 3.How can the authors demonstrate that the selected physical features are sufficient to characterize the key physical events in the corresponding PDE systems? Different problems may emphasize different aspects of physical behavior. Therefore, the event-fidelity metrics should provide a more comprehensive characterization of the relevant physical phenomena. 4.The framework relies heavily on manually designed features and subjective choices, primarily for well-understood PDE systems. However, neural operators are also expected to generalize to previously unseen PDEs and capture complex physical behaviors. The current evaluation framework does not establish how such event features and criteria can be systematically defined for new or more complex PDE systems, raising concerns about its broader applicability and generalizability. 5.The paper places excessive emphasis on experimental analysis, while the motivation behind the proposed method and the relevant background are not sufficiently explained. The overall formatting and organization also need improvement. For example, the title is unnecessarily split across multiple lines, and the Introduction section is relatively brief, providing insufficient background information and motivation for the proposed approach.

Questions for authors

Please refer to the Critical Weaknesses.

Reviewer_hCQbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

This paper introduces Critical-Point-Bench (CPB), a method for selecting neural operators. CPB selects the models that reproduce the expected events, such as the shock, and then ranks them by their error near the transition. On four 1D tasks, it changes the selected model on Allen-Cahn and selects no model on PDEBench Burgers.

Critical strengths

The motivation is clear: a low average error does not guarantee that a shock or a pattern is reproduced correctly. The experiments are carefully designed.

Critical weaknesses

The practical benefit of CPB is limited. It changes the selected model only on Allen-Cahn, where both models already reproduce the events, and the error near the transition decreases by only 0.0013. The largest improvements in the paper, such as the 90% error reduction on Swift-Hohenberg, come from changing the model rather than from CPB.

The PDEBench result is difficult to interpret. Within each training run, the checkpoint is still selected by the average validation error, and there is no evidence that any model could pass the event check, so it is unclear whether the models or the selection procedure are at fault.

The presentation needs substantial improvement. The text is very dense, no figure compares predictions with reference solutions, and the Allen-Cahn curves in Figure 1 cannot be distinguished.

Questions for authors

Do the results change if the checkpoint of each training run is also selected with CPB?

Is there a case where the event check rejects the model with the lowest average error?

On Allen-Cahn, why is a 0.0013 reduction in the error near the transition worth a 52.6% increase in the average error?

How are the transition region and the event thresholds defined on PDEBench?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.