- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper studies the decision problem of whether to train a neural error corrector for a coarse PDE solution or refine that solution classically. For a given number of deployments, classical methods are considered affordable if their total computational cost does not exceed the end-to-end cost of the learned corrector, including reference-data generation, training, and inference. The corrector is worth training only if it outperforms both the best affordable classical method and a cheap linear regression baseline. However, this requires knowing the corrector's final error upfront, which is precisely what training would reveal. Therefore, the authors propose estimating the corrector's final performance from a small pilot run, using its early learning progress to predict the remaining improvement. The coefficients of this predictor are fitted on historical PDE problems and evaluated using leave-one-family-out validation, followed by a prospective evaluation on an unseen PDE family.
Critical strengths
- The paper studies a practical problem of choosing a performance-optimal method given a constrained budget.
Critical weaknesses
-
Lack of contextualization within existing literature. The paper insufficiently positions its main contribution within the established literature on learning-curve-based performance prediction, extensively reviewed by Mohr & van Rijn (2024, Learning Curves for Decision Making in Supervised Machine Learning: A Survey). Moreover, the authors' distinction between predicting remaining learning gain and extrapolating final error is not convincing. Writing , their estimator is algebraically equivalent to $ \log\hat e_{\mathrm{full}} = -w_0-w_1\log e_1+(w_1-w_2)\log e_2+(1+w_2)\log e_3, $ i.e., a constrained linear regression on intermediate log-errors, where the regression coefficients (excluding the intercept) sum to one. A more thorough contextualization is needed to clarify the methodological novelty of the proposed estimator beyond its application to cost-aware PDE solver selection.
-
Limited methodological contribution and insufficient experimental evidence. The paper reads more like an empirical case study than a mature methodological contribution to machine learning. Beyond the matched-cost formulation, the proposed method is a relatively simple linear predictor, and its broader significance remains unclear. The experimental evaluation further weakens the contribution: most baselines are either trivial heuristics or alternative estimators proposed by the authors. In particular, there is no convincing comparison with established learning-curve prediction methods under comparable computational budgets. Moreover, the presentation of results is somewhat misleading. Table 1 emphasizes aggregate performance, whereas the per-family breakdown in Table 6 reveals that the proposed estimator matches B2-d10 on three of the four PDE families, with its entire improvement concentrated on Burgers. This considerably weakens the evidence for generalizable methodological progress. The prospective evaluation on only ten regimes from a single additional PDE family provides limited further support. More generally, the manuscript is repetitive and would benefit from a more concise presentation, clearer separation of empirical observations from methodological contributions, and a more critical examination of the results. I would strongly recommend moving Table 6 to the main text, potentially replacing Table 1. Overall, the work appears more appropriate for a specialized computational PDE venue than for a general machine learning conference.
Minor weaknesses:
- Figure presentation. Several figures have readability issues:
- Figure 2: The legend overlaps with the local-linear/global-phase annotation.
- Figure 4: The Fisher–KPP family is included in the legend and discussed in the caption, but its points are not visible.
- Figure 5: The logarithmic y-axis makes zero-regret results for Fisher–KPP and Helmholtz invisible, potentially misleading the reader.
- Figure 7: Several method labels overlap, making the low-cost region difficult to interpret.
- Incomplete notation and terminology. The error metric (e), central to the proposed method, is not explicitly defined in the main text (relative error is only specified in Appendix D). The abbreviations M1b, M0c, and B2-d10 are also introduced without explanation. Furthermore, Appendix C uses to denote the full learned cost, whereas Eq. (1) introduces , without clarifying their relationship.
- Method presentation. Section 3.6 provides a lengthy numerical example, whereas a concise pseudocode description of the complete decision procedure would make the proposed method easier to understand and reproduce.
- Insufficient presentation of deployment-volume sensitivity. Section 6.6 and Appendix N only describe the changes in decisions as the number of deployments varies. A table or figure showing the corresponding decisions or breakeven points would better substantiate these claims.
- Unclear reference to earlier work. Section 6.3 describes B2-d10 as a calibration developed in the authors' "earlier work," but does not provide a corresponding reference or explain the origin of this method.
Questions for authors
I would like to emphasize that the answers to the questions below are unlikely to change my overall assessment of the paper, which is primarily driven by the more fundamental concerns outlined above. Nevertheless, I noted these questions while reading the manuscript and share them in the hope that they will help the authors clarify and strengthen the work in future revisions or submissions.
-
Evaluation protocol and reference solutions (Section 3.2–3.4). How exactly are and estimated before making a decision? How many validation instances are used, and where do their high-fidelity reference solutions come from? Are these the same instances used for training the pilot corrector, or a separate validation set? More generally, are , , and measured on the same fixed validation set? How is the cost of obtaining these references accounted for?
-
Cost accounting (Section 3.1, Appendix C). The paper reports computational costs in seconds and as fractions of full training cost. What hardware and software environment was used to obtain these measurements, and how were costs across numerical solvers and neural training made comparable? Moreover, when the decision is to refine classically, is the cost already spent on the pilot subtracted from the available refinement budget?
-
Role of the ridge baseline (Section 3.2, Section 3.5). Why is the ridge corrector treated solely as an eligibility criterion rather than as a third deployable alternative? If it outperforms both the neural corrector and the classical refinement, the proposed decision rule still selects Refine. How is this justified from an operational cost–accuracy perspective?
-
Ridge regressor specification (Section 3.2, Section 6.1). The ridge baseline is described as ridge regression on local stencil features, but its regularization is not specified. Is an penalty actually applied? If so, what is the regularization coefficient, and how is it selected? Is the coefficient fixed across PDE families, or tuned separately for each problem?
-
Choice of pilot configuration (Section 3.3, Section 6.1). What motivates the particular selection of 10% of training pairs, 20% of the epoch budget, and an 8-channel CNN? Likewise, why are the progress features evaluated at 5%, 10%, and 20%? Has the sensitivity of the decision quality and computational cost to these choices been investigated? If so, could it be presented as an ablation study?
-
Data preprocessing and training (Section 6.1, Appendix B). Are the inputs and regression targets normalized when training the neural correctors? Are the stencil features used by the ridge regressor normalized? The appendix also states that the pilot is trained on the first 26 training pairs. Are the pairs randomly ordered, and has the sensitivity to their selection been investigated?
4 comments
Response to Reviewer 9Scs (2/3)
W1: Literature and what the estimator is
"The paper insufficiently positions its main contribution within the established literature on learning-curve-based performance prediction, extensively reviewed by Mohr & van Rijn (2024, Learning Curves for Decision Making in Supervised Machine Learning: A Survey)."
The measured part of the distinction is that a frozen absolute calibration scores 0.879 against 0.970 for the raw pilot on 33 cross-family cases, on a classical-dominated transfer set where the majority policy is also 0.970 (Section 6.4). Revision. Section 2 and Section 5.
W2: Contribution and the per-family evidence
"In particular, there is no convincing comparison with established learning-curve prediction methods under comparable computational budgets."
Answered above under What is new, and the curve baseline at matched cost for the baseline half and under The prospective set is ten regimes, eight of them Refine for the prospective half; for the per-family point, the table you read as Table 6 moves into the main text as Table 2, beside the aggregate in Table 1. The two estimators tie on advection-diffusion at 0.132 and on Fisher-KPP and Helmholtz at zero, and the difference is Burgers, 0.022 against 0.042. On the advection-diffusion fold 27 of 32 cases are Learn and the rule is correct on 27, with 4 false positives and 1 false negative. The margin over the frozen calibration is one case in 74, macro regret 0.038 against 0.044, and we report it as a one-case margin (Section 6.4; Table 1). On the venue question we have no measurement to offer; Section 6.2 now separates development, model-selection and prospective evidence, and the Discussion is compressed.
Minor weaknesses
"Figure 2: The legend overlaps with the local-linear/global-phase annotation."
The paper fixes two of the four figure points and leaves two: Fig. 4 now draws the Fisher-KPP points that its caption discusses, Fig. 5 uses a symlog axis so the eight zero cells in Table 2 sit on the zero line, Fig. 2's legend still covers part of the diagonal annotation, and Fig. 7's labels still crowd the low-cost region. On notation, M1b, M0c and B2-d10 are now defined in Section 6.3 and named in the Table 1 rows, App. C uses , the same symbol as Eq. 1, and the error is relative , stated in the Table 5 caption and App. D, though still not in Section 3. On method presentation, no pseudocode box is added: the procedure is Eq. 3-Eq. 6 plus the Section 3.4 paragraph, and Section 3.6 keeps the worked example. On deployment volume, Section 6.6 and App. N give the decisions across , several regimes move from Refine to Learn as grows and none move back, and no breakeven table is added, though Table 5 lets a reader recompute a flip point. On the earlier-work reference, Section 6.3 now defines B2-d10 as a log-linear power-law map from pilot error to full error, fit once on advection-diffusion and never refit, and since it is our own earlier calibration there is no third-party reference to add.
Q1: Evaluation protocol and reference solutions
"Are these the same instances used for training the pilot corrector, or a separate validation set?"
and come from the classical frontier at the affordable resolutions, from the pilot run, and all three are scored on the same 12 episodes (Section 3.1). Across the calibrated estimate moves by 17.3% and stays above the true full error; that check is reported here only.
Response to Reviewer 9Scs (3/3)
Q2: Cost accounting
"What hardware and software environment was used to obtain these measurements, and how were costs across numerical solvers and neural training made comparable?"
The second half of your question is answered above under The pilot inside the matched budget; on hardware, the paper reports wall-clock seconds per arm (Table 7) and does not add a hardware table, and App. C defines the denominator of every reported fraction.
Q3: The ridge baseline as a gate
"Why is the ridge corrector treated solely as an eligibility criterion rather than as a third deployable alternative? If it outperforms both the neural corrector and the classical refinement, the proposed decision rule still selects Refine."
The ridge is a learned control fitted to the same training pairs, not a classical arm, so the budget comparison cannot price it as refinement (Section 3.2; Section 6.1). The ground-truth winner is defined as the gate winner, so a ridge-best regime is one where the corrector should not be trained at all (Section 3.5). The gate winner and the better arm differ in 1 of 84 decisions, AC11, where all three errors are within 5% and the regret is zero (Section 3.5; Section 6.5). Revision. Section 3.5 states the two-arm oracle and the gate.
Q4: The ridge regressor
"Is an penalty actually applied? If so, what is the regularization coefficient, and how is it selected?"
It is an -penalized fit on local stencil features [1, u, , ] (Section 3.2). The paper does not report the coefficient or how it was selected, and Table 5 gives the resulting error per regime. The control is a floor that any corrector must beat, not a tuned competitor, so a coefficient search is not part of the decision rule.
Q5: The pilot configuration
"What motivates the particular selection of 10% of training pairs, 20% of the epoch budget, and an 8-channel CNN? Likewise, why are the progress features evaluated at 5%, 10%, and 20%?"
The pilot is set at 10% of the training pairs and 20% of the epoch budget, which puts the decision at about 15% of full training (Section 3.4). The choice is not a single point: a 3x3 grid over width {8, 16, 24} and budget fraction {0.10, 0.20, 0.30} gives a calibrated estimate between 0.025 and 0.048, monotone in budget and flat across width (0.04785 at every width), against a true full error of 0.016 (App. B.2). The 5%, 10% and 20% checkpoints come from that one run and define and in Eq. 4; the grid bounds the estimator's input rather than selecting a favorable cell. Revision. App. B.2 reports the grid.
Q6: Preprocessing and pair order
"The appendix also states that the pilot is trained on the first 26 training pairs. Are the pairs randomly ordered, and has the sensitivity to their selection been investigated?"
The pilot takes the first 26 pairs in a fixed order, and the full corrector takes all 256 (App. B.2). A second seed with a shuffled pair order moves the pilot estimate by 6.2%, and both estimates stay above the true full error; that check is reported here only. On normalization, the paper states only that training minimizes MSE on the reference residual (App. B.4) and does not report input or target scaling. Revision. App. B.2 states the pair order.
Follow-up to Reviewer 9Scs
We would like to follow up on our rebuttal before the author-reviewer discussion closes on Nov. 18 at 11:59 p.m. AoE. The rebuttal answers each of your comments in detail. In particular, it covers the learning-curve baselines at matched cost. At the same pilot cost, three-point power-law and ensemble curve fits reach 62/74 correct decisions, against 66/74 for the two-feature estimator (Section 6.4, Table 8). The rebuttal also covers the pilot, which is now charged in Eq. 1 and leaves all 84 decisions unchanged (Section 7), and the reference protocol behind every decision-time error (Section 3.1). We hope these additions give a fuller picture of the work.
We would also like to highlight the main result of the paper. Learned correctors for PDE solvers are already widely studied, and they are usually judged by their accuracy against a fixed reference after training. Refining the grid spends the same resource that the corrector spends on training data. Our paper formulates this as a matched-cost decision. The Learn-or-Refine rule decides before the full training cost is paid, at about 15% of full training. Over 74 earlier regime-architecture cases it decides 66/74 correctly in leave-one-family-out evaluation. After the estimator was frozen, it decides 9 of 10 regimes of an unseen Allen-Cahn family correctly, with zero mean and zero worst-case decision regret. These results are, to the best of our knowledge, new in the literature on learned PDE correction. Your review notes that the paper studies a practical problem, the choice of the best method under a constrained budget, and we hope the new controls complete this picture.
We would be very grateful if you could read the rebuttal before the discussion closes and tell us what you think. If it resolves your concerns, we kindly ask you to consider raising your score. If anything remains open, please let us know and we will answer right away.
Thank you for your time and consideration. We look forward to your reply.
Response to Reviewer 9Scs (1/3)
We thank the reviewer for their comments.
The prospective set is ten regimes, eight of them Refine
All three reviews converge here, and the paper reports the prospective result against the 8/10 majority policy rather than as a headline accuracy. The interval is [0.60, 0.98], attached to a ten-regime check on one unseen family rather than to a general claim (Section 6.5). The rule catches both Learn cases (AC04, AC14), and its single error is AC11, where efull, ecls and eridge are within 5% and the regret is zero (Section 6.5; Table 6). Against never training the paired counts are 2 to 1, exact p=1.0 (Table 3). Five planned regimes were dropped or replaced before any outcome was observed, which leaves the set at ten (Section 6.5; App. P).
The pilot inside the matched budget
The pilot is charged inside Clearn rather than subtracted from the refinement budget: Eq. 1 carries it as a fourth term, so a classical arm is affordable when its per-rollout cost times N stays within that same budget (Section 3.1; the same accounting answers Reviewer gTYB's Q1). Charging the 63.7 s pilot moves 0 of 84 decisions across the 74 historical and ten prospective cases (abstract; Section 7). The decision cost on Allen-Cahn is 14.7%, against 4.9% for the classical-only probe, which trains no pilot (App. C; Table 3). The pilot is a separate 8-channel network and the full corrector is wider (Section 6.1), so no pilot parameter is reused and nothing is amortized into the corrector.
Where the decision-time references come from
Every decision-time error is scored on nval=12 pilot-evaluation episodes drawn from a separate RNG stream, and that cost is charged inside Cpilot (Section 3.1; Eq. 1; this is the protocol behind Q1 below). The 26 pilot pairs are training data, and the held-out test set supplies the labels that score a decision, so the references are not the pairs that trained the pilot. The three progress checkpoints are read from one pilot run, so e_{5\\%}, e_{10\\%} and e_{20\\%} are measured on the same fixed episodes (Section 3.1; App. C).
What is new, and the curve baseline at matched cost
The estimator is a damped log-linear extrapolation and the paper states its form, with a fitted slope of 1.613 against the analytic 2.322 (Section 5). The two-feature form is a constrained linear regression on intermediate log-errors, as you show; the claim is the damping and the matched-cost decision it feeds, not a new extrapolation law (Section 5). Section 2 now positions it against the learning-curve literature (Mohr & van Rijn, 2024; Domhan et al., 2015; Li et al., 2017), where the decision is which learned configuration to keep or when to stop it, not whether to learn at all. The matched-cost curve fits that two reviews asked for are the comparison reported in the general response, and Table 8 lists the fits.