acceptodds
Under review as a conference paper at ICLR 2027

Searching Harder Hurts at a Fixed Budget: The Unreported Candidate-Set Size in Adaptive Collocation

Abstract

Adaptive collocation for physics-informed neural networks picks training points by scoring candidates and keeping the best. Across this literature spans four orders of magnitude and is almost never reported. It is not only a cost knob: keeping the best of is truncation selection at threshold , and the optimism of a maximum grows with the number of items scored, so sets how hard the criterion is optimised. We sweep at a fixed budget of points over shared seeds, against a uniform control re-drawn on the rules' own schedule. Two rules with unrelated scores, a residual magnitude and a training-dynamics score, are both level with that control at and both fail every seed by , on a sharp one-dimensional fit and on a Poisson problem, where that control holds at in : what collapses is the selection form, not the criterion. Published rules that soften that same maximum fail later and less, under , and of thirty analysis choices on the first of those. Let the set grow instead, as the shipped implementation does, and the same rule takes no harm from ; held at a fixed budget of its own final size it fails all seeds of a larger pool again. The harm is searching hard at a fixed budget, not searching hard. Which rules are exposed starts with one question: does selecting a point remove its own score? Necessary, not sufficient: our own arms subtract and collapse anyway; the objective has to be a function of the selected set. The effect survives four times the budget and seventeen times the pool, where the rule whose objective is defined on the selected set stops being immune; a registered attempt at 2,000 points missed its own accuracy gate. Report , and re-draw the control.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
physics-informed neural networks, adaptive sampling, collocation points, data selection, active learning, truncation selection, selection pressure, hyperparameter reporting, neural tangent kernel, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_FhC8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Core idea: Adaptive collocation methods for physics-informed neural networks (PINNs) pick B training points by scoring V candidates and keeping the best. Keeping the best B of V is truncation selection at quantile 1 − B/V, so V controls how hard the criterion is optimised. Yet V is almost never reported, and V/B varies across papers by about four orders of magnitude.

Main experiments: At a fixed budget of B = 40, the authors sweep V/B from 1.25 to 30 over 64 shared seeds on 1D tasks (a square-wave fit, “crenel”, and a Poisson problem with a sharp layer, “poisson1d”). They compare against a uniform control re-drawn on the rules’ own schedule.

Key findings:

  1. Pointwise rules degrade as V grows. RAR-G (the add-largest-residuals rule from DeepXDE), dynamicAL, PINNACLE, BADGE and RAD get worse. Two rules with unrelated scores are level with the control at V/B = 1.25 and fail every seed by V/B = 20, so the collapse is attributed to the selection form, not the score.
  2. Growth removes the harm. If the point set is allowed to grow, as in the shipped libraries, the same rule takes no harm. The damage is from searching hard at a fixed budget.
  3. Growth removes the harm. If the point set is allowed to grow, as in the shipped libraries, the same rule takes no harm. The damage is from searching hard at a fixed budget.
Critical strengths
  1. The candidate-set size is rarely reported, and the RAR-G snippet that scores 100k candidates to add one point makes the point well.
  2. It uses 64–100 seeds, failure rates rather than means over 3–10 seeds, shared seeds, and paired tests. It also has a multiverse analysis, pre-registered predictions, and failures reported openly, such as the failed 2,000-point gate and the failed replication on three tasks.
  3. The held-versus-re-drawn gap (35/64 vs 1/64 on poisson1d) is large, and the point that it changes which rules “cross” is valuable for benchmarking.
  4. The growing-set versus fixed-budget contrast is clean and directly relevant to how the cited libraries actually run.
Critical weaknesses
  1. The re-drawn control and the growing-versus-fixed-budget contrast are the most useful and defensible contributions. However, the evidence supports a much narrower claim than the title and abstract suggest. The harm is shown mainly at B = 40 on one- and two-dimensional problems that are either unsolved (burgers1d) or built to be irreducible (crenel), and it fades or becomes impractical by B ≈ 80 to 400.
  2. Everything is at B = 40, with 1–2D tasks, a 2×64 tanh network, and 4000 Adam steps. The field uses thousands of points. The authors’ own evidence shows how fragile the finding is outside this regime: Selection stops changing failure rates by B ≈ 80. At B = 400, PINNACLE’s V effect is “no longer large enough to act on.” A registered B = 2,000 cell missed its accuracy gate. On burgers1d, no cell reaches even a time-independent predictor.
Questions for authors
  1. Can you show the effect at a budget where uniform still has headroom and a published rule is used as published, on a PDE that is actually solved (for example Burgers with a hard constraint and L-BFGS)?
  2. Can you survey V/B across published PINN adaptive-sampling papers and codebases in a table?
  3. Which prediction of your theory could have failed and did not? So far the theory seems to accommodate both sign directions (capacity, GRAD-MATCH at B ≥ 80, crenel one-step decrease).
  4. Can you pin the reference and landmark counts at every rung to remove the estimator confound?
  5. Why is the headline not stated as “fixed-budget top-B selection on irreducible-residual tasks”?
Reviewer_BKHPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the impact of candidate-set size in adaptive collocation for physics-informed neural networks, showing that increasing the number of evaluated candidates under a fixed training-point budget can unexpectedly degrade predictive performance. Through experiments and an analysis based on the neural tangent kernel, the authors attribute this degradation to the mismatch between pointwise selection criteria and the final training objective, particularly when highly selected regions contain errors that the network cannot effectively reduce.

Critical strengths
  1. The paper identifies candidate-set size as an important yet frequently overlooked factor in adaptive collocation, challenging the common assumption that evaluating more candidates necessarily improves sampling quality.

  2. The experiments demonstrate substantial performance degradation as V/BV/B increases.

  3. The paper connects adaptive collocation with neural tangent kernel dynamics and analyzes why improving an instantaneous selection criterion may not improve terminal prediction accuracy.

Critical weaknesses
  1. The main findings are established primarily with a budget of only 40 collocation points, substantially smaller than those commonly used in practical PINN applications, for examples, 2000 to 100000 points. The authors acknowledge that their experimental budgets are one to two orders of magnitude below conventional PINN benchmarks. Although additional experiments extend the budget to 160 and 400 points, the attempted experiment with 2,000 points fails its accuracy admissibility criterion, preventing evaluation of the central phenomenon at a more realistic scale. This substantially limits the practical generalizability of the conclusions.

  2. The evaluation relies heavily on low-dimensional problems, including synthetic regression tasks that are not PDEs. Several of the strongest failure cases arise from sharp one-dimensional functions, while the paper does not evaluate spatially two-dimensional or three-dimensional PDEs. More importantly, the authors do not conduct experiments on established large-scale benchmarks such as PDEBench or PINNacle[1]. Consequently, it remains unclear whether the phenomenon has a comparable practical impact in realistic scientific computing applications.

  3. The theoretical explanation is based on linearized neural network dynamics and a fixed neural tangent kernel, whereas actual PINN training involves nonlinear, time-varying dynamics and repeatedly updated collocation sets. The first-order expansion provides an intuitive explanation for the mismatch between pointwise scoring and terminal error, but it does not establish a rigorous guarantee that increasing candidate-set size degrades performance in realistic training.

[1] Hao Z, Yao J, Su C, et al. Pinnacle: A comprehensive benchmark of physics-informed neural networks for solving pdes[J]. Advances in neural information processing systems, 2024, 37: 76721-76774.

Questions for authors

The main experimental findings are established primarily with B=40, whereas practical PINN applications often involve substantially larger collocation sets. Although the paper includes some larger-budget experiments, the attempted experiment with 2,000 points does not meet its accuracy criterion. Can the authors demonstrate that the proposed phenomenon persists under representative PINN training budgets where the baseline models achieve satisfactory accuracy?

Reviewer_JsyjOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
3
Read review
Paper summary

The paper studies the problem of choosing training points for learning the solution to a PDE. For the current iterate solution represented by a neural network, we can probe it at these training points to compute residuals that indicate how far the current iterate solution is from a true solution. The neural network can then be optimized to set those residuals to zero.

The points at which this training happens are picked by sampling many points and selected according to how important the residuals are determined to be. Picking many points and selecting only the largest residuals biases the sampling towards difficult regions and may not provide much training signal elsewhere, so a better selection mechanism is needed.

The point of the paper is the ratio of points sampled to the points kept for training (V/B) is an important quantity that should be measured and reported.

Critical strengths

This is an interesting problem, and the recommendation that is made seems useful broadly for people studying this problem.

Critical weaknesses

I found this paper very hard to read. Even just the abstract. For instance, consider this sentence:

It is not only a cost knob: keeping the best BB of VV is truncation selection at threshold B/VB/V, and the optimism of a maximum grows with the number of items scored, so VV sets how hard the criterion is optimised.

I am not sure that truncation selection is a common concept, and I dont understand what the optimism of a maximum refers to.

There are also some stylized sentences like

And almost nobody reports it.

as the opening sentence in a paragraph. I wouldn't call this an error, but I think the little space in the introduction can be better used explaining, for instance, that this paper is primarily about finding training points for training the solution to a PDE by using autograd to compute residuals. I am not even yet sure this is the case (though I think PINNs are largely about this), a conversation with GPT revealed this to me.

Questions for authors

No questions

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.