acceptodds
Under review as a conference paper at ICLR 2027

Test-Time Compute Allocation in Random-Feature PDE Solvers: Width versus Search

Abstract

Under a fixed work budget, should a random-feature partial differential equation (PDE) solver widen one trial space or search across several? We compare these allocations using the same feature distribution and fitting procedure. An exact error decomposition separates candidate-set quality from final selection loss using a post-solve reference-error oracle. Across 60 independently sampled linear advection–diffusion–reaction problems that activate search, six budgets, and four seeds, the task-geometric search/wide error ratio is 9.32 (95% CI 7.92–10.91); search wins on none of the 60 tasks under this paired statistic. Oracle selection from the same candidate sets still yields a ratio of 9.26, while final selection adds only 0.7% to the geometric mean error. Thus, almost the entire gap is already present in the available candidates. Ablations nevertheless show that physics-based selection is useful within those populations. Finite-set ranking guarantees and a spectral residual bound for a restricted elliptic operator class clarify when verification controls ranking and solution error. These findings identify candidate generation and compute allocation as the main bottlenecks for the tested search policy: accurate final verification cannot compensate for the absence of a competitive candidate.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
test-time compute, random-feature methods, PDE solvers, compute allocation, physics-based model selection, scientific machine learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_DjXhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies the allocation of work budget in random-feature PDE solvers. The two strategies HybridWide and Gate-RFM+ (one focuses on a wider trial space and the other focuses on several smaller candidates with physics-based selection among them) are compared under the same feature distribution and fitting process. The main experiments on 60 linear advection-diffusion-reaction problems show that the search policy is worse than the policy with a wider model. An oracle analysis also indicates that most of the gap is present in the generated candidate sets.

Critical strengths
  1. The paper studies an interesting and practically relevant problem: how to allocate limited computation in random-feature PDE solvers. The results offer insights into the trade-off between increasing model capacity and searching over more candidates.

  2. The experiments are extensive, comparing several allocation strategies under different budgets and including detailed ablation studies.

Critical weaknesses

My main concerns are the following two points:

  1. The significance of the contribution is not very convincing to me. The question itself is interesting and meaningful, and the paper does provide a detailed analysis of why the search policy performs worse than a wider model. However, even with the additional evaluations, it remains unclear how far these findings extend beyond specific solver designs and allocation rules studied in the paper. As a result, while the negative result is useful, I'm not sure that it provides substantial new understanding of compute allocation.

  2. The experimental design feels somewhat ad-hoc, particularly how the work budget is defined and allocated between fitting and verification. It's not very clear to me how robust the conclusions are to different cost assumptions or allocation rules.

Also, the presentation introduces many implementation details without making their relevance to the central conclusion clear, which is not very friendly to first-time readers.

Questions for authors
  1. Could the authors clarify what broader insights about compute allocation can be established from the results? And which of them depend on the specific experimental settings?

  2. How were the costs of fitting and verification determined?

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your careful review. Your concerns about significance, cost assumptions and presentation led us to add direct comparisons and clarify the contribution. All locations refer to the revised manuscript.

W1 & Q1: Broader insights and dependence on the setting. We agree that the experiments do not establish a universally optimal allocation. We now emphasize three conditional lessons (Sec. 6, p. 10, Appendix H.4, p. 41).

First, candidate quality and final selection require separate evaluation. For a completed population with positive errors, selected/wide equals (oracle/wide) × (selected/oracle). The primary geometric ratios are 9.318, 9.255 and 1.0068. Improving only final selection cannot repair those populations. Intermediate promotion, fitting or candidate construction can still change them. This is an algebraic diagnostic requiring reference-error auditing, not a new approximation theorem or an online controller for unknown solutions (Sec. 3.3, p. 4).

Second, useful scoring need not justify the allocation producing the candidates. Random final choice is 255-fold worse within the same completed populations despite their small remaining oracle gap. This separates scoring value from population-generation value (Sec. 5.3, p. 8).

Third, width/diversity competition depends on construction and costs. With N_f=8M and no held-out evaluation, an unrounded fit costs 24M^2+(40/d)M^3. Ideal equal one-off splits reduce width as K^(−1/2) or K^(−1/3) in the respective dominant-term regimes, with crossover near M=3d/5. This is conditional cost algebra, not a hardware or accuracy law (Appendix H.2, p. 36). Periodic reversals and enrichment results demonstrate why the numerical ordering cannot extend to all solvers or PDEs. The finite-set ranking and restricted elliptic bounds do not ensure competitive candidates exist.

W2 & Q2: Cost origin and allocation robustness. We appreciate that the original convention needed explanation. The inherited 1/64 scale kept operator and linear-algebra terms visible at pilot widths. That pilot used a different leading-QR expression and does not derive or calibrate the later proxy. We now distinguish these conventions and measured online time (Appendix H.1, p. 35).

We reran the action cost as floor(3N_fM + (4N_fM^2+8M^3)/d + v(N_s+N_a)(3M+K_v)), with arithmetic denominator d in {16,64,256} and held-out price v in {0.25,1,4}. Search uses N_f=max(256,8M), N_s=256, N_a=512 and K_v=32. Wide has N_s=N_a=0. Default prices are d=64 and v=1. Smaller d makes arithmetic more expensive, including enrichment screening. Scores and evaluation sizes stay fixed. Every promotion pays a full refit, acceptance work is reserved, and fresh acceptance points follow population completion. Reference errors never select an online action.

The five policies are original ladder-wide/search, integer-width filled-wide, filled-search and frozen E64. Filled-wide takes the largest affordable integer width. Filled-search preserves the original search prefix and uses remaining work for wider prefixes or new banks chosen by search scores. All actions are charged. Width caps and action sizes can still leave unequal unused work, so this is a common-ceiling comparison, not equal wall time.

All 38,880 distinct runs completed on the same 60 PDEs, six budgets and four seeds without failures or overspending. We reran affordability decisions rather than relabeling errors. Default prices reproduce 4,320 earlier wide/search/E64 paths, and every filled-search prefix matches. Inference averages paired logs within PDE. Marginal 95% intervals resample the 60 PDEs, with a separately declared 441-contrast global Holm correction (Appendix H.2, p. 36).

Across nine pooled price cells, selected/wide is 2.92–140, oracle/wide 2.88–139, and filled-search/filled-wide 4.39–291. The selected and oracle intervals all exceed one, with Holm-adjusted p=0.00882. Individual tasks and budgets can differ. E64/filled-wide is 0.344 (0.245–0.486), 0.929 (0.647–1.315) and 4.16 (2.99–5.76) at d=16,64,256. Its default advantage is inconclusive, while the opposite effects at 16 and 256 both have adjusted p=0.00882 (Table 5, p. 9, Table 37, p. 38). All budgets and adverse outcomes remain in Appendix H.3 (p. 37). These finite checks reuse tasks and do not calibrate arbitrary hardware costs. Earlier verification-quota and deadline studies remain separate (Appendix F.3, p. 28).

Presentation. Thank you for this suggestion. The Introduction now foregrounds allocation diagnosis. Sec. 5.3 (p. 8) links ablations to representation, generation or selection. Table 5 (p. 9) summarizes costs and allocation, while Appendix H.5 (p. 41) retains every component outcome, including null and adverse results. The revised main text remains 10 pages.

Thank you again for helping us clarify the broader insights and their limits.

Thank you for the detailed response and additional experiments. The cost-sensitivity analyses and clarifications address much of my concern about the experimental setup. However, I still find the broader significance unclear: the findings remain closely tied to the particular solver designs and allocation rules studied, and I am not convinced that they offer enough new understanding beyond these settings. I appreciate the authors’ efforts, but I will maintain my original rating of weak rejection.

Reviewer_V88pOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Motivated by the test-time compute allocation, this paper investigates how a fixed computational budget should be allocated in random-feature PDE solvers. Particularly, it compares fitting a single wide random-feature bank with searching over multiple independently initialized, narrower banks using a population-search strategy with residual-based verification. Through extensive numerical experiments, the paper demonstrates a negative result that the population-search approach underperforms the single wide-bank baseline given the same computational budgets, even when the best candidate is selected (oracle selection). By decomposing the performance gap into candidate-generation quality and selection error, the paper provides a more detailed understanding of why the proposed population-search strategy fails to outperform the wide-bank approach.

Critical strengths

The paper presents, under controlled computational budgets, a systematic numerical comparison across multiple PDE instances and random seeds. Particularly, the oracle-selection analysis is useful for distinguishing limitations of candidate generation from those of final candidate selection. Finally, ablation studies greatly help with clarifying the individual algorithmic components' contributions.

Critical weaknesses

Personally, I find the motivation for the proposed PDE-solving strategy insufficiently convincing. It is unclear to me why searching over independently fitted random-feature banks is a compelling approach to solving PDEs. For instance, adaptive or principled feature selection could exploit information about the PDE to construct a more effective approximation space. Furthermore, if such adaptive procedures are computationally feasible, it is unclear why one should prefer searching over multiple narrower banks rather than allocating the computational budget to a single wider bank.

Although the experiments demonstrate that the proposed search strategy underperforms a single wide bank, even with oracle selection, this observation is not particularly surprising. Each narrower bank has fewer features available to approximate the PDE solution, potentially limiting its approximation capacity. More importantly, while the paper discusses the theoretical breadth–width trade-off, it provides limited insight into the conditions under which the potential benefits of random search could outweigh this disadvantage.

The scope and generalizability of the conclusions are limited. The experiments focus on a specific population-search strategy and a selected family of PDE instances. Consequently, it remains unclear how the findings relate to broader adaptive random-feature methods, particularly those that modify or enrich the approximation space based on residual information.

Questions for authors

Q1 (Motivation) What motivates searching over independently fitted random-feature banks rather than using PDE residual information for adaptive feature selection? Is this type of strategy well-established in the PDE literature? If not, what makes it a compelling alternative to existing representation-adaptive approaches?

Q2 (Terminology) Could the authors clarify why the residual-based evaluation and selection procedure is referred to as “verification”?

Q3 Can the authors provide insight into the conditions under which population search might outperform a single wide bank? For example, how does feature width affect the distribution of approximation errors across independently generated random-feature banks?

2 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your careful review and for suggesting residual-informed feature construction. We added this comparison and revised the motivation and conclusion. W1–W3 denote your three unnumbered weakness paragraphs in their original order. All locations refer to the revised manuscript.

W1 & Q1: Motivation and adaptive-feature comparisons. We agree that residual-adaptive construction is an important comparator. Gate-RFM+ tests a controlled allocation hypothesis: whether diversity across random banks compensates for smaller widths and repeated fitting costs. It is not an established standard PDE algorithm or a recommendation to prefer restarts. The original comparison favors wide over this particular policy (Sec. 2, p. 2, Appendix G.1, p. 31).

Both policies already use PDE coefficients and forcing statistics. Bank search promotes lineages by residual scores, but each lineage grows through random prefixes. Our new enrichment control instead selects features by normalized correlation between their weighted PDE responses and the current residual. E16 adds batches of 16. E64 selects the largest affordable batch from {16,32,48,64} before scoring a four-times-larger pool. Proposal, screening and refitting are charged. Matched random controls use identical widths and costs.

The 8,640-run follow-up reuses the same 60 PDEs, six budgets and four seeds. E16/ladder-wide is 1.73 (95% CI 1.29–2.32), whereas E64/ladder-wide is 0.320 (0.221–0.452). E16/Random16 and E64/Random64 are 0.0666 and 0.0505. Statistics average paired logs within PDE, with PDE-level marginal intervals (Table 4, p. 9, Appendix G.2, p. 32). However, default E64/filled-wide is inconclusive: 0.929 (0.647–1.315, Table 5, p. 9). Residual guidance helps against matched random construction, while superiority to a strong wide control depends on growth and costs. We discuss AG-RaNN and SFRBE-RaNN, but do not present our control as a reproduction of either full method.

Q2: Meaning of verification. Thank you for identifying this ambiguity. We now define it as label-free physics-based evaluation and selection. Fitting points determine coefficients. Search points are reused for promotions, so candidates can depend on them. Fresh independent acceptance points are drawn only after the completed population is fixed. Neither scoring sample contains reference-solution labels (Sec. 3.4, p. 5). The ranking guarantee concerns finite weak moments under boundedness and independence. The spectral result requires its restricted elliptic assumptions and unresolved tail. A finite score therefore does not certify general PDE solution accuracy (Appendix G.6, p. 34).

W2 & Q3: When diversity can overcome reduced width. We agree that a distributional explanation is needed. For independent width-M banks with fitting data fixed, the ideal K-bank oracle has survival probability [1−F_M(e)]^K. Under a fixed budget, width also changes. The narrow-bank lower tail must improve enough to beat the affordable wide-bank median, and final selection must preserve that gain. This conditional model does not assume shared-feature descendants are independent or predict a crossover for an unseen PDE (Appendix F.4, p. 29).

Figure 4 (p. 34) reports 10th/50th/90th bank-error percentiles at five widths for three feature laws, aggregated over 16 fixed settings. These are distributional quantiles, not confidence bands. In the fixed-point hybrid audit, width reduction costs 65.2 while two-bank diversity recovers 1.87, giving oracle/wide 35.0 (3.57–868, Table 28, p. 31). These measure fitted errors, not continuous best approximation. Conversely, at periodic Burgers budget 6M, search uses 32–48 coefficients versus 64 for wide yet achieves ratio 0.773 (0.756–0.791, Table 27, p. 30). E64 also loses to ladder-wide at 24M and 48M, with ratios 1.74 and 2.14 (Table 32, p. 33).

W3: Generalizability. We restricted the negative finding to the tested bank-search policy. Row adaptation, pruning and residual enrichment change different components. Optimized feature laws, depth growth and full nonredundant block enrichment remain outside this comparison (Sec. 6, p. 10). Across nine pooled price settings, search/wide remains 2.92–140. E64/filled-wide changes from 0.344 (0.245–0.486) with expensive arithmetic to 4.16 (2.99–5.76) with cheaper arithmetic. Its default comparison is inconclusive. All budgets and adverse outcomes are retained, and these 38,880 runs reuse the same 60 PDEs (Appendix H.3, p. 37). We recommend comparing strong wide and adaptive controls under explicit costs before generalizing.

Thank you again for helping us test this alternative and clarify the scope.

Dear authors, thank you for putting substantial effort to explaining and running new experiments.

W1 & W3. I appreciate the reviewers being able to narrow their claims, but as mentioned in your rebuttal to reviewer aB7w, the baselines and comparisons are against controlled method (e.g. Gate-RFM+) which does not seem to be the standard approach. That being said, the comparison is being made against somewhat synthetic approach, which gives more conviction on the lack of motivation (W1) for the comparison.

Q3. Thank you for clarifying the terminology.

W2 & Q3. I appreciate the authors' honest comparison and substantial reruns of experiments. Appendix G.3 do seems to have outcomes supporting my concern, but this weakens the paper's main message, as random-bank search vs width no longer appears to be the most informative comparison.

Therefore, my main concerns regarding the motivation stays, and I am maintaining my score.

Reviewer_aB7wOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2
Read review
Paper summary

The paper pertains to RFM for representing PDEs. It investigates how to optimally allocate a fixed compute budget when using RFM, i.e., whether it is better spent increasing the width of a single feature bank or exploring multiple smaller candidate models and selecting among them. The two approaches are experimentally evaluated on 60 tasks from a range of 2D linear advection-difffusion-reaction PDEs. Across all primary tasks, with six different budgets and four random seeds, the single-wide baseline outperforms the proposed search policy.

Critical strengths
  • Useful distinction between two sources of failure: The paper distinguishes between failing to generate sufficiently good candidates and failing to identify the best candidate among those generated. This is a useful framework for understanding the limitations of search-based allocation strategies.

  • Clear research question and experimental diagnostics: The paper sets out a relatively clear research question and the experimental framework needed to compare width and search. The authors conduct a broad set of experiments, including comparisons with several alternative search strategies and feature representations, as well as ablation studies. They honestly reported valuable and surprising results.

Critical weaknesses
  • Limited generality of the main conclusion: The experiments provide evidence for a specific task distribution, feature representations, and search configurations. However, the broader conclusion that increasing width is generally preferable to search requires further evidence across more heterogeneous PDE settings and feature distributions. The paper does include additional experiments, but it remains unclear how sensitive the conclusions are to the choice of basis functions and the characteristics of the underlying PDEs.
  • Weak presentation and evaluation metrics: The performance metrics, i.e., compute workload (equation 4) and “online time”, require clarification. Result tables compare numerous methods and metrics with inconsistently formatted numbers, making interpretation difficult; highlighting the most relevant results and explicitly indicating whether lower or higher values are better would improve readability. The section headings are inconsistent, mixing topic descriptions, methodological labels, and substantive conclusions, which makes the paper's structure difficult to grasp. Additionally, the paper would benefit from a clearer explanation of the motivation for using RFM and its general relevance within the broader landscape of PDE-solving methods.
Questions for authors
  1. RFM: Could the authors compare or explain the accuracy and efficiency of the investigated RFM methods with other learned representations, such as deep learning surrogate models? What are the main advantages and weaknesses compared to these? In such approaches, extensive benchmark datasets are available, including PDEBench and AirfRANS. Can the method presented by the authors be tested on such data and benchmarks?
  2. “Test-time”: The paper cites Snell et. al. 2025 (https://arxiv.org/abs/2408.03314)’s investigation of test-time compute budgets in LLMs where the training and testing distinction is clear. Could the authors clarify how this relates to their work? This paper’s title, “test-time compute…” seems to perform a similar investigation. What does test-time mean here?
  3. Gate-RFM+: Is the method “Gate-RFM+” your contribution or a standard method? The “+” indicates an improvement over an established method; nevertheless, it should be clarified or cited.
  4. Theorems: The paper provides several theoretical results, but the main conclusion appears to be driven primarily by the empirical comparison. Could the authors clarify how the theoretical results inform the central finding? Are there theoretical insights that could help identify when the opposite outcome might be expected? Or do they only provide the experimental framework?
  5. Basis functions: An RFM uses a finite span of selected basis functions. It may perform poorly if the feature bank does not contain suitable functions, even if the coefficients are estimated accurately. How sensitive are the results to the feature distribution and its ability to represent the underlying solution? Can the authors identify problem characteristics that would help determine a suitable feature distribution in advance?
  6. Warm-starting search strategies: The paper compares several search strategies. The proposed search procedure progressively widens surviving feature banks, but each promotion appears to require a full refit. Have the authors considered incremental approaches that reuse information from previous fits as the feature bank grows, rather than repeatedly solving the coefficient-fitting problem from scratch? Could such approaches reduce search overhead and potentially change the trade-off between width and search?
  7. Model size: If my understanding is correct, the investigated wide model has more learned parameters than the searched ones. Usually, larger models are expected to show better performance. Yet this work’s main result - width better than search - is presented as a surprising outcome. Can the authors explain the novelty in their findings?
  8. AI-Use statement: We recommend providing a complete statement with all disclosures. What is currently missing: “We have not used generative AI tools for <required disclosures>, and <other required disclosures> are not applicable to this work.”
4 comments
AuthorsOpenReview ↗

Official Comment by Authors

Thank you for your careful review. We revised the claims and comparisons. All locations refer to the revised manuscript.

W1: Generality. We agree that the result does not establish a universal width advantage. The primary evidence concerns 60 independent gate-active ADR PDEs. Budgets and seeds are repeated measurements. Periodic adapters can reverse the ordering, and residual-guided enrichment changes the comparison. Its default E64/filled-wide ratio is 0.929 (95% CI 0.647–1.315), so superiority to this stronger wide control is inconclusive. We narrowed the Abstract and conclusion accordingly (Sec. 6, p. 10).

W2: Work, timing and presentation. Eq. (4) is a declared work proxy, not calibrated hardware FLOPs. Every promotion pays a full refit. Online timing includes the gate, proposals, construction, fitting, scoring and allocation, and excludes reference construction, error auditing and oracle selection. Equal budget ceilings need not imply equal elapsed time or consumed work. We separate utilization and timing, standardize formats and metric directions, and retain full ablations (Sec. 3.2, p. 3, Appendix H.1, p. 35). RFM makes trial spaces and allocation costs explicit, without establishing general superiority over other PDE solvers.

Q1: Learned surrogates and public benchmarks. We added corrected PDEBench advection/Burgers and all four AirfRANS test pools. PDEBench wide/search/FNO median relative L2 errors are 0.0198/0.0116/0.00214 and 0.309/0.265/0.0193. FNO is more accurate. The MLP is also more accurate in every AirfRANS pool. PDEBench RF uses PDE/initial data without interior labels, whereas AirfRANS RF is supervised regression. These are controlled implementations, not official-code reproductions. Neural baselines use one training seed, and no equal-total-compute advantage is claimed (Table 6, p. 10, Appendix F.1, p. 25).

Q2: Test-time. This means solving after a new PDE and its prescribed inputs arrive, including basis sampling and coefficient fitting. The LLM analogy concerns allocating online computation among candidates. It does not assume an identical training/testing pipeline (Introduction).

Q3: Gate-RFM+. This is our controlled policy. The “+” distinguishes our strengthened implementation from an internal prototype, not a published predecessor. HybridWide is our name for the shared-feature single-bank control. Established RFM and successive-halving components are cited separately (Sec. 3.2, p. 3).

Q4: Theory and reversals. Theory does not require search to lose. The exact selected/wide decomposition separates oracle/wide from selected/oracle. Primary ratios satisfy 9.318 ≈ 9.255 × 1.0068, locating almost all loss in the completed populations. Intermediate promotion or fitting can still change them. Appendix F.4 (p. 29) gives a conditional independent-bank crossover criterion. Theorem 1 concerns finite weak-moment ranking under boundedness and fresh acceptance. Theorem 2 requires its restricted elliptic assumptions and unresolved tail (Sec. 3.4, p. 5).

Q5: Feature choice. The 46,080 fits cover 24 fixed settings, not independent PDE replications. With fixed fitting points, hybrid bank-median errors improve with width, while tanh can worsen. Audit-only reference-field fits distinguish fitting failure from inadequate span. PDE coefficients, advection direction and forcing localization guide our shared feature law, but no universal optimal-law rule is established (Appendix F.2, p. 27, Figure 4, p. 34).

Q6: Incremental fitting. We added row/column QR reuse with scaled SVD. Across 64 paired paths, maximum relative prediction discrepancy is 3.13e-9. Updated/cold time ratios are 1.014 for nested rows and 1.615 for independently resampled rows. Thus this implementation shows no general speedup. All 768 deadline runs, including 77 overruns, remain reported (Appendix F.3, p. 28).

Q7: Model size and novelty. At periodic Burgers budget 6M, search uses 32–48 coefficients versus wide’s 64, yet achieves ratio 0.773 (0.756–0.791). In the fixed-setting hybrid audit, width reduction costs a factor 65.2 while two-bank diversity recovers 1.87, giving oracle/wide 35.0 (3.57–868). Our contribution is the controlled allocation diagnosis and its measured competing effects and reversals, not a new approximation theorem (Appendix F.6, p. 29).

Q8: AI disclosure. The statement on p. 11 specifies Codex assistance with experimental design, preprocessing, manuscript revision and supplementary mathematical analysis, including Appendix F.4. It states that public reference fields were not AI-synthesized, qualitative/thematic analysis is inapplicable, and automated consistency checks were performed. Authors retain responsibility for the text, code, mathematical claims and results.

Thank you again for helping us clarify the evidence.

Thank you for the great effort put into running additional experiments and improving the manuscript's formatting and presentation. I appreciate the revisions and am happy to raise my presentation score accordingly, but some of my concerns remain.

Regarding the neural baselines, thank you for adding the PDEBench and AirfRANS comparisons. Although FNO and MLP achieve lower errors than the RFM variants in those tests, I do not think this is a primary reason to penalize the paper, as RFM represents a distinct paradigm with different trade-offs worth investigating.

However, my main remaining concern is that the new results appear to contradict the central takeaway of the original submission:

Reversal on New Benchmarks: Originally, the single wide baseline outperformed search across all 60 primary tasks. In the new experiments, this ordering reverses: search achieves lower errors than the wide baseline (0.0116 vs. 0.0198 and 0.265 vs. 0.309). While this is now a much more complete submission, the new experiments contradict the initial findings and leave the overall takeaway inconclusive. The extent of these additions substantially shifts the core thesis and scope of the original submission.

Could you please clarify:

  1. Did I understand correctly that search outperforms the wide baseline on the new experiments?

  2. Why does this reversal occur compared to the original 60 tasks? Do the authors have any insights into the regimes where one method would outperform the other?

AuthorsOpenReview ↗

Official Comment by Authors

Thank you for the careful follow-up and for recognizing the additional experiments and presentation improvements. Your concern is important. We agree that the new benchmarks rule out interpreting our original result as a universal preference for width. The defensible conclusion is conditional, and this is a substantive narrowing of the original takeaway.

1. Does search outperform the wide baseline on the new benchmarks?

Yes, under the originally reported periodic adapter and width ladder. The quoted errors are the reported benchmark medians. To examine the ordering more carefully, we completed a follow-up on the existing 936 test trajectories per PDE, with two paired RF seeds and three budgets. Table 43 (p. 43) reports geometric means of paired error ratios, rather than ratios of those medians. Values below one favor search.

PDEBudgetSearch/original wideSearch/integer-width wide
Advection0.3M2.3682.368
Advection1.2M1.7991.986
Advection4.8M0.6940.943
Burgers6M0.7730.966
Burgers24M0.9220.702
Burgers96M0.8691.001

The stronger wide control uses the largest affordable integer width under the same periodic cost ceiling and width cap. At the largest advection budget, the original wide policy used 38.2% of the ceiling, whereas the control uses 99.1%. Thus, part of the original search advantage reflects the discrete allocation convention. The remaining advection ratio is 0.943 [0.916, 0.972]. For Burgers at 96M, the ratio becomes 1.001 [0.990, 1.011], so we no longer claim an advantage over this stronger wide control. These are marginal 95% trajectory-bootstrap intervals. The corresponding globally Holm-adjusted two-sided sign-flip p-values are 0.00408 and 1.0, subject to the paired sign-symmetry assumption. We retain this null result.

2. Why does the ordering change, and which regimes favor each allocation?

We investigated three distinct factors instead of attributing the reversal to PDE dimension or periodicity alone.

First, the completed candidate population matters (Table 45, p. 43). We audited all 84,864 fitted states. At the largest periodic budgets, oracle/search-population error divided by original-wide error is 0.657 for advection and 0.799 for Burgers. Final selection multiplies these by 1.057 and 1.088. In the original 60-task ADR study, the corresponding factors were 9.255 and 1.0068. Thus, the observed difference already exists in the completed fitted populations. Improving final selection alone cannot repair the original populations. This diagnostic does not evaluate candidates that better intermediate promotion might have generated.

Second, the width-diversity balance depends on the feature law (Tables 46-47, p. 44). On 64 previously specified trajectories per PDE, we fitted four bank lineages at each diagnostic width and averaged all subset oracle minima exactly. For advection, comparing width 128 with four width-64 banks gives a net oracle ratio of 1.797 [1.625, 2.012] under the spectrum-conditioned law, but 0.929 [0.908, 0.948] under uniform spatial frequencies. The associated width penalties are 2.461 and 1.068, and diversity gains are 1.369 and 1.151. Both ratios satisfy penalty/gain = net ratio. These fit-only affordable oracle comparisons are diagnostics, not executable free-oracle policies. This width pair does not reconstruct the staged search population. Uniform features also substantially worsen absolute errors, so a more favorable relative allocation ratio does not imply a better solver.

Third, we checked iteration-cap sensitivity (Tables 48-49, p. 45). On the same 64-trajectory subset, all four policies' capped winners were fixed and continued. At 24M, the integer-wide/original-wide ratio changes only from 1.321 to 1.323, so the observed width nonmonotonicity persists in this diagnostic. Two states remain nonstationary and are retained. This extra computation is outside the original budget and does not establish an equal-budget converged policy.

These results support width when its error reduction outweighs the affordable diversity benefit, as in low-budget advection, and support search when competitive candidates can be generated and selected within the ceiling. Budget utilization, feature law and fitting behavior must be stated. Our tests reuse existing trajectories and provide no fresh independent confirmation or universal crossover predictor. We revised the abstract (p. 1), Sections 5.5 (pp. 9-10) and 6 (p. 10), and Appendix I (pp. 42-45). Thank you for prompting these concrete qualifications.

Thank you for the detailed answers throughout the rebuttal phase. I recognize the effort put into improving the manuscript. At the same time, the new findings substantially shifts the central conclusions and tells a different story. Rather than layering these additions into the initial work, I believe it would be much stronger if reorganized and rewritten for a future submission. For this reason, I am maintaining my overall rating of 2, though I sincerely appreciate the engagement throughout the review process.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.