acceptodds
Withdrawn from ICLR 2027

When Does a Predictable Residual Become a Scientific Correction? A Residual-Transport Audit for AI-Assisted Scientific Discovery

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Automated scientific-discovery systems can make the residuals of established models predictable, but predictive gain alone does not identify what has been found: a missing scientific contribution, a regime-specific offset, or structured noise. We formulate this downstream problem as correction acceptance and introduce a residual-transport audit that evaluates frozen corrections across scientifically meaningful boundaries. The central observation is that held-out-regime performance mixes two forms of evidence: improvement relative to information available before the regime is observed, and explanation of variation within that regime after its own residual level is accounted for. We derive the resulting decomposition and build it into a leakage-safe audit with frozen candidate selection, regime-level replication, uncertainty, and a measurement-noise stopping rule. Across thermodynamics, nuclear physics, mammalian metabolism, cosmology, and a supplementary asset-pricing control, the audit separates corrections that transport from patterns that merely locate regimes. A known equation-of-state correction transports across unseen fluids (R^2_pool=0.993), and a simple correction to Geiger-Nuttall predicts nuclides discovered decades later (R^2=0.635). In contrast, mammalian temperature is repeatedly selected yet fails within unseen taxonomic orders, while COBE/FIRAS requires no further correction. The contribution is an evaluation layer for AI-assisted science: candidate generation may be flexible, but scientific acceptance should be indexed by the environments in which the corrected model is expected to hold.

Primary area:
datasets and benchmarks
Keywords:
correction acceptance, model discrepancy, out-of-distribution generalization, evaluation methodology, scientific machine learning, symbolic regression, within-between decomposition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RDQEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces correction acceptance as a distinct evaluation task in AI-assisted scientific discovery, arguing that improved residual prediction alone is insufficient to justify modifying an established scientific model. The authors propose a residual-transport audit that evaluates whether candidate corrections generalize across scientifically meaningful regimes. The framework distinguishes prospective predictive improvement from within-regime performance through a decomposition of residual variance, and incorporates frozen candidate selection, stability assessment, uncertainty estimation, and a measurement-noise stopping rule.

The framework is evaluated across several scientific domains, including thermodynamics, nuclear alpha decay, mammalian metabolism, and cosmology, with an additional asset-pricing control. The results show that some known corrections generalize across unseen regimes, while others improve overall prediction without meeting the proposed within-regime criterion. The paper aims to establish a conceptual and operational framework for assessing when predictive residual structure provides sufficient evidence to support a scientific model correction.

Critical strengths
  • The paper provides a useful formal analysis of a potential limitation in conventional out-of-distribution evaluation. The proposed offset decomposition illustrates how positive predictive performance on unseen regimes can be influenced by between-regime differences, even when the correction does not outperform a regime-specific mean baseline. This distinction is mathematically explicit and supported by empirical examples, providing a useful perspective on how predictive improvements should be interpreted when evaluating scientific model corrections.

  • The paper provides a broad empirical evaluation covering diverse scientific domains and validation scenarios. The experiments span thermodynamics, nuclear physics, mammalian metabolism, and cosmology, with an additional asset-pricing control. Importantly, they examine different aspects of scientific correction acceptance, including the recovery of known corrections, generalization across different scientific boundaries, failures of within-regime transport, and cases where further correction may be unnecessary due to measurement noise. This breadth helps illustrate the relevance and applicability of the proposed framework across different scientific settings.

Critical weaknesses
  • The conceptual novelty and necessity of the proposed framework are not sufficiently established. The distinction between candidate generation and scientific verification has been discussed in prior work (Cornelio et al., 2026). Similarly, existing studies have emphasized purpose-specific scientific model validation (Jakeman et al., 2026) and structured external validation for assessing model generalizability (Ho et al., 2020). While framing correction acceptance as a separate task is interesting, the paper does not clearly establish what conceptual advance this framing offers beyond these existing approaches, or why a separate audit framework is necessary. A more explicit discussion of these closely related works would help clarify the paper's conceptual novelty and contribution.

  • The experiments provide limited evidence for the effectiveness of the proposed audit framework. Although the evaluation covers several scientific domains, the experiments mainly demonstrate that different evaluation criteria can lead to different acceptance decisions. They do not systematically establish whether these decisions are more reliable or scientifically justified than those obtained using existing validation approaches. Furthermore, the within-regime R2R^2 retains errors in predicting the regime mean, so negative scores do not necessarily indicate a failure to capture within-regime variation. This complicates the interpretation of some key rejection results. More direct comparisons and a clearer validation of the audit decisions would strengthen the empirical support for the proposed framework.

References

Cornelio, C., Ito, T., Cory-Wright, R., Dash, S., & Horesh, L. (2026). The need for verification in artificial intelligence-driven scientific discovery. Philosophical Transactions of the Royal Society A, 384, 20240591.

Jakeman, J. D., Barba, L. A., Martins, J. R. R. A., & O'Leary-Roseberry, T. (2026). Verification and validation for trustworthy scientific machine learning. Machine Learning: Science and Technology, 7, 025055.

Ho, S. Y., Phua, K., Wong, L., & Goh, W. W. B. (2020). Extensions of the external validation for checking learned model interpretability and generalizability. Patterns, 1(8), 100129.

Questions for authors
  1. What does the proposed correction acceptance framework offer beyond existing scientific verification and purpose-specific validation frameworks, such as Cornelio et al. (2026) and Jakeman et al. (2026)? Can the authors identify a concrete limitation of these existing approaches that the proposed framework addresses?

  2. How can the reliability of the proposed audit decisions be established independently of the audit criteria themselves? Is there evidence that these decisions are more scientifically justified than those obtained using existing validation approaches, rather than simply being different?

  3. How should a negative within-regime R2R^2 be interpreted when EgE_g includes errors in predicting the regime mean? Could the authors distinguish failures to capture within-regime variation from failures to predict the regime-level offset, particularly in the nuclear decay and mammalian metabolism experiments?

  4. In the leave-one-element-out nuclear decay experiment, the direct correction uses ZZ, which is constant within each element. Since this makes a non-positive within-element R2R^2 mathematically inevitable, what additional evidence does this experiment provide for the proposed audit framework beyond this structural limitation of the candidate correction?

Reviewer_nqrqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies correction acceptance problem for AI-assisted scientific discovery. It proposes a residual-transport audit that distinguishes prospective predictive improvement from performance relative to the held-out regime's own residual mean. The framework combines this distinction with regime-level validation, stability selection, uncertainty estimation, and a measurement-noise stopping rule.

Critical strengths

1, The paper addresses a meaningful problem beyond conventional equation fitting.

2, The empirical examples illustrate why the evaluation boundary matters.

3, The evaluation protocol is relatively careful and transparent.

Critical weaknesses
  1. Insufficient justification for the acceptance criterion. The proposed within-regime (R^2) combines within-regime prediction errors with regime-level calibration errors. Thus, a negative score does not necessarily indicate a failure to capture transferable scientific relationships. Moreover, a correction depending only on atomic number (Z) necessarily produces non-positive within-element (R^2), raising concerns about the interpretation of the nuclear experiment.

  2. Limited methodological novelty and insufficient validation. The core decomposition is based on established statistical techniques, and the within-regime (R^2) is equivalent to conventional (R^2) on held-out groups. The experiments do not sufficiently demonstrate whether the proposed acceptance criterion offers meaningful advantages over simpler group-aware evaluation criteria in identifying transferable scientific corrections.

  3. Limited evaluation with symbolic regression methods. The experiments primarily use predefined physical terms or restricted nonlinear feature libraries rather than modern symbolic regression methods. Additionally, the robustness of the audit to different correction representations, such as multiplicative, relative, or logarithmic forms, remains unclear.

Questions for authors

See Weaknesses.

Reviewer_gxKTOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the post-generation evaluation of candidate model modifications in AI-assisted scientific discovery, asking whether predictable residuals represent missing physical laws, regime-specific offsets, or measurement noise. The authors formulate this task as "correction acceptance" and propose a leakage-safe "residual-transport audit". Their central observation is that standard held-out regime R2R^2 conflates between-regime mean offsets (BgB_g) with within-regime explanatory variance (WgW_g), allowing a frozen correction to score positively on an unseen regime simply by predicting its group intercept while failing to capture internal dynamics. Pairing this decomposition with stability selection, bootstrap uncertainty, and a noise-floor stopping rule, the authors evaluate their protocol across five classic datasets in thermodynamics, nuclear physics, metabolism, cosmology, and finance.

Critical strengths
  1. Useful observation: Decoupling candidate generation from boundary-indexed acceptance exposes a critical blind spot in AI4Science: even strict leave-one-regime-out cross-validation yields false positives when models merely predict between-regime offsets
  2. Clean Empirical Counterexamples: The metabolic analysis across three independent datasets cleanly demonstrates that a feature can have a statistically significant within-order association (p=0.034p=0.034) and positive pooled R2R^2, yet still fail to predict variation inside unseen taxonomic orders.
Critical weaknesses
  1. Technical contribution not clear and mathematically trivial: The core theoretical contribution (Proposition 1 and Corollary 1) is an elementary sum-of-squares identity (TSS=Wg+Bg\text{TSS} = W_g + B_g, standard in basic ANOVA and Mundlak decompositions) repackaged as formal proposition.
  2. Disconnect from modern AI4Science: Despite motivating the paper around symbolic regression, SINDy, and PINNs, the experiments evaluate zero contemporary AI discovery systems, relying entirely on hand-picked features or basic polynomial expansions with Lasso.
  3. Over-engineered toy benchmarks: All five experiments use tiny tabular datasets (mostly N<500N < 500) to recover textbook historical facts. Worse, the EOS benchmark fits noiseless computed points queried from NIST reference formulas rather than raw experimental data, and the nuclear leave-one-element-out test evaluates a direct ZZ correction within single elements where atomic number ZZ is constant by definition—guaranteeing its failure on within-element variance by construction.
Questions for authors
  1. Why were contemporary equation-discovery algorithms (e.g., PySR, AI-Feynman, SINDy) omitted in favor of polynomial Lasso? Can you demonstrate your audit on the Pareto frontier of candidate equations generated by an actual symbolic regression system on a non-trivial benchmark?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.