acceptodds
Under review as a conference paper at ICLR 2027

Not Every Defect Is in the Law

Abstract

Systems for discovering physical laws usually search over equations while fixing the experimental representation: what quantities exist, what instruments measure, and what experimenters control. We show that defects can instead lie elsewhere in the theory, and that law search can hide this distinction even with a perfect fit. We call this failure mode absorption: a discrepancy originating in the experimental representation is reproduced by rewritten physical laws. In a mass–spring system with a delayed sensor, two law edits reproduce the recorded channel to \(5.1\times10^{-16}\) relative error while leaving the incorrect measurement model unchanged. For linear systems, absorption occurs when the theory matches the apparatus's direct input–output feedthrough, has enough states to realize the observed dynamics, and can express those dynamics with its declared dimensions and parameters. Across 100 drawn linear systems, law-only accounts exist in all 100 cases. Additional drives resolve none of these ambiguities, whereas an added measurement resolves every case; independently constructed systems and established law-discovery methods show the same qualitative behavior. These results explain the title's central claim: . A perfect law fit can recover observable behavior without identifying its physical origin, making the choice of what to measure part of scientific discovery itself.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
scientific discovery, symbolic regression, model identifiability, experimental design, program repair, typed representations

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.25 over 4 reviews, above 96% of the others.
Reviewer_LV4MOpenReview ↗
Rating
3(+2)
Confidence
3
Informativeness
2(+1)
Presentation
3
Soundness
2(+1)
Read review
Paper summary

This paper studies a failure mode in automated physical-law discovery in which a discrepancy originating outside the physical laws, such as in the measurement apparatus, can instead be explained by modifying the inferred laws. The authors call this phenomenon "absorption."

The paper proposes a representation in which concepts, physical relations, observation models, parameters, and constraints are treated as separately editable components of a theory. Within this framework, the authors study when different repairs can produce observationally equivalent predictions.

For linear time-invariant systems, the paper characterizes conditions under which a law-only modification can reproduce the same input-output behavior as a modification of the experimental representation. The paper further argues that when two explanations have identical input-output behavior on the observed channel, changing the input cannot distinguish them, whereas measuring an additional quantity can. These ideas are demonstrated on several simulated systems and tested with existing law-discovery methods and external examples.

Critical strengths
  1. The paper is clearly written and presents a coherent framework for separating different components of a scientific theory, including physical laws and the experimental/measurement interface.

  2. I agree with the high-level message that a good fit to observations does not necessarily identify the physical origin of an effect. This is an important issue for automated scientific discovery systems, especially if such systems are expected to reason not only about equations but also about experiments and measurement processes.

  3. The paper goes beyond a single illustrative example and tests the proposed phenomenon across multiple systems, different system structures, measurement noise, existing law-discovery methods, and an independently implemented physical system.

  4. I found the experimental-design perspective useful: when two candidate explanations are observationally equivalent under the currently measured channel, additional excitation of the same input-output system may provide no new information, while observing a new quantity can distinguish the explanations.

Critical weaknesses

My main concern is that I find the central observation of the paper rather expected, and I am not yet convinced that the paper establishes sufficient novelty beyond classical system identification and identifiability considerations.

At a high level, if the measurement apparatus changes the mapping between the underlying physical state and the observed variables, it is natural that fitting equations directly to those observed variables can produce different inferred equations. Similarly, if two models produce exactly the same input-output map on the only measured channel, then no amount of changing the input can distinguish them through that channel. One needs to observe a quantity on which the two models make different predictions. To me, these conclusions follow quite directly from the basic setup.

The mass-spring example makes this issue particularly clear. A first-order lag introduced by the sensor can instead be represented as an additional dynamical process inside the inferred system. From a dynamical-systems or system-identification perspective, this seems like a fairly standard ambiguity between the internal realization of a system and the measurement process. The fact that the same observed transfer function can admit different internal physical interpretations therefore seems expected rather than surprising.

Relatedly, the main linear-system characterization appears closely connected to classical realization theory. Two of the central conditions essentially state that the direct feedthrough must match and that the model must contain enough states to realize the observed transfer function. I would like the paper to much more clearly identify which part of the theoretical result is fundamentally new relative to classical realization and structural identifiability theory, rather than a reformulation of these ideas in the language of automated law discovery.

I think the potentially more novel contribution is not the observation that measurement errors can be absorbed into an inferred dynamical law, but the proposal that an automated scientific-discovery system should jointly search over laws, observation models, experimental roles, and other components of a theory. However, this distinction is not sufficiently sharp in the current framing. The paper often presents "absorption" itself as the central discovery, whereas I currently see the unified repair space and its implications for AI scientific-discovery systems as potentially the more interesting contribution.

A second concern is that I am not sure who the intended audience of the paper is. For readers from dynamical systems, control, or system identification, much of the central phenomenon may already be familiar through realization and identifiability. For physicists, the fact that an incorrect measurement model can lead to an incorrect inferred physical law also seems conceptually straightforward. For an AI/ML audience, the paper would benefit from a clearer demonstration that this issue creates a concrete failure mode for current AI scientific-discovery systems or benchmarks that is not already captured by existing notions of identifiability.

As a result, I currently find it difficult to identify the precise scientific gap that this paper fills. The paper combines ideas from scientific discovery, system identification, experimental design, and theory revision, but the central contribution relative to each of these communities needs to be stated much more sharply.

Questions for authors
  1. What is the main conceptual novelty of "absorption" relative to classical realization theory and structural identifiability? If an observed input-output map admits multiple internal realizations, or if the observation model itself is uncertain, it seems expected that the physical origin of an observed effect may not be identifiable. Which part of the phenomenon demonstrated here could not already be described using existing system-identification language?

  2. Relatedly, which part of Proposition 1 should be viewed as a new theoretical result? The direct-feedthrough and minimum-state-dimension conditions appear closely related to standard realization theory. Is the main novelty the physical typing constraint, or is there another aspect of the proposition that goes beyond classical results?

  3. What is the intended primary audience and use case of the paper? Is the main contribution intended for machine learning and automated scientific discovery, for dynamical systems/system identification, or as a more general conceptual point about scientific inference? I think clarifying this would also help clarify the appropriate novelty standard.

  4. If the intended audience is the AI-for-science / automated-discovery community, can the authors demonstrate a concrete case in which a state-of-the-art scientific-discovery agent reaches a scientifically misleading conclusion because of this issue, and where allowing the agent to revise the observation model changes the scientific conclusion? Such an example would make the practical importance of the framework much clearer.

  5. What is fundamentally gained by introducing the new term "absorption"? Is it describing a phenomenon that is not already naturally characterized as observational equivalence, non-identifiability, or realization ambiguity? A clearer comparison between these concepts would help establish the need for the new terminology.

  6. I found the unified repair-space formulation potentially more interesting than the basic observation about measurement errors. Would the authors consider reframing the main contribution around automated theory revision in which the experimental representation itself is editable, rather than around the claim that measurement effects can be absorbed into inferred laws?

1 comment

I'd love to thank the authors for their detailed feedback, which addressed most of my concerns. Therefore, I'm raising my score.

Reviewer_MnrKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper problematizes common approaches in physical law discovery approaches, making the point that phenomena in measurements can be attributed either to the mathematical law that describes observations or to the model of the measurement apparatus itself. They verify in three model systems that changing a measurement model can yield data that can be fit by editing the laws governing the physical system. Therefore, not all measurements determine the physical law, if the measurement model is not editable. Further along this point, the authors show that adding measurements can disambiguate physical theories that previously agreed. They prove an exact condition for identifiability in linear systems and demonstrate that existing physical law discovery platforms are vulnerable to misidentifying the true underlying system if their measurement models are fixed.

Critical strengths

This paper is exceptionally well written, well-motivated, and easy to follow. The mass spring system is a very useful anchor for the reader as they make their way through the concepts. The special case of the linear system adequately illustrates the point and the examples in more realistic scenarios also support the conclusions of the paper.

Critical weaknesses

My main concern for this paper is its novelty. The central point of the paper is important, but it is well known to a control theory audience. It is not surprising that enlarging the search space over which we define a physical law (i.e. including changes to the measurement model) can lead to degenerate descriptions of the same system. This idea is well understood in linear systems, as well as some nonlinear systems, and the novel theoretical contribution (including typed variables) in this area is quite small. However, viewed as a position paper, this work is useful to orient a machine learning audience that is not familiar with the existing literature, so I vote for a weak accept if the area chair deems its contributions novel enough.

Questions for authors
  1. It seems that the randomly drawn experiment in the linear system is designed to prove the point and thus cannot possibly fail. It would be useful to think of an experiment that demonstrates the frequency of absorption for the linear case that has a nonzero failure rate.
  2. This may be beyond the scope of this work, but it would be insightful to comment on methods to find informative new measurements that disambiguate candidate theories.
Reviewer_grAxOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper argues that when observations disagree with model predictions, revising only the physical law and leaving the experimental representation unchanged is problematic. For linear time-invariant systems, it gives a theoretical characterization of when errors in the observation model can be absorbed by changes to the physical law, so that different physical explanations produce identical observations. It then proposes a broader theory repair framework that also modifies components other than the physical model, and tests it on several synthetic experiments.

Critical strengths
  • For linear time-invariant systems, the paper gives a theoretical characterization of when observation-model errors can be absorbed by changes to the physical law, so that different physical explanations produce identical observations.
  • The paper proposes a systematic framework for repairing the theory as a whole. It treats as repairable several components that prior law-discovery work usually holds fixed, such as the observation model, variable roles, and admissible ranges.
  • The paper shows, theoretically and empirically, that changing input conditions alone may not separate observationally equivalent theories. This highlights the importance of introducing new observed variables.
Critical weaknesses
  • The paper gives too little justification for why minimizing syntactic or structural edit distance should select the physically correct repair. Robustness across several cost metrics is examined, but cases where the true repair costs more than an incorrect one are not analyzed.

  • Widening the set of repair targets makes it more likely that the correct repair is found, but it can also add incorrect candidates that fit the observed data. The effect of this widening on final repair accuracy and on the rate of incorrect selections is not systematically evaluated.

  • The paper systematically derives 21 repair operators but implements only 16, and expressions are restricted to a specific grammar. It is unclear how well the proposed search space covers the repairs that real scientific discovery requires.

  • The main experiments use synthetic data in which a single artificial defect is introduced into a known low-dimensional ODE. There are also evaluations on an external benchmark and on real data, but they do not show convincingly that the method can identify the correct repair in real scientific problems.

Questions for authors

Q1. Does expanding the search space actually improve repair accuracy? The paper’s central proposal is to make components beyond the physical law repairable, but a larger search space may also produce more incorrect candidates. How do the recovery rate of the true repair and the rate of selecting an incorrect repair change as the repair targets grow from (R) alone, to (R+O), to all components? Without this comparison, the practical benefit of a larger search space is not demonstrated.

Q2. Why should edit-cost-based selection favor physically correct repairs? The method ranks candidates by structural edit distance, but a smaller change does not necessarily mean a physically correct one. How does the method handle cases where the true repair costs more than an incorrect one? Have selection accuracy in such cases, and the method’s ability to report the problem as non-identifiable, been evaluated?

Q3. How effective is a search limited to two edits for general theory repair? The algorithm relies on symbolic candidate enumeration and allows at most two edits. How do search performance and computational cost change when three or more edits, or simultaneous changes to several components, are required? Could a comparison with more efficient search methods be added to show that the proposed candidate generation algorithm itself is effective?

Q4. Can the method be shown to identify correct repairs in real scientific problems? The main experiments use low-dimensional synthetic systems with a single artificial defect, and the real-data experiments do not verify that the correct physical explanation is recovered. Could the authors show that the method finds the correct repair in more realistic physical systems with multiple defects or an incomplete model?

Reviewer_ZDbvOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper addresses a much overlooked aspect of symbolic equation discovery from data, namely that aspects of the experiment/experimental apparatus might be influencing the results, not necessarily an incorrect physical law. The paper proposes a five parameter theory formalization framework T = (C, R, O, Theta, A) that incorporates the experimental apparatus and includes what the authors call "type repair operations". They show that with one law edit a defective component is recoverable, but with two law edits an instrument lag can be “absorbed” into rewritten dynamics. This notion of "absorption" is formally defined by the authors - it is when an effect originating outside of the physical laws can be reproduced exactly by rewriting the laws. They provide a set of three structural conditions for absorption in linear systems and show that only an added measurement channel, not a new drive, separates the accounts. They also provide a set of empirical results that largely confirm the validity of the main proposition and constructions. Additionally, they do a certain amount of external validation using established symbolic discovery systems and existing benchmarks.

Critical strengths

I am actually going to mention three important strengths of the paper, namely:

Clear And Important Thesis. Symbolic physical law discovery systems usually treat the equations as uncertain and the experimental setup as fixed. The paper shows that a perfect fit can then place an effect in the wrong part of the theory. The mass–spring example with a lagging sensor is a beautiful example.

Elegant Theoretical Framework. Treating a theory as a five component system, T = (C, R, O, Θ, A), with components: concepts (C), relations (R), observation model (O), parameter (Theta), and constraints (C), with typed repair operations puts law edits/rewrites and the observation-model, domain and role edits in one search space.

Vivid Result for Linear Systems.

Critical weaknesses

While I found the paper to overall be of considerable value, with the chief contributions being the overall thesis and theoretical framework, the theoretical and empirical contributions seemed to me to be more modest.

Limited novelty of theoretical contribution. While the key result for linear systems (Proposition 1) is elegant, conditions (i) and (ii) are acknowledged to follow from classical realization theory, so only condition (iii), i.e., the dimensional-writability test, but the check is only sufficient and not necessary, and can return “undecided.” Also, I don't quite understand why the construction reproduces the recording on just one of the 32 systems where the predicate says “no.”

Artificial experimental results/findings. In the absorbed cases the cheaper (correct) account always wins, so the paper never shows a case where a cost-based preference would pick the wrong explanation. The absorbing account (a spring term depending on a rescaled sensor state,) seemed very strange to me, and I can’t imagine an actual scientist accepting it without additional data. The paper does not discuss which priors or constraints would rule it out in practice.

Questions for authors

I am on the fence between a weak accept and strong accept so the answers to these questions are especially important.

Q1: Proposition 1 says a law-only account reproduces the apparatus if and only if three conditions hold: the feedthrough matches (D* = e), the theory declares enough states (n ≥ d), and the required dynamics are writable with the declared dimensions and parameters. In Appendix N you report that your constructive procedure reproduces the recording on 16 of the 16 systems where the predicate says “yes,” but also on 1 of the 32 systems where it says “no". Appendix P shows this in the “cut” variant, where the instrument state is omitted and condition (ii) is predicted to fail (built|no = 1/9). It seems to me that an exact “if and only if” should rule out such a case.

Q2: In every absorbed case you test, the single-edit sensor repair is cheaper than the law-only account, and you report no equal-cost ambiguous pairs. In Section 4.1 you also note that choosing the cheaper account “reflects a preference built into the representation or objective, not information contained in the recording.” As things stand, the experiments therefore do not show a practical failure of a simple cost-based preference. Can you give an example, in simulation or real data, where the cheaper account under your cost measures is the wrong one?

Q3 (very speculative): I wonder if the work described is sophisticated enough to be used to try and resolve the experimental differences in the two approaches used to measure the neutron lifetime – a classical unsolved experimental problem in physics.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.