- Rating
- 3(+2)
- Confidence
- 3
- Informativeness
- 2(+1)
- Presentation
- 3
- Soundness
- 2(+1)
Read review
Paper summary
This paper studies a failure mode in automated physical-law discovery in which a discrepancy originating outside the physical laws, such as in the measurement apparatus, can instead be explained by modifying the inferred laws. The authors call this phenomenon "absorption."
The paper proposes a representation in which concepts, physical relations, observation models, parameters, and constraints are treated as separately editable components of a theory. Within this framework, the authors study when different repairs can produce observationally equivalent predictions.
For linear time-invariant systems, the paper characterizes conditions under which a law-only modification can reproduce the same input-output behavior as a modification of the experimental representation. The paper further argues that when two explanations have identical input-output behavior on the observed channel, changing the input cannot distinguish them, whereas measuring an additional quantity can. These ideas are demonstrated on several simulated systems and tested with existing law-discovery methods and external examples.
Critical strengths
-
The paper is clearly written and presents a coherent framework for separating different components of a scientific theory, including physical laws and the experimental/measurement interface.
-
I agree with the high-level message that a good fit to observations does not necessarily identify the physical origin of an effect. This is an important issue for automated scientific discovery systems, especially if such systems are expected to reason not only about equations but also about experiments and measurement processes.
-
The paper goes beyond a single illustrative example and tests the proposed phenomenon across multiple systems, different system structures, measurement noise, existing law-discovery methods, and an independently implemented physical system.
-
I found the experimental-design perspective useful: when two candidate explanations are observationally equivalent under the currently measured channel, additional excitation of the same input-output system may provide no new information, while observing a new quantity can distinguish the explanations.
Critical weaknesses
My main concern is that I find the central observation of the paper rather expected, and I am not yet convinced that the paper establishes sufficient novelty beyond classical system identification and identifiability considerations.
At a high level, if the measurement apparatus changes the mapping between the underlying physical state and the observed variables, it is natural that fitting equations directly to those observed variables can produce different inferred equations. Similarly, if two models produce exactly the same input-output map on the only measured channel, then no amount of changing the input can distinguish them through that channel. One needs to observe a quantity on which the two models make different predictions. To me, these conclusions follow quite directly from the basic setup.
The mass-spring example makes this issue particularly clear. A first-order lag introduced by the sensor can instead be represented as an additional dynamical process inside the inferred system. From a dynamical-systems or system-identification perspective, this seems like a fairly standard ambiguity between the internal realization of a system and the measurement process. The fact that the same observed transfer function can admit different internal physical interpretations therefore seems expected rather than surprising.
Relatedly, the main linear-system characterization appears closely connected to classical realization theory. Two of the central conditions essentially state that the direct feedthrough must match and that the model must contain enough states to realize the observed transfer function. I would like the paper to much more clearly identify which part of the theoretical result is fundamentally new relative to classical realization and structural identifiability theory, rather than a reformulation of these ideas in the language of automated law discovery.
I think the potentially more novel contribution is not the observation that measurement errors can be absorbed into an inferred dynamical law, but the proposal that an automated scientific-discovery system should jointly search over laws, observation models, experimental roles, and other components of a theory. However, this distinction is not sufficiently sharp in the current framing. The paper often presents "absorption" itself as the central discovery, whereas I currently see the unified repair space and its implications for AI scientific-discovery systems as potentially the more interesting contribution.
A second concern is that I am not sure who the intended audience of the paper is. For readers from dynamical systems, control, or system identification, much of the central phenomenon may already be familiar through realization and identifiability. For physicists, the fact that an incorrect measurement model can lead to an incorrect inferred physical law also seems conceptually straightforward. For an AI/ML audience, the paper would benefit from a clearer demonstration that this issue creates a concrete failure mode for current AI scientific-discovery systems or benchmarks that is not already captured by existing notions of identifiability.
As a result, I currently find it difficult to identify the precise scientific gap that this paper fills. The paper combines ideas from scientific discovery, system identification, experimental design, and theory revision, but the central contribution relative to each of these communities needs to be stated much more sharply.
Questions for authors
-
What is the main conceptual novelty of "absorption" relative to classical realization theory and structural identifiability? If an observed input-output map admits multiple internal realizations, or if the observation model itself is uncertain, it seems expected that the physical origin of an observed effect may not be identifiable. Which part of the phenomenon demonstrated here could not already be described using existing system-identification language?
-
Relatedly, which part of Proposition 1 should be viewed as a new theoretical result? The direct-feedthrough and minimum-state-dimension conditions appear closely related to standard realization theory. Is the main novelty the physical typing constraint, or is there another aspect of the proposition that goes beyond classical results?
-
What is the intended primary audience and use case of the paper? Is the main contribution intended for machine learning and automated scientific discovery, for dynamical systems/system identification, or as a more general conceptual point about scientific inference? I think clarifying this would also help clarify the appropriate novelty standard.
-
If the intended audience is the AI-for-science / automated-discovery community, can the authors demonstrate a concrete case in which a state-of-the-art scientific-discovery agent reaches a scientifically misleading conclusion because of this issue, and where allowing the agent to revise the observation model changes the scientific conclusion? Such an example would make the practical importance of the framework much clearer.
-
What is fundamentally gained by introducing the new term "absorption"? Is it describing a phenomenon that is not already naturally characterized as observational equivalence, non-identifiability, or realization ambiguity? A clearer comparison between these concepts would help establish the need for the new terminology.
-
I found the unified repair-space formulation potentially more interesting than the basic observation about measurement errors. Would the authors consider reframing the main contribution around automated theory revision in which the experimental representation itself is editable, rather than around the claim that measurement effects can be absorbed into inferred laws?
I'd love to thank the authors for their detailed feedback, which addressed most of my concerns. Therefore, I'm raising my score.