- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a model that incorporates general physical structure into neural networks while learning system-specific physics from data. The imposed structure includes energy conservation, the form of dissipation, and how physical parameters such as charge and gravity enter the governing equations. The learned components include the energy function, dissipation rates, and constitutive relations. The authors evaluate their approach on three physical systems: an electromagnetic cavity, a particle-field simulation, and a shallow-water fluid. They experimentally demonstrate that the model can distinguish conservative from dissipative systems, identify nonlinear constitutive relations, extrapolate to unseen physical parameter values such as reversed charge and modified gravity using prescribed parameter dependencies, and maintain long-term stability over horizons up to 100 times the training horizon. The paper claims four main contributions. First, building on structure-preserving approaches such as Hamiltonian Neural Networks (HNNs), it integrates physical constraints on energy dissipation, extrapolation to unseen physical parameters, long-term stability, and robustness to disturbances within a unified framework. Second, it focuses on interventions on physical parameters (counterfactuals) and establishes conditions under which the model can correctly predict the resulting dynamics. Third, through ablation experiments that remove individual structural components, it investigates which components are responsible for each desirable property. For example, the authors show that long-term stability depends not only on the properties of the energy function but also on the consistency of energy exchange between particles and fields. Fourth, the paper introduces a mechanism for selecting among predefined candidate physical model structures using the Bayesian Information Criterion (BIC). It further applies conformal prediction to provide a finite-sample guarantee, under certain statistical assumptions, on the probability of retaining the true model within the candidate set.
Critical strengths
One of the major strengths of this paper is that it presents a clear research concept: incorporating known physical structures and parameter dependencies into a model while learning unknown constitutive relations and dissipative properties from data, with the aim of enabling extrapolation to physical conditions not observed during training. In particular, I believe that this research direction, which combines physical constraints with data-driven model inference to enable predictions beyond observed conditions, will be important for the future development of AI for Science aimed at scientific discovery.
Critical weaknesses
I understand that one of the main contributions of this paper is its ability to extrapolate to unseen physical parameter values by prescribing how these parameters enter the governing equations. The authors carefully discuss the non-identifiability and gauge freedoms of physical models in Appendices F.2 and F.5, demonstrating their awareness of these issues. However, it remains unclear to what extent such non-identifiability affects extrapolation predictions and under what conditions the uniqueness of these predictions can be guaranteed. The proposed framework incorporates additional physical constraints, such as energy conservation and the coupling structure between particles and fields, which may resolve these ambiguities. However, it is not clear which constraints contribute to the uniqueness of extrapolation predictions. Furthermore, Proposition 5 assumes that the learned functions g and h coincide with their true counterparts, but whether this assumption can be guaranteed from the training data and the imposed structural constraints is a separate question. In this context, a recent work(https://openreview.net/pdf?id=BDoxdoPBmO), present a framework for evaluating structural uncertainty in physical models based on the rank of the constraint operator. Could the authors clarify whether degrees of freedom that are unidentifiable from the training observations can also affect predictions at unseen parameter values? Possible approaches include analyzing the rank and null space of the observation sensitivity operator or comparing extrapolation predictions from multiple models that fit the training data equally well. Although the extrapolation experiments presented in this paper are interesting, I believe it is important to distinguish whether their success arises from the uniqueness of the extrapolated dynamics or from the selection of an appropriate solution under specific training conditions. This distinction is essential for assessing the reliability and generalizability of the proposed approach.
Questions for authors
Q1. Are the extrapolated dynamics uniquely identifiable from the training data?
Proposition 5 establishes the correctness of extrapolation under the assumption that the learned functions and coincide with their true counterparts. However, if the training data are restricted to a single parameter value , the transformation reproduces the same training dynamics but yields different predictions at unseen parameter values, provided that such a transformation is admissible under the model constraints. Could the authors clarify under what conditions the imposed physical structure guarantees the uniqueness of extrapolated dynamics, even when the individual functions and are not uniquely identifiable? In particular, which constraints resolve this ambiguity in the charge-reversal and gravity-intervention experiments? Methods for assessing identifiability before training, such as the rank-based framework proposed by a recent work (https://openreview.net/pdf?id=BDoxdoPBmO), may provide a useful reference.
Q2. Is the use of BIC as an approximation to the marginal likelihood justified in this setting?
The conventional BIC approximation relies on regularity conditions, including local parameter identifiability and a nonsingular Fisher information matrix. If non-identifiability remains under the proposed structural constraints, the resulting model may be singular, in which case the conventional BIC is not generally a valid asymptotic approximation to the marginal likelihood (Watanabe, 2013). Moreover, the conservative case with a zero dissipation rate lies on the boundary of the parameter space, potentially introducing additional regularity concerns. Could the authors clarify whether the candidate models satisfy the regularity assumptions underlying BIC? The paper explains that the neural network components are shared across candidate models and that the BIC penalty focuses on differences in structural hypotheses. How is this treatment of model complexity justified from the perspective of marginal likelihood approximation? Given the incorrect model selection observed in the conservative cavity experiment, have alternatives such as WBIC been considered?
Q3. What is fundamentally new compared with existing structure-preserving methods?
The main components of the proposed framework are based on existing methods, while Proposition 5 and the conformal prediction guarantee follow directly from established results. Compared with existing structure-preserving neural networks augmented with the same parameter dependencies and dissipation constraints, what new capabilities or methodological advances does the proposed approach provide?
Q4. How does the proposed method compare quantitatively with the closest existing structure-preserving methods?
The ablation studies and comparisons with unstructured models are carefully conducted. However, comparisons with closely related approaches, such as Dissipative HNNs, port-Hamiltonian neural networks, and GENERIC-based networks, remain largely conceptual. Could the authors provide quantitative comparisons under matched experimental conditions, particularly regarding long-term stability and extrapolation performance?
I thank the authors for their detailed rebuttal and for their revision of the manuscript.
Section 2.1 now relates the queries to Pearl's definition of counterfactuals and clarifies that, with deterministic simulators and a fully observed initial state, they coincide with interventions; the title and the terminology have been changed accordingly (W4, Q4). The setup has improved as well (Q1, Q2), as has slightly the presentation (W2, Q3).
My presentation concerns remain. Section 2.1 is a problem setup rather than a background section, and the authors adopt notions such as Poisson brackets, Casimirs, coercive energies, and the confining energy class without ever introducing or referencing them (W1). Since ICLR is a general-purpose ML conference, I would have expected at least references. Proposition 1 is still core to Sections 3.3 and 4.1, but stated only in the Appendix (W3). More generally, the main body is still hard to assess, as most results in Section 4 rely on the Appendix. For instance, none of the intervention results in Section 4.3 is reported in a table or figure in the main body; Table 3, cited there, is in App. C.4.
To my understanding of the experiments from a causal perspective, the tested interventions are always of the form do(λ=λ′), i.e., on a scalar that is constant in the training data. However, λ is injected into the model through the supplied map λgθ+hθ. Therefore, the model seems able to generalize to interventions "by design", i.e., transfers are explicitly modeled rather than learned. Learning how to learn to model an intervened value, e.g., from data over several values of λ, would be the interesting causal question, which is not addressed.