acceptodds
Under review as a conference paper at ICLR 2027

Discovering Common Conservation Laws Across Gradient-Flow Environments

Abstract

Common conservation laws remain invariant across a family of neural-network training problems. Tangency on finitely sampled fields is ambiguous: nearly parallel fields create false near-invariants, while functionally independent candidate rows can exceed the normal-space dimension available to common laws. We formulate this finite identification problem as *discover–certify–preserve*. Multi-environment neural deflation (MEND) jointly learns a vector-valued candidate from training fields. A held-out certificate then tests field tangency, pointwise rank, first-bracket compatibility, and codimension capacity on frozen candidates. Conditional matrix bounds connect residuals to normal-space error when candidate and direction singular margins, derivative accuracy, and support coverage are resolved. On shallow-network controls, the full certificate resolves all 40 predeclared boundary/null decisions on fresh tasks; matched stages show that brackets reject task-local candidates and capacity rejects the remaining overcounts. Preservation is a separate deployment question. The learned-law Conservation-Constrained Adam evaluation fails its formal utility gate because of pathwise conditioning events, whereas supplied exact attention laws are preserved at Adam-comparable loss. The evidence supports a finite, boundary-conditioned certification framework. Blind law-count recovery, nonquadratic discovery, and robust learned-law optimizer utility remain open evaluation problems.

Primary area:
learning theory
Keywords:
Neural conservation laws, Finite identification, Tangency certification, Neural-network deflation, Conservation-constrained optimization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_NcNUOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies how to discover, certify and preserve conservation laws shared across gradient-flow environments, and design a workflow, combining MEND for discovery, a set of checks for certification, and CCA for preservation. The workflow is tested on a simple network, while the downstream evaluation using learned laws does not meet its overall success criterion.

Critical strengths
  1. The motivation and method are clear. The authors systematically analyze several failure cases in discovering and validating conservation laws and design checks to address them.
  2. The authors provides detailed ablation experiment and explaination for their workflow, and report both positive results and limitations.
Critical weaknesses
  1. The empirical validation of the certificate is limited by experiment scopes. The experiments use a very small network with known quadratic conservation laws and focus on predefined law-count boundaries. It will be better to show whether the certificate would be transferred to unknown law families, larger models, or candidates produced by other discovery methods.
  2. The practical value of certified laws seems not very much. The paper explains well why each test is needed to avoid false or task-local laws, but not why obtaining certified common laws is useful in practice. The only downstream use with learned laws, constraining Adam with CCA, fails its own evaluation criterion due to conditioning issues along the optimizer path, and the successful preservation experiment uses analytic laws in a separate attention setting.
Questions for authors
  1. What will happen on candidates produced by other conservation laws or discovery methods? Do the existing thresholds in Eq.15 keep reliable without retuning, and how sensitive are the certification decisions to these thresholds?
  2. Is there a concrete downstream setting in which learning and certifying common laws is useful when analytical laws are unavailable or difficult to derive? Beyond reducing conservation law drift, is there evidence of an optimization benefit from using the certified learned laws?
Reviewer_ekjbOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
1
Soundness
2
Read review
Paper summary

This paper introduces a framework (MEND) for learning and certifying common conservation laws in neural networks. In this context, a conservation law is a function of the model's parameters that remains constant as the weights evolve under gradient flow evolution. Given a pre-specified KK, a small, separate model is trained to predict a KK-dimensional conservation law Cϕ:Rd→RKC_\phi:\mathbb R^d\to\mathbb R^K where dd is the total number of trained parameters in the model. Conservation laws CϕC_\phi which hold across a family of vector fields (here, different sets of training data) are called common conservation laws. The paper introduces a discover-certify-preserve method for such conversation laws:

  1. Discover: Learn common conservation laws CϕC_\phi through gradient descent training. These conservation laws hold approcimately up to some training tolerance.
  2. Certify: Build a set of 4 certification criteria that together ensure the KK learned outputs represent genuinely distinct conservation laws.
  3. Preserve: Introduce a projection operation into Adam optimization that preserves those learned conservation laws. The conversation laws relate to gradient flow but are enforced during Adam optimization.

The core contribution seems to be the certification step. The authors design 4 criteria that ensure that the KK learned conservation laws are distinct and meaningful. For example, near-constant conservation laws are excluded, as are laws with highly correlated gradients. The 4 criteria are motivated with examples. In a 12-parameter toy MLP setting, the paper verifies that the discover-certify step successfully recovers an (a priori known) boundary of KK distinct conservation laws; the experiment on the preserve step with MLPs seems to be non-conclusive. In a separate experiment, using an attention model and analytically derived conversation laws, the paper shows that the modified Adam (CCA) preserves conservation laws during training without drop in performance.

Critical strengths

The validation framework is derived from mathematical principles and systematically validated in the MLP setup. The authors illustrate the different criteria using examples, and controlled MLP experiments illustrate the roles of all four criteria in detecting different kinds of redundant or inconsequential conservation laws.

Critical weaknesses

First, the empirical validation of the discover-certify-preserve mechanism is relatively limited. The authors use a toy MLP setting with 12 parameters to validate the discover and certify steps of the mechanism when conservation laws are already know a priori. Whether the discover and certification steps also generalize to setups where the conservation laws can't be derived analytically remains open. The preserve step is independently validated in an attention model using analytical conservation laws, and the discover-certify steps are not validated on this model. Thus, there is no complete, conclusive evaluation of the discover-certify-preserve mechanism.

Second, the motivation for the preserve step in the mechanism is not sufficiently clear. The paper treats conservation law preservation as an objective in itself, without establishing its practical value. Gradient flow and Adam are two fundamentally different optimizers, and I don't see an obvious reason that violating a gradient-flow conservation law is undesirable. The paper should establish the motivation for this essential step in their proposed discover-certify-preserve mechanism more clearly.

Third, I found the paper hard to read. The authors use overly complicated language with strong technical jargon. Important concepts and notation are often introduced without clear definition and their meaning has to be inferred by the reader. This makes it unnecessarily difficult to understand the methodology and interpret the results. Some examples I want to highlight include

  • The beginning of the abstract is already challenging to understand for someone who is not familiar with the paper:

Common conservation laws remain invariant across a family of neural-network training problems. Tangency on finitely sampled fields is ambiguous: nearly par- allel fields create false near-invariants, while functionally independent candidate rows can exceed the normal-space dimension available to common laws.

  • In the most technical Section 2.2, mathematical notation is partly unspecified and it's meaning has to be induced from context, e.g., DE(∞),BV,ρfld,ρbr\mathcal D^{(\infty)}_{\mathcal E}, \mathcal B_V, \rho_{fld}, \rho_{br}.

  • At the beginning of Section 3.2, the empirical findings are expressed with complicated language that is hard to follow:

The prespecified v15 learned-law deployment gate has formal status FAIL. It consumes all 20 cer- tified positive-control candidates in three new realizable tasks per model and compares five paired optimizer arms. The gate requires every arm and all 20 model-level clusters. [...] The prespecified estimand also includes method-triggered safeguard events, which makes pathwise rank maintenance an additional deployment requirement.

Questions for authors
  1. What limitations to the discover-certify method could prevent application to more complex models? In the paper, you only evaluate it on the 12-parameter MLP. Could the same method also be applied to the attention model?
  2. What numerical instabilities did you encounter when applying the modified Adam (CCA) training using learned preservation laws on the 12-parameter MLP? What are the implications of these failures for the practical applicability of the full discover-certify-preserve pipeline on larger models?
  3. What is gained by preserving gradient-flow conservation laws during Adam optimization, beyond achieving smaller conservation law drift?
  4. Please provide a brief additional overview of the certify criteria introduced in Section 2.2. For each criterion, briefly state its goal and give its mathematical formulation exactly with clearly defined notation.
Reviewer_u1zuOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper tests whether a learned candidate map defines common conservation laws across multiple training environments.

Critical strengths
  1. The paper is structurally complete.
  2. It includes substantial theoretical modeling.
Critical weaknesses

This paper is difficult to follow. From the very beginning of the introduction, it does not explain why the problem matters for research. It is also full of defensive writing (for example, L27–L28), and the figures and layout are quite careless(figure 1, page 8-9).

Table 9 reads like an internal protocol log, not a single scientific claim. Each row asks a different question, and only v14 and v15 are treated as formal outcomes while v3 and v5 failed. Gaps such as v4 and v6 are never explained. The main text repeats the same pattern, with version labels and formal-status language substituting for a clear statement of what was tested and what follows from it.

v14 only tests pre-specified K0 versus K0+1 after blind counting failed. The 40/40 result therefore certifies a known boundary on controls rather than discovering common conservation laws from data.

Missing related work section.

Questions for authors

See the Weakness

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.