acceptodds
Under review as a conference paper at ICLR 2027

The Readout-Gauge Conflict: Anchored Equation Discovery with Learned Koopman Closure

Abstract

Learned Koopman representations can improve equation discovery, yet their flexibility can make the reported equation depend on the chosen coordinates: an auxiliary closure coordinate may absorb a named library term without changing the predicted vector field, even as the coefficient assigned to that term changes. We call this failure of the scientific readout the readout-gauge conflict. The Resolvent-based Tail-Residual Network (ReTRN) addresses this conflict by declaring the candidate equation library before fitting, adapting only auxiliary closure coordinates, and reading the equation from the same generator used to fit the lifted dynamics. We characterize the closure transformations that preserve this anchored readout and train the adaptive closure from sparse ordered observations through multiscale temporal moments, without numerical differentiation. Controlled experiments show that anchoring, rather than neural capacity alone, preserves coefficient recovery: across six systems, ReTRN gives the best result in 11 of 12 primary comparisons and all 180 paired rollout comparisons, and transfers to seven additional equation families.

Primary area:
learning on time series and dynamical systems
Keywords:
Continuous-time system, Data-driven model discovery, Koopman operator

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_umnLOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes ReTRN to address the Readout-Gauge Conflict in learned Koopman representations, where different representations may produce identical predicted dynamics but different scientific equation coefficients. ReTRN fixes the scientific equation space, learns an auxiliary neural closure, and uses multi-scale temporal moments to recover governing equations. The authors validate the method through theoretical analysis, targeted interventions, and experiments on multiple dynamical systems.

Critical strengths

The paper explicitly distinguishes dynamical prediction from scientific equation coefficient recovery. Theoretical analysis, targeted interventions, and component ablations support the roles of the fixed equation space and learned closure. Experimental results across multiple dynamical systems further demonstrate the effectiveness of the method.

Critical weaknesses

ReTRN relies on a predefined scientific equation space and relatively restrictive theoretical assumptions. Moreover, accurate coefficient estimation does not necessarily lead to reliable equation support recovery, limiting the method's applicability to unknown dynamical systems.

Questions for authors
  • Damped Pendulum achieves an AUPRC of 1, but exact support recovery is 0/10 in Tables 10–11. If the sparsification threshold search range is expanded, can the existing validation criterion select the correct equation without using ground-truth information? Have the authors explored alternative selection criteria?
  • Are the identified nonlinear terms and their coefficients stable when the candidate dictionary order or the 10% validation-error tolerance is varied? Does the corresponding long-horizon prediction performance remain consistent?
3 comments

We appreciate your recognition of the distinction between accurate dynamical prediction and interpretable equation recovery, as well as the theoretical analysis and targeted experiments supporting ReTRN. We have examined two practical aspects of that distinction: selecting the correct equation support without coefficient truth, and assessing the stability of nonlinear terms under changes in the candidate dictionary and sparsification criterion.

W1 / Q1. Can ReTRN select the correct equation support without ground truth?

Yes, in the tested Pendulum setting, a frozen coefficient-gap selector recovers the exact support, and expanding the original prediction-based threshold grid alone does not. The Pendulum fits achieve AUPRC = 1, so the active terms already rank above the inactive candidates. Their support errors arise when short-validation prediction retains small extra terms. On the original ten fits, expanding the threshold grid under minimum-validation error, 1/5/10% parsimony, or one-standard-error selection still gives 0/10 exact supports.

To use the information in that ranking, we normalize coefficient magnitudes by the largest coefficient of each output, pool the positive normalized magnitudes, and place the threshold at the largest adjacent multiplicative gap whose geometric midpoint lies in [10−4,0.30]\left\lbrack 10^{- 4},0.30 \right\rbrack. The rule and ReTRN configuration were frozen before 20 new independent seven-observation Pendulum replicates were generated. We apply it to each candidate head/closure split, select the split using disjoint short validation, and evaluate frozen equations on 32 new three-second trajectories per replicate. Neither true coefficients nor long tests enter selection, and no coefficient refitting is used.

Frozen selectorExact supportMean F1FP totalSelected NRMSE3 s MSE mean ± SD
Minimum validation4/200.66551470.0013052.16e-08 ± 3.3e-08
10% parsimony5/200.6941300.0011241.91e-08 ± 2.59e-08
One standard error5/200.70621270.0011161.69e-08 ± 1.93e-08
Coefficient gap20/20108.7e-052.03e-07 ± 2.98e-07

The result directly answers the support-recovery question: the frozen gap rule removes all false positives across 20 new replicates, whereas the ordinary validation criteria retain extra terms. All four selectors retain every true term. The gap-selected equations have lower coefficient error, while their mean three-second prediction MSE is higher (2.03e-07 versus 2.16e-08 for minimum validation). This quantifies the trade-off between exact equation structure and finite-horizon prediction error.

Does the gain come from the threshold or from choosing a different fitted model? We held each replicate’s fitted model fixed and applied both threshold rules to the minimum-validation candidate (MvalM_{val}) and the gap-selected candidate (MgapM_{gap}):

Fixed modelThresholdExact supportMean F1FP totalSelected NRMSE3 s MSE
MvalM_{val}Validation4/200.66551470.0013052.16e-08
MvalM_{val}Gap20/20100.0006781.86e-05
MgapM_{gap}Validation6/200.7937660.000171.79e-08
MgapM_{gap}Gap20/20108.7e-052.03e-07

For either fixed model, gap thresholding recovers 20/20 exact supports, establishing that the improvement in structural selection comes from the cutoff rule itself. Choosing the fitted representation still matters for parameter and prediction accuracy: under the same gap rule, the selected NRMSE is 6.78e-4 for MvalM_{val} and 8.70e-5 for MgapM_{gap}. The complete gap-based procedure jointly selects a suitable representation and its sparse equation. The candidate equation library remains predeclared. For uncertainty over nested choices of UU, Appendix A.5 uses a frozen short-validation warning-and-abstention rule: it accepted 39/40 independent deployments, and every accepted space contained all true terms on post-freeze inspection. This assesses candidate-space adequacy within the tested families, complementing the support-selection result above.

Q2. Do the recovered nonlinear terms and predictions remain stable when dictionary order or validation tolerance changes?

Within the tested cubic-to-quartic expansion, the active nonlinear coefficients remain accurate and three-second predictions improve. Changing the sparsification tolerance leaves the retained coefficient values unchanged and usually leaves the selected support unchanged. The measured heave case also identifies where stronger pruning starts to remove a predictively important nonlinear term.

We varied the polynomial dictionary order from cubic to quartic on seven-observation Duffing and Reaction systems, using five matched fits per setting. The equation heads contain 9/14 and 10/15 functions, respectively. The expanded pipeline also uses next-degree auxiliary closure. All coefficient NRMSE values use the same complete-quartic reference denominator:

System and ∣U∣\lvert U\rvertShared cubic coefficientEquation NRMSE3 s MSE
Duffing 9-3.000705 ± 0.0002370.000456 ± 8.95e-050.000492 ± 7.1e-05
Duffing 14-2.999988 ± 0.0000401.44e-05 ± 8.84e-061.72e-06 ± 2.19e-06
Reaction 10-0.998121 ± 0.0002290.001365 ± 0.0004363.13e-06 ± 2.38e-06
Reaction 15-1.000215 ± 0.0001740.000141 ± 0.0001553.16e-08 ± 6.97e-09

The shared cubic coefficients approach their true values (−3- 3 and −1- 1), and both coefficient and rollout errors decrease in all five matched pairs for each system. Thus, expanding the tested dictionary does not destabilize these recovered mechanisms.

We then varied the short-validation error tolerance from 0% to 25% without coefficient refitting. Retained coefficients therefore remain fixed by construction. The selected supports are identical in 19/20 fits, and one cubic Reaction fit removes small extra terms at 25%. Every true term remains selected, and all 120 three-second rollouts remain finite. At the 10% rule, exact-support counts are 0/5 and 3/5 for cubic/quartic Duffing, and 4/5 for both Reaction orders, showing that the remaining errors involve extra terms.

Does the tolerance preserve the nonlinear dynamics in experimental data? On the measured heave-decay record, the selected equations share the following terms, with aa and bb determined by the sparsification tolerance:

$ { \dot{ x }} _ { 1 } = 0.9999633x_{2}$

x˙2=−68.87096x1−1.669794x2−16.85411x1x2{\dot{x}} _ {2} = - 68.87096x_{1} - 1.669794x_{2} - 16.85411x_{1}x_{2}

+533.2143x13+1.228700x23−4732.883x14+ 533.2143x_{1}^{3} + 1.228700x_{2}^{3} - 4732.883x_{1}^{4}

+ax12x2+bx12x22+ ax_{1}^{2}x_{2} + bx_{1}^{2}x_{2}^{2}

ToleranceaabbDynamic termsValidation MSEHeld-out displacement MSE
0%, 1%, 5%−87.66034−97.8047481.10765e-51.77298e-5
10%−87.66034071.21650e-51.34112e-5
15%, 20%, 25%0061.26964e-51.44848e-4

At 10%, the rule removes x12x22x_{1}^{2}x_{2}^{2} while retaining x12x2x_{1}^{2}x_{2} and achieves the lowest held-out MSE among the tested tolerances. At 15%, it also removes x12x2x_{1}^{2}x_{2}, increasing held-out displacement MSE by approximately 10.8-fold relative to 10%. The linear backbone and the other retained coefficients remain stable across these choices. The first three measured records are used for training and validation, and record four is held out intact. This identifies a concrete tolerance boundary for the measured system: moderate pruning retains the recovered nonlinear response, whereas stronger pruning degrades long-horizon prediction.

Thank you for addressing my concerns. I appreciate this theory-motivated work, but given the remaining limitations to its applicability, I will maintain my rating.

Reviewer_9MdYOpenReview ↗
Rating
4(+1)
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper addresses the issue of obtaining interpretable governing equations via Koopman learning. The authors identified the 'readout-gauge conflict', where using neural networks to address Koopman closure causes inaccurate equation coefficients leading to ambiguous interpretability. They propose ReTRN to resolve this, which partitions observables into a user-prescribed equation head U(x) and a residual T(x) to absorb leakage. The method bypasses numerical differentiation problems and trains the continuous-time generator via multi scale exponentially weighted temporal moments. Theoretical results include an error bound for coefficient recovery and empirical investigations cover several ODE systems with ablations on key components.

Critical strengths
  • The paper provides a formalization of the readout-gauge conflict, calling out the flawed assumption that deep Koopman representations can be interpreted by just inspecting linear coefficients. It is good that a work is explicitly highlighting the issue of internal gauge symmetry and showing this theoretically.
  • The work does away with issues around numerical derivatives which is noisy especially for sparse and short trajectories.
Critical weaknesses
  • The paper could be strengthened by discussing past works that dynamically learns symbolic candidate terms rather than just assume that a small fixed dictionary of terms is sufficient to interpret the system. Works such as [1,2,3] rely on methods such as pre-trained foundation models and search algorithms to discovering much less constrained expressions, doing away with Assumption 2.1 that ReTRN has on knowing all candidate term. I view this paper as complementary, and adding discussion with these other related works would help better position the paper.
  • Related to the above, ReTRN parameter discovery degrades significantly when U is misspecified, and expanding U to include many unknown terms will cause singular value collapse. Also the paper's appendix indicates numerical issues with the matrix logarithm for e.g.. As per the author's theorem, this blows up the error bound. This can be discussed in more detail.
  • The novelty in terms of residual architecture is also limited, as there are already past works that couples an interpretable base with a learned residual via neural networks [4,5].

[1] Lau et al, README: Rapid Equation Discovery with Multimodal Encoders, AI4Math ICML 2025 [2] Meidani et al, SNIP: Bridging Mathematical Symbolic and Numeric Realms with Unified Pre-training, ICLR 2024 [3] d'Ascoli et al, ODEFormer: Symbolic Regression of Dynamical Systems with Transformers, ICLR 2024 [4] Rackauckas et al., Universal Differential Equations for Scientific Machine Learning, arxiv 2020 [5] Yin et al, Augmenting Physical Models with Deep Networks for Complex Dynamics Forecasting, ICLR 2021

Questions for authors
  • How sensitive is ReTRN to misspecification of U and numerical stability? The appendix shows that standard SciPy matrix log fails for all 60 models. What's the empirical tolerance of redundant or higher order terms entering U before coefficient recovery degrades significantly?
  • What's the computation cost comparison of ReTRN v.s. other methods, such as WSINDy? The latter seems to perform well and should be fast, though it seems to degrade badly on a few examples like the Competition model -- is it due to instability related to hyperparamter selection? Providing more of such analysis could make clearer the advantages of ReTRN over other methods including derivative-free baselines.
  • What prevents T from gradually drifting and partially absorbing U's dynamics over extended training periods?
3 comments

We appreciate the reviewer’s recognition of the readout-gauge formulation and of learning governing equations from short trajectories without numerical-derivative targets. We address the scope of equation discovery, numerical reliability, computational cost, and training stability below.

W1. How does ReTRN handle unknown equation structures?

Symbolic regression addresses the discovery of functional structure, while ReTRN estimates coefficients and selects active terms within a declared candidate library UU. The current method has not been combined with README, SNIP, or ODEFormer. A future integration could use symbolic regression to propose functions for UU, followed by anchored coefficient estimation with ReTRN, potentially extending the range of discoverable equations. We will include these methods in our discussion.

For the current method, Appendix A.5 tests selection among predeclared nested candidate spaces using short-validation warnings and abstention. The frozen rule accepted 39/40 independent deployments, and all 39 accepted spaces contained the true terms on post-freeze inspection. The 30 deployments with prospectively locked recovery tests achieved 30/30 exact supports, and nine accepted Lorenz–84 deployments had adequacy tests only. This supports candidate-space screening within the tested families. Mechanisms absent from every candidate space remain outside the current model class.

W2 and Q1. How much misspecification and redundancy can the method tolerate? Is the matrix logarithm stable?

Omitting an active mechanism is consequential: ReTRN cannot recover its named coefficient if the term is absent from UU. In the capacity-matched Duffing intervention (Appendix A.9), assigning the true cubic term to closure increases coefficient NRMSE 214-fold.

Redundant candidates are tolerated within the tested range. We expanded UU on seven-observation Duffing and Reaction data and measured the training-scaled singular values and condition numbers over five matched fits:

System and UU degree∣U∣\lvert U\rvertσmin⁡(U)\sigma_{\min}(U)σmin⁡(Z)\sigma_{\min}(Z)κ2(U)\kappa_{2}(U)κ2(Z)\kappa_{2}(Z)
Duffing cubic913.81 to 14.66.068 to 6.679.505 to 10.0823.97 to 26.35
Duffing quartic146.068 to 6.672.39 to 2.66523.97 to 26.3571.43 to 80.06
Reaction cubic101.307 to 1.4920.2955 to 0.3428124.7 to 141.8642 to 741.3
Reaction quartic150.2951 to 0.33920.0634 to 0.07239654.7 to 748.13544 to 4018

All 20 designs remained full rank. Even as κ2(Z)\kappa_{2}(Z) reached 4018 on Reaction, the complete expanded-head and next-degree-closure pipeline reduced coefficient NRMSE from 1.36e-3 to 1.41e-4 and Duffing improved from 4.56e-4 to 1.44e-5. Three-second errors also decreased. Thus, the tested redundancy did not cause rank collapse or impaired recovery.

SciPy’s matrix logarithm did not fail on the 60 reported fits. The original exact-zero acceptance guard routed estimates with numerical error around 10−1510^{- 15} to a fallback. Its generators agree with SciPy’s within 1.58e-13. With a tolerance-based guard, all 60 operators pass, selected supports remain unchanged, and the largest relative coefficient and absolute three-second MSE changes are 1.68e-12 and 2.15e-11, respectively.

W3. What differs from existing physical–neural models?

The learned network enters a different mathematical object, and the reported equation is different. In physical–neural hybrid models, such as UDE- or APHYNITY-style formulations, a learned correction can enter the state vector field directly: x˙=U(x)B+gθ(x)\dot{x} = U(x)B + g_{\theta}(x). The final dynamics then include the neural correction, making allocation between the physical and learned terms central to interpretation.

ReTRN learns auxiliary Koopman observables, Zθ=[U,V+rθ]Z_{\theta} = \left\lbrack U,V + r_{\theta} \right\rbrack, and reports the explicit equation x˙=UBU\dot{x} = UB_{U} from the same fitted generator. Its learned tail improves the lifted dynamics but is excluded from the reported governing equation. The contribution lies in this anchored generator readout, its coordinate invariance properties, and temporal-moment closure learning. In the matched-capacity construction control (Appendix A.10), protecting equation coordinates reduces Reaction coefficient NRMSE from 0.101 to 0.00311 across 20 paired fits. We will clarify the relationship to UDEs, APHYNITY, and Donà et al. in the paper.

Q2. What is the computational cost, and why is WSINDy inaccurate on Competition?

Cost was measured under a common protocol: identical seven-point data, one host, two logical CPU cores and numerical threads, and three timing repetitions. Times include fixed-configuration fitting and short-validation equation selection, and hyperparameter search is excluded for every method. The supplement pairs these timings with the timed models’ coefficient and rollout errors.

MethodLorenz (s)Reaction (s)Magnet (s)
KLM0.3071 ± 0.057980.2306 ± 0.0048740.3622 ± 0.01443
RTM0.5679 ± 0.015010.4957 ± 0.015750.5497 ± 0.02129
WSINDy10.41 ± 1.1338.9 ± 0.069039.258 ± 0.03884
ReTRN3.261 ± 0.54679.983 ± 0.166526.01 ± 0.3077

ReTRN is faster than WSINDy on Lorenz, slightly slower on Reaction, and slower on Magnet. KLM and RTM remain inexpensive.

The Competition discrepancy originates in the seven-point weak equations themselves. Substituting the true coefficients leaves a 1.76–2.67% residual, which is amplified by weak-design condition numbers of 3.52×1033.52 \times 10^{3}–4.24×1034.24 \times 10^{3} into 18.5–150% relative coefficient error in ordinary least squares. WSINDy’s SR3 fit has essentially the same coefficient NRMSE in these cases, showing that the error is present before sparse selection. At 31 points, used only to diagnose temporal discretization, the true-equation weak residual falls to approximately 5×10−75 \times 10^{- 7}. This explains why coarse weak-form discretization and conditioning substantially affect this Competition result.

Q3. What prevents the learned tail TT from absorbing dynamics represented by UU?

The learned tail can contain functions that overlap with UU. ReTRN does not impose an orthogonality constraint that prevents this. Instead, it keeps the named functions in UU fixed and reads the governing equation as UBUUB_{U} from the fitted generator. Zero initialization provides the analytic starting representation, while the temporal-moment objective is evaluated in the declared coordinates and generally changes under head–tail mixing. These choices specify how the coefficients are estimated, without forcing the learned tail to remain orthogonal to UU.

We therefore tested whether the overlap actually changes the reported coefficients. For the data-supported shear T↦T−UAθT \mapsto T - UA_{\theta}, the induced change is ΔBU=AθBT\Delta B_{U} = A_{\theta}B_{T}. We measured this change, the learned residual’s overlap with UU, and the returned equation’s accuracy at the same saved checkpoints in five independent extended-budget runs per system. AθA_{\theta} was estimated from training anchors, with disjoint short-validation states used for the diagnostic.

SystemBudgetOverlap normDgaugeD_{gauge} (%)Equation NRMSE3 s equation MSE
Lorenz1×0.99090.0032710.0048634.817
Lorenz2×0.95360.015170.0050884.877
Lorenz4×0.93280.040760.0057056.238
Reaction1×0.99290.12530.0019781.92e-05
Reaction2×0.99470.59940.0023046.66e-05
Reaction4×0.99531.060.002417.95e-05
Magnet1×0.93890.070580.0005841.78e-09
Magnet2×0.93980.089460.0004398.07e-10
Magnet4×0.94990.13160.000421.08e-09

Overlap is ∥UAθ∥F/∥rθ∥F\parallel UA_{\theta} \parallel_{F}/ \parallel r_{\theta} \parallel_{F} and Dgauge=100∥AθBT∥F/∥BU∥FD_{gauge} = 100 \parallel A_{\theta}B_{T} \parallel_{F}/ \parallel B_{U} \parallel_{F}, measured in raw coefficient coordinates. In Reaction, gauge sensitivity grows from 0.1253% to 1.06% as coefficient error increases. In Magnet, gauge sensitivity also rises, yet coefficient error decreases. In Lorenz, overlap declines while sensitivity increases. Thus overlap alone does not determine coefficient accuracy. Its direct effect on the readout is governed by AθBTA_{\theta}B_{T}, while other optimization effects may also change recovery quality. ReTRN selects checkpoints using disjoint short validation, which provides an empirical selection rule but no guarantee against later absorption.

Thank you for the detailed clarifications and additional experiments, which have helped resolve my concerns. While the proposed method still has major limitations (e.g., pre-determined dictionary) in performing the full equation discovery task, it could be a valuable addition to the overall pipeline, complementing upstream symbolic regression methods (e.g., README, SNIP, ODEFormer etc). If the authors revise their paper to include discussion of their proposed approach in the context of these methods as they have indicated, I believe this paper will be a good contribution to the literature. Hence, I increase my score.

Reviewer_83HSOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper identifies the readout-gauge conflict in Koopman-based equation discovery: learned closure coordinates can absorb named library terms, so the reported coefficients change even though the predicted dynamics do not. The proposed ReTRN fixes the candidate equation library before fitting, adapts only the auxiliary closure coordinates through a zero-initialized residual network, and reads the coefficients directly from the fitted generator. Experiments on canonical, coupled, controlled, and real-world systems show consistent gains over Koopman and sparse-regression baselines.

Critical strengths
  1. Clear problem framing with a principled design. The paper formalizes how learned closure coordinates can shift named coefficients without changing predictions (Prop. 3.1), and ReTRN follows directly from this. It fixes the equation library, adapts only a zero-initialized closure residual, and therefore starts exactly from the KLM estimator. This is the main conceptual contribution and is useful to the community, even though the theory itself is elementary.

  2. Rigorous, transparent experiments. The evaluation uses independent replicates and paired tests with Holm correction, locks all configurations before test data are generated, and openly reports abstentions and non-significant results.

Critical weaknesses
  1. The conflict is not resolved by the method. Because rθr_\theta is unconstrained, the learned tail can still contain equation-space directions. In fact, 80–97% of the learned residual energy lies in span(U) (App. A.8), which is exactly the shear that Prop. 3.1 shows alters BUB_U. The readout stays stable only because BT≈0B_T \approx 0 (the closure contributes about 0.1% of the field), i.e., where the conflict is harmless anyway. This undermines the paper's central claim.

  2. The theory does not explain the method's benefit. Prop. 3.1 is elementary block linear algebra. Thm. 3.3 holds for any lift containing U, including KLM's, and its constants (sY,CΓ,M2s_Y, C_\Gamma, M_2) are never reported, so the bound may be vacuous. As a result, the theory does not account for why a learned closure improves the readout.

  3. Key baselines and related work are missin-free methods suited to sparse trajectories(RK4-SINDy, integral SINDy, multiple shooting on f=UBf=UB), as well as hybrid-model identifiability work (APHYNITY, Dona et al. 2022) that studies the same absorptiother the novelty nor the advantage overexisting methods is established.

Questions for authors

Q1. Orthogonalized closure. What happens to the readout and performance if rθr_\theta is projected onto the orthogonal complement of span(U), which removes the shear in Prop. 3.1 by construction? If anchoring alone is meant to suffice, why is this constraint unnecessary?

Q2. Derivative-free baselines. How does ReTRN compare with RK4-SINDy, integral SINDy, and multiple shooting on f=UBf=UB on the seven-observation benchmark, both noise-free and with 1–5% observation noise?

Q3. Tightness of Thm. 3.3. Can you report sYs_Y, CΓC_\Gamma, M2M_2, and δlog⁡\delta_{\log} for the trained models, and compare the right-hand side of Eq. (9) with the realized error? Does the bound actually improve from KLM's lift to ReTRN's?

6 comments

We thank the reviewer for recognizing the principled formulation of the readout-gauge conflict, the fixed-head design, and the transparent evaluation with independent replicates and frozen configurations.

W1 / Q1. What does anchoring guarantee, and is residual orthogonalization necessary?

Anchoring fixes which equation coefficients are reported; it does not eliminate residual overlap. ReTRN fixes the candidate functions UU and reads their coefficients from the same fitted generator. Zero initialization supplies an analytic starting point but does not prevent later overlap. We will clarify that arbitrary head–tail mixing need not preserve the reported coefficients.

To see the distinction, write the lifted state field as UBU+TBTUB_{U} + TB_{T}, where BUB_{U} contains the reported equation coefficients and BTB_{T} weights the auxiliary closure. The reviewer’s transformation gives

T′=T−UA,BU′=BU+ABT,UBU′+T′BT=UBU+TBT.T^{\prime} = T - UA,\quad\quad B_{U}^{\prime} = B_{U} + AB_{T},\quad\quad UB_{U}^{\prime} + T^{\prime}B_{T} = UB_{U} + TB_{T}.

The coefficient shift is ABTAB_{T}, so high residual overlap alone does not quantify readout instability. On the 20 native confirmation fits per system used below, we fit the residual shear on training anchors after training. The mean relative shifts are 2.65e-7%, 4.11e-4%, and 0.0482% for Lorenz, Reaction, and Magnet, respectively. We measure this as 100∥ΔBU∥F/∥BU∥F100 \parallel \Delta B_{U} \parallel_{F}/ \parallel B_{U} \parallel_{F}, converting ΔBU=AθBT\Delta B_{U} = A_{\theta}B_{T} and BUB_{U} to raw coefficient coordinates before sparsification. The following experiment tests whether enforcing residual orthogonality improves coefficient recovery.

Native ReTRN recovers accurate coefficients in these tests without enforcing residual orthogonality. To test the proposed constraint, the comparison variant subtracts a training-time least-squares projection of rθr_{\theta} onto the span of UU on the training anchors. This projection is used only in the orthogonalization control, and native ReTRN uses T=V+rθT = V + r_{\theta} without it. Orthogonality is enforced for the residual on these anchors.

Both variants receive 32 validation-search configurations per system, including strong native reference settings, followed by 20 new independent paired data draws. Each pair shares training and validation data. Configurations and equations are frozen before new three-second tests. These are separately tuned pipelines with architectures and search records in Supplement S1.

SystemTraining variantCoefficient NRMSE3 s equation MSE
LorenzNative0.005023 ± 0.0008077.893 ± 5.821
LorenzProjected residual0.005109 ± 0.0008299.275 ± 6.935
ReactionNative0.00039 ± 0.0002633.99e-08 ± 5.8e-08
ReactionProjected residual0.002925 ± 0.002271.79e-05 ± 7.3e-05
MagnetNative0.000305 ± 0.000151.57e-10 ± 6.01e-11
MagnetProjected residual0.000297 ± 0.0001772.94e-11 ± 1.01e-11

Mean ± sample SD over 20 pairs. Coefficient NRMSE measures the head before sparsification and prediction uses the selected analytic equation without the neural tail. All 120 fits and rollouts succeed. Selected-coefficient and support results are in S1.

Native ReTRN has lower coefficient error in 19/20 Lorenz pairs and 16/20 Reaction pairs (Holm-adjusted p = 1.34e-4 and 2.46e-4). Magnet has no significant coefficient difference (p = 0.812). Projection improves its prediction in all 20 pairs (p = 1.14e-5). Lorenz’s mean paired relative prediction change has a 95% interval of −19.8% to 10.8%, so its mean forecasting benefit is uncertain. The projected residual’s normalized training cross correlation is at most 6.24e-14, confirming that the requested constraint was enforced.

Together, the small measured shear-induced shifts and these paired results support accurate recovery by the declared ReTRN estimator despite residual overlap. Our claim therefore concerns a specified and empirically validated readout, not coefficient uniqueness across arbitrary prediction-equivalent representations. Projection worsens coefficient accuracy on Lorenz and Reaction, while improving Magnet prediction. Its effect is system-dependent. The matched-capacity control in Appendix A.10, discussed below, separately tests the benefit of protecting the equation coordinates.

W2 / Q3. What does the learned closure contribute, and how informative is the theoretical bound?

A small BTB_{T} does not imply inactive closure. When UU contains the true equation, the ideal state equation is f=UB⋆f = UB^{\star}, with zero tail weight. Closure can still improve the evolution of other observables, whose derivatives may lie outside UU.

We tested this on frozen seven-observation models by comparing generator predictions with the Lie derivatives of the same named higher observables. Synthetic right-hand sides enter only this held-out diagnostic. Relative Frobenius defects over 20 matched KLM/ReTRN pairs are

SystemKLM defectReTRN defectReTRN lowerHolm p
Lorenz0.36430.364120/202.86e-05
Reaction0.029960.0297312/200.33
Magnet0.25690.155820/202.86e-05

Closure error falls substantially on Magnet, slightly on Lorenz, and not significantly on Reaction. This shows that small state-output tail weights can coexist with useful closure, but it does not fully explain coefficient gains on every system.

Two controls test specific sources of benefit. Appendix A.10 compares protecting the equation coordinates with letting the same network modify them, at equal neural capacity and matched optimization. Both models in this separate placement control use a common training-only least-squares projection onto the analytic library. Coefficient NRMSE is 0.00555/0.0328 on Lorenz, 0.00311/0.101 on Reaction, and 2.19e-4/0.260 on Magnet (protected/distributed), lower in all 20 pairs per system (Holm p = 1.72e-5). This tests where learned capacity is placed, rather than isolating native direct readout.

Second, the seven-observation temporal-versus-diagonal-Gram control tests the temporal coupling in the loss, with equal separate validation tuning. Lorenz’s three-second MSE is 2.13 versus 3.40, improving in 17/20 pairs (Holm p = 0.00290). Reaction and Magnet differences are not significant. We will distinguish these seven-observation controls from the denser-observation ablations in Appendix A.6.

Theorem 3.3 gives conditional readout-error control through moment loss, conditioning, and approximation terms, under rank and principal-log assumptions. It also applies to KLM and does not promise that learned closure improves every term. We confirm that its numerical bound is very loose and cannot explain the performance ranking.

The requested factors and realized errors below are means over 20 matched models per method and system. Error uses the feature-scaled full state-output block in Eq. (9), a different metric from coefficient NRMSE.

System and liftsYCΓM2 proxyδlog,hRHS proxyActual error
Lorenz KLM0.73971.37e+071.39e+0515.471.55e+100.6269
Lorenz ReTRN0.74191.68e+071.39e+0515.471.89e+100.6272
Reaction KLM0.055351.98e+0836961.6091.62e+110.03099
Reaction ReTRN0.072351.19e+0836400.80216.78e+100.02127
Magnet KLM0.03132.51e+07370.60.090856.1e+100.001724
Magnet ReTRN0.0328.4e+07404.40.24558.04e+100.001931

Here the step size is 0.05 and the temporal-weight factor is 0.272901. M2 is a sampled curvature proxy and CΓ is a floating-point contour diagnostic. The reported RHS values are therefore numerical diagnostic proxies, not certified upper bounds.

The main source of slack is the logarithm-stability term, approximately 1e10–1e11, compared with 1e2–5e3 for the curvature term and 0.09–15.5 for the logarithm remainder. The contour maximum and norm inequalities multiply worst-case amplification factors and discard directional cancellation. Replacing that coarse contour estimate with a positive-shift resolvent integral lowers the numerical sensitivity estimates by five to seven orders of magnitude, but they remain far above actual error (S3). This diagnoses conservatism and does not certify a tighter bound.

The original RHS improves on Reaction but worsens on Lorenz and Magnet. We will therefore present the theorem as conditional error control and use the preceding experiments to substantiate the measured benefits of the construction.

W3 / Q2. How does ReTRN differ from prior methods, and how does it compare experimentally?

APHYNITY and Donà et al. (2022) address ambiguity in physical–neural decompositions of the state vector field, and we will acknowledge and cite those contributions. ReTRN instead learns auxiliary Koopman observables and reads a governing equation containing only fixed candidate functions from the fitted generator. The neural tail is excluded from the returned equation. Its distinct contributions are the anchored generator readout, the associated invariance conditions, and temporal-moment closure learning from sparse ordered observations. We do not claim comparative superiority over these hybrid methods without matched experiments.

Derivative-free baseline comparisons

We added integral SINDy, an implementation of the RK4-SINDy objective, and multiple shooting with shared equation coefficients on the declared library. As requested, the comparison covers the seven-observation benchmark without noise and with 1% and 5% observation noise.

The noiseless comparison extends the submitted main tables. All original ReTRN, KLM, RTM and WSINDy values are retained, with the three requested baselines added. Across six systems and both metrics, ReTRN achieves the best overall mean rank among seven methods with full coverage (2.42), followed by the RK4 objective (2.83). Individual systems show complementary strengths across the methods.

Noiseless coefficient NRMSE continuing the main table. Mean with sample SD in parentheses over 20 training resamples.

MethodVan der PolDuffingLorenzCompetitionReactionMagnet
ReTRN0.008095 (0.0005659)0.0002206 (0.0001076)0.005624 (0.0008226)0.005631 (0.001067)0.00269 (0.0006274)0.0005176 (0.0002517)
KLM0.01052 (0.001061)0.0007185 (9.964e-05)0.00779 (0.001128)0.006322 (9.853e-05)0.004983 (0.0016)0.0006389 (0.0002457)
RTM0.1718 (0.001875)0.1325 (0.0004334)0.04112 (0.004037)0.1351 (0.003976)0.01759 (0.00157)0.002069 (4.696e-05)
WSINDy0.009181 (0.001387)0.01525 (0.005467)0.0486 (0.006167)0.7631 (0.4772)0.03179 (0.007691)0.0008752 (0.0004257)
Integral SINDy0.00155 (0.0001592)0.003395 (0.0004033)0.005767 (0.0007941)0.2071 (0.03727)2.447e-05 (1.614e-06)1.856e-07 (6.117e-09)
RK4 objective2.566e-05 (9.209e-06)5.17e-05 (5.182e-05)0.0006695 (0.00033)0.01358 (0.003911)0.02605 (0.001832)0.0008936 (0.0003614)
Multiple shooting1.319e-05 (5.398e-06)4.407e-05 (4.735e-05)0.006654 (0.001564)0.01924 (0.008232)0.01789 (0.005408)0.0008941 (0.0003598)

Noiseless three second prediction MSE continuing the main table. Mean with sample SD in parentheses over 20 training resamples.

MethodVan der PolDuffingLorenzCompetitionReactionMagnet
ReTRN0.0003251 (9.928e-05)1.009e-05 (7.957e-06)2.379 (0.7706)2.392e-05 (1.074e-05)7.169e-07 (2.094e-07)5.519e-11 (1.926e-11)
KLM0.0004326 (0.0001222)0.000223 (3.081e-05)11.13 (2.763)2.996e-05 (2.737e-06)2.043e-06 (2.747e-07)1.588e-09 (1.068e-09)
RTM1.339 (0.007461)20.19 (0.055)63.15 (7.19)0.1037 (0.001699)0.0001757 (8.909e-06)8.595e-08 (2.634e-08)
WSINDy0.0001176 (3.54e-05)0.03592 (0.05152)39.69 (3.251)4.108e+05 (9.985e+05)2.79e-05 (2.102e-05)4.037e-09 (2.641e-09)
Integral SINDy2.678e-05 (6.761e-06)0.01266 (0.001172)9.832 (1.375)1700 (7604)5.874e-09 (1.203e-09)8.434e-15 (5.377e-16)
RK4 objective1.062e-08 (8.155e-09)5.76e-08 (8.994e-08)1.007 (0.6755)0.0005693 (0.0002519)1.713e-06 (8.425e-07)2.133e-09 (1.84e-09)
Multiple shooting8.111e-10 (1.042e-09)9.511e-08 (1.252e-07)12.84 (5.698)0.0009802 (0.000677)2.728e-05 (2.179e-05)2.089e-09 (1.387e-09)
PySR reference--60.13 (1.671)-7.918e-06 (7.196e-07)1.525e-05 (2.52e-05)

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.