acceptodds
Under review as a conference paper at ICLR 2027

Neural ODEs Meet Concurrent Learning: Stable Online Learning with Lyapunov Guarantees

Abstract

Neural ODEs learn dynamics from trajectory losses, but their adjoint gradients lack the regressor-times-parameter-error structure on which Lyapunov analyses of online adaptation rest, so training on streaming data comes without stability guarantees. We show that this structure is in fact present: the adjoint gradient decomposes exactly into a positive semi-definite trajectory operator acting on the parameter error plus a nonlinear perturbation with explicit, horizon-dependent bounds. A quadratic Lyapunov function then certifies online Neural ODE training over sliding windows under computable gain and horizon conditions, and the same certificate extends to stored data: its drift branch recovers concurrent learning, and its trajectory branch yields NODE-CL, a stored-segment Gauss-Newton method built on batched forward sensitivities that needs no state-derivative estimates. On four DeepMind Control Suite domains, NODE-CL attains the lowest median prediction error on three under velocity measurement noise, where observer-based concurrent learning degrades by up to 8x, and stays within a factor of 1.6 of the best stored-data baseline with clean measurements.

Primary area:
learning on time series and dynamical systems
Keywords:
Neural ODEs, online learning, concurrent learning, nonlinear dynamics, trajectory learning, Lyapunov stability, system identification, Gauss-Newton, adaptive learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_L111OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the lack of Lyapunov stability guarantees for online identification of Neural ODEs from streaming data. The theory establishes UUB bounds, improved exponential rates under window persistence of excitation (WPE), and an excitation-free bound on the average functional error. Experiments are conducted on four DeepMind Control Suite domains and two diagnostic systems.

Critical strengths
  1. Theorem 1 fills the structural gap between the trajectory losses of Neural ODEs and Lyapunov-based adaptive-control analysis.
  2. The lemmas and propositions support the main results level by level, with all constants given explicitly.
  3. Appendix verifies a portion of the theoretical results item by item on the actual code and simulated signals, closing the loop between the theory and its empirical verification.
Critical weaknesses
  1. Footnote 2 and Figure 11(c) acknowledge that the gains were tuned for performance and that the sampled implementations remain stable well below the certified thresholds; however, the paper provides no instance of feasible gains computed from the theorem constants.
  2. In Table 1, the observer-based baselines fix kf = 200, a value tuned on clean data rather than per noise level, whereas Table 2 shows that CL-LS improves the function error simply by raising kf to 800. The benchmark experiments lack a control in which kf is retuned according to the noise level.
  3. On clean data, NODE-CL clearly outperforms the same-segment NODE-replay only on pendulum and falls behind on both cartpole and acrobot. The paper offers no explanation or discussion of these results.
Questions for authors
  1. What distinguishes the proof methodology of this paper from existing Lyapunov stability proof techniques, and in what respects is the derivation novel?
  2. If X(\tau) in Theorem 1 contains velocity measurement noise, how should the perturbation bound in Eq. (21) be modified? Can the noise term enter the proof in a form that can be absorbed by the sigma-modification gain kσ, so that the theory covers the noisy setting actually evaluated in the experiments?
2 comments
AuthorsOpenReview ↗

Response to Reviewer L111 (1/2)

Thank you for the review. The three weaknesses and two questions in order.

W1: an instance of feasible gains computed from the theorem constants. Provided in Part B of the general response and in the new Appendix L.5. On S1 seed 0, with closed-form upper bounds on the derivative constants of the network over Ω×Θϵ\Omega\times\Theta_\epsilon, margin ϖ=1\varpi = 1 and segment length Ts=2T_s = 2 ms, the containment condition Eq. (17) holds and Eq. (25) at α=2\alpha = 2 reads kσ>0.076k_\sigma > 0.076. The run with (α,kσ)=(2,0.084)(\alpha, k_\sigma) = (2, 0.084) and Γ∈[10,100]\Gamma \in [10, 100] satisfies the hypotheses of Theorem 2 and is stable; the leakage holds the estimate at the origin, which is why such gains are not the ones used. At the tuned segment length of 50 ms the same condition requires a far larger kσk_\sigma (1.3×1051.3\times10^{5} with constants maximized over sampled parameters), against 1.2×10−21.2\times10^{-2} when the constants and the error bound are taken along the realized run, and 10−910^{-9} in the experiments. Part B explains why a gap of that size is expected: a worst case over a parameter set chosen large, slope constants and error bounds that multiply, and norm bounds that assume adversarial alignment. The condition is sufficient in the same sense as a step-size bound η≤1/L\eta \le 1/L in optimization theory, the certified baselines share the property, and Part B lists what holds at the tuned gains and what Theorem 2 adds.

W2: kfk_f retuned per noise level. Done (Appendix L.2, Table 7, Section 4.1): CL and CL-LS at σv=3×10−3\sigma_v = 3\times10^{-3} with kf=αo∈{50,100,200,400,800}k_f = \alpha_o \in \{50,100,200,400,800\}, four domains, five seeds, all else unchanged. All entries are from one machine (Table 4 reruns Table 1 there).

pendulumcartpoleacrobotreacher
CL-LS, kf=200k_f = 2000.7531.9843.1080.215
CL-LS, best kfk_f on the grid0.575 (50)0.820 (50)1.926 (50)0.136 (100)
CL, best kfk_f on the grid0.361 (100)0.891 (50)2.226 (200)0.976 (200)
NODE-CL, not retuned0.0430.3021.4740.075

Retuning helps the observer-based methods, and NODE-CL remains ahead of retuned CL-LS by factors of 13, 2.7, 1.3, 1.8 and of retuned CL by 8.4, 3.0, 1.5, 13. The direction is the one the observer analysis gives: the continuous-time floor zˉo\bar z_o falls as kfk_f grows, while the sampled observer differentiates the velocity noise with a factor that grows with kfk_f, so noise moves the best kfk_f downward (to 200 or less on every domain) and the error rises steeply above 200 on the cartpole and reacher; the floor term limits how far lowering kfk_f can help. The improvement at kf=800k_f = 800 in Table 2 is on the noise-free diagnostic runs, where only the floor term is present. NODE-CL has no observer and no gain that must track the noise level. On the pendulum with twenty seeds (Table 5 and Appendix L.1) the best medians on the same grid are 0.572 for CL-LS and 0.355 for CL, against 0.099 for NODE-CL at its Table 1 setting.

W3: clean-data results against NODE-replay. Two points, both in the revision.

  • Size of the effect. On the cartpole and acrobot the per-seed ranges of the two methods overlap (Table 3): 0.29 to 0.57 (NODE-CL, median 0.381) against 0.25 to 0.49 (NODE-replay, median 0.311) on the cartpole, and 0.31 to 1.79 (median 1.545) against 0.75 to 2.55 (median 1.002) on the acrobot, where every method has an error of order one state standard deviation at the 1 s horizon. On the pendulum the ranges are disjoint, and over twenty seeds NODE-CL is lower on 17 (median 0.037 against 0.140, Table 5). Over twenty acrobot seeds (Table 6) the two methods are level, 0.953 against 0.977 (clean) and 1.253 against 1.108 (noise), and both are ahead of CL-LS, CL, and the single-step law at both noise levels.
  • Mechanism. For a fixed GSG_{\mathcal S} and an inactive gate, the gain dynamics of Eq. (10) have the equilibrium Γ=(λˉ−1I+GS/β0)−1\Gamma = (\bar\lambda^{-1}I + G_{\mathcal S}/\beta_0)^{-1}, that is, Γ≈β0GS−1\Gamma \approx \beta_0 G_{\mathcal S}^{-1} in the excited directions and the cap λˉ\bar\lambda elsewhere, so NODE-CL takes a capped Gauss-Newton step. Its advantage over Adam's per-coordinate normalization is the off-diagonal and scale information in GSG_{\mathcal S}, which matters most where GSG_{\mathcal S} is worst conditioned (pendulum, 3×10153\times10^{15} against 10910^{9} to 101110^{11} on the other domains). The constant-gain ablation agrees: with clean data, removing the gain from NODE-CL costs 8.2x, 6.8x, 2.2x, and 1.3x on the pendulum, reacher, cartpole, and acrobot, the order of the condition numbers (3×10153\times10^{15}, 3×10113\times10^{11}, 2×1092\times10^{9}, 1×1091\times10^{9}). Adding NODE-CL's clipping and leakage to NODE-replay does not explain the difference either: the clipping never activates, and over twenty pendulum seeds the two NODE-replay variants agree to within 12 percent.
AuthorsOpenReview ↗

Response to Reviewer L111 (2/2)

Q1: what is new in the proof. The Lyapunov function and the projection and leakage steps are the standard ones. The new element is the object they are applied to. Existing Lyapunov analyses of online adaptation, including concurrent learning and its DNN extensions, start from a gradient that is already linear in the parameter error, a regressor times θ~\tilde\theta plus a bounded term, obtained from a derivative, observer, or integral residual in which the model enters at measured states (integral concurrent learning, Parikh et al., 2019, now cited in Section 1, removes the derivative estimate when the model is linear in the parameters). The adjoint gradient of a trajectory loss has no such form; it depends on θ~\tilde\theta through a nonlinear rollout. Theorem 1 shows that it nevertheless equals Hθ~+ηaH\tilde\theta + \eta_a exactly, with H⪰0H \succeq 0 built from the mean-value transition matrix of the error dynamics and ηa\eta_a bounded by constants that depend explicitly on the horizon, through factors of TT and the Gronwall factor κT\kappa_T. Three consequences follow: online Neural ODE training over a window is certified (Proposition 3), the horizon becomes a computable condition instead of a modeling assumption, and the Gauss-Newton matrix assembled from forward sensitivities is shown to approximate the exact operator HSH_{\mathcal S} with a quantified gap (Lemma 11), which is what brings a second-order gain inside the certificate. The revised introduction states this.

Q2: velocity measurement noise in Theorem 1. Yes, for bounded noise, and in the form you anticipate. Appendix I (Proposition 5) takes the measured trajectory Xm=X+ϱX_m = X + \varrho, ∥ϱ∥≤ϱˉ\Vert \varrho\Vert \le \bar\varrho, with the rollout started from the measured state and the loss evaluated against the measured trajectory, as implemented. The decomposition Ξam=Hθ~+ηaϱ\Xi_a^m = H\tilde\theta + \eta_a^\varrho holds with H⪰0H \succeq 0 constructed as in Theorem 1, from the true trajectory and the rollout started at the measured state, and Eq. (21) becomes

∥ηaϱ∥≤b1∥θ~∥2+b2ϱ∥θ~∥+b3ϱ,d0ϱ=κTεˉ+eLFTϱˉ,\Vert \eta_a^\varrho\Vert \le b_1\Vert \tilde\theta\Vert ^2 + b_2^\varrho\Vert \tilde\theta\Vert + b_3^\varrho,\qquad d_0^\varrho = \kappa_T\bar\varepsilon + e^{L_F T}\bar\varrho,

b2ϱ=TDSwe (2d0ϱ+ϱˉ),b3ϱ=T (d0ϱ+ϱˉ)(wˉ+DSd0ϱ),b_2^\varrho = T D_S w_e\,(2 d_0^\varrho + \bar\varrho),\qquad b_3^\varrho = T\,(d_0^\varrho + \bar\varrho)(\bar w + D_S d_0^\varrho),

with b1b_1 unchanged, under the window condition eLFTϱˉ+κT(υˉθ~ˉ+εˉ)<ϖe^{L_F T}\bar\varrho + \kappa_T(\bar\upsilon\bar{\tilde\theta} + \bar\varepsilon) < \varpi. Proposition 3 and Theorem 2 hold for bounded noise with b2,b3b_2, b_3 (and g1,g0g_1, g_0 of Lemma 11) replaced by these constants. The gain condition of Proposition 3 changes by a term linear in ϱˉ\bar\varrho (that of Theorem 2 also through g0ϱg_0^\varrho, quadratic in ϱˉ\bar\varrho), the residual changes through b3ϱb_3^\varrho, and kσk_\sigma absorbs the noise exactly as it absorbs the approximation error. The proposition gives the mechanism behind the noise results of Table 1: in the observer pathway the noise is differentiated and enters with a factor that grows with the observer gain; in the trajectory pathway it is integrated over the segment and enters with the factor TeLFTT e^{L_F T}, close to TT for short segments.

Reviewer_9s1hOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes NODE-CL, an online Neural ODE identification method that replays stored trajectory segments and updates the parameters with a dynamic Gauss–Newton-type gain, projection, and leakage, without requiring state-derivative estimates. Its main analytical result is a trajectory-error decomposition of the adjoint gradient, which yields conditional Lyapunov bounds on the parameter estimation error for both windowed online Neural ODE training and the stored-segment method; the same analysis is shown to recover a concurrent-learning law as its drift-based instance. The method is evaluated on four DeepMind Control Suite domains for predictive accuracy and robustness to velocity measurement noise, against concurrent-learning baselines and an Adam replay control.

Critical strengths
  • Combining trajectory-based Neural ODE training with Lyapunov-certified concurrent learning is well motivated and, to my knowledge, new.
  • The analysis appears sound in the parts I checked, and the derivations are explicit and easy to follow.
  • The experiments support the main robustness claim: under velocity measurement noise, NODE-CL achieves the lowest median error on three of four benchmarks.
  • The paper is well written. I also appreciate the transparent reporting of the mixed clean-data results and of the gap between the theoretical conditions and the experimental settings.
Critical weaknesses
  1. I would appreciate clarification of how informative the certificate in Theorem 2 is in practice. If I understand correctly, projection already provides a parameter-error bound, while sufficiently informative historical data may yield a tighter bound through the theorem. Footnote 2 also acknowledges the conservativeness of the sufficient conditions and the use of performance-tuned gains. Could the authors provide a representative numerical diagnostic satisfying the stated conditions, comparing the radius in Eq. (27) with the projection-only bound? Reporting the excitation level through GSG_S and Lemma 11, or directly evaluating HSH_S via Eqs. (51)–(52), would help, with numerical estimates distinguished from certified bounds.

  2. NODE-replay is a useful control, but its regularization is not fully clear. Does it use projection, leakage, or comparable safeguards, and were these choices considered in tuning? If the configurations differ, a reasonably tuned replay baseline with comparable regularization, where appropriate, would help establish how much the proposed design adds beyond trajectory-based replay. A reasonably tuned constant-gain version of NODE-CL, analogous to the CL/CL-LS pair in Table 1, would also help isolate the contribution of the dynamic gain.

  3. The velocity-noise results support an important benefit of NODE-CL. The excitation-richness ablation in Fig. 7(b) is useful, although the authors report weak, non-monotonic variation in the excited dimension. Comparisons with classical CL methods across different measured levels of historical excitation would further clarify the tradeoffs. On one representative system, could the authors vary the informativeness of the initial data and then apply a common weakly exciting input? Reporting an excitation measure and prediction errors for NODE-CL and CL/CL-LS under comparable, explicitly stated storage budgets (for example, following the equal-memory variant in Table 2) would help identify where NODE-CL is more or less competitive.

If the authors can satisfactorily address these points through clarification or additional evidence, I would be happy to raise my score.

Questions for authors

The main requests are in the Weaknesses section. One smaller question:

The scalar gate ςΓ\varsigma_\Gamma in Eq. (10) attenuates the Gauss–Newton term in the gain dynamics across all directions when λmin⁡(Γ)\lambda_{\min}(\Gamma) approaches the floor. Beyond the gate ablation in Fig. 7(a), how often is ςΓ<1\varsigma_\Gamma < 1 in the NODE-CL benchmark runs? Was gating individual eigendirections considered?

5 comments
AuthorsOpenReview ↗

Response to Reviewer 9s1h (1/3)

Thank you for stating precisely what would change your assessment. We take the requests in order.

W1: how informative Theorem 2 is, with a numerical diagnostic. Part B of the general response gives the diagnostic on S1 seed 0, and Appendix L.5 adds it to the paper. The answers to your specific questions:

  • A run satisfying the stated conditions: with closed-form upper bounds on the constants, Ts=2T_s = 2 ms and ϖ=1\varpi = 1, containment holds and Eq. (25) reads kσ>0.076k_\sigma > 0.076; the run at kσ=0.084k_\sigma = 0.084 is stable and the leakage holds the estimate at the origin.
  • Radius of Eq. (27) against projection: your reading is correct. For that run Eq. (27) at φs=0\varphi_s = 0 gives 94, against 13.4 from projection, and S1 gives no positive excitation margin: λmin⁡(HS)\lambda_{\min}(H_{\mathcal S}) and λmin⁡(GS)\lambda_{\min}(G_{\mathcal S}) are zero to machine precision at each of the 20 evaluated instants (13 to 14 of 25 directions excited). S1 therefore does not demonstrate a parameter radius tighter than projection; Eq. (27) improves on projection only through φs>0\varphi_s > 0.
  • HSH_{\mathcal S} against GSG_{\mathcal S} (measured on the tuned run at 20 instants between 25 s and 120 s): the largest $\Vert G_{\mathcal S} - H_{\mathcal S}\Vert $ is 9×10−99\times10^{-9}, a relative 4×10−54\times10^{-5}, against the Lemma 11 bound evaluated along the run, 1.3×10−31.3\times10^{-3}. The Gauss-Newton matrix built from batched sensitivities approximates the trajectory operator of the analysis to that accuracy.
  • What the theorem contributes at the excitation the benchmark provides: the admissibility of the Gauss-Newton gain (for α>1/2\alpha > 1/2 the trajectory gradient dominates the HSH_{\mathcal S} part of the term the gain dynamics add to the Lyapunov derivative, and Lemma 11 bounds the mismatch), the dependences of rate and radius on excitation, horizon, and noise, and, new in the revision, a bound on the time-averaged prediction error over the stored segments that needs no excitation (Proposition 7, under the conditions of Theorem 2). On S1 NODE-CL shows the structure the analysis describes: over five seeds the full parameter error ratio is 0.79, the excited-subspace component contracts to 0.29, and the functional error falls to the 10−310^{-3} level.

W2: regularization of NODE-replay, its tuning, and a constant-gain NODE-CL. NODE-replay in the submission is Adam with the same admission rule, segment budget, segment length, and update schedule as NODE-CL (each method's stored set evolves with its own estimate), with no projection or leakage. All numbers below are from one machine (Table 4 reruns Table 1 there).

Learning rate. It was selected on the diagnostic system S2 and held fixed. A sweep over {10−3,3×10−3,10−2,3×10−2,10−1}\{10^{-3}, 3\times10^{-3}, 10^{-2}, 3\times10^{-2}, 10^{-1}\} on all four domains (Appendix L.3) shows that the Table 1 value 3×10−23\times10^{-2} is the best on the grid on every domain with clean data. With noise, the best value on the grid per domain, chosen after the fact on the evaluation seeds, gives 0.120, 0.653, 0.899, 0.078 on pendulum, cartpole, acrobot, reacher, against 0.043, 0.302, 1.474, 0.075 for NODE-CL at one fixed setting. NODE-CL keeps the lowest noisy median on the pendulum and cartpole (2.8x and 2.2x) and on the reacher (0.075 against 0.078); on the acrobot, five seeds do not separate the two methods, so we ran twenty (Table 6): at the Table 1 settings the medians are 0.953 against 0.977 (clean) and 1.253 against 1.108 (noise) with overlapping interquartile ranges, and with the NODE-CL gain cap selected on five separate tuning seeds (λˉ=104\bar\lambda = 10^4) NODE-CL has the lowest noisy median of all methods (0.916). The best NODE-replay rate under noise differs by two orders of magnitude across domains; the NODE-CL numbers of Table 1 use one gain setting on all four domains.

AuthorsOpenReview ↗

Response to Reviewer 9s1h (2/3)

Regularization. NODE-replay with radial clipping to Θϵ\Theta_\epsilon after each Adam step and leakage kσ=10−9k_\sigma = 10^{-9} is now a row of Table 4: medians 0.200, 0.368, 1.577, 0.077 (clean) and 0.047, 0.741, 1.895, 0.083 (noise), against 0.178, 0.311, 1.005, 0.077 and 0.198, 0.741, 1.823, 0.089 for the plain version on the same machine. The clipping activated in no update of any run (largest $\Vert \hat\theta\Vert $: 62; boundary: 63), and the leakage changes each step by a relative 2×10−112\times10^{-11}. The paired trajectories of the two variants agree to 10−710^{-7} for the first 20 s and separate once the segment set is full and their admission decisions begin to differ, after which individual seeds move in both directions. Five seeds do not resolve the medians of this method, so we repeated the pendulum with twenty seeds for all methods (Table 5):

pendulum, 20 seedsclean medianNODE-CL lower onnoise medianNODE-CL lower on
CL0.347200.53620
CL-LS0.171190.75919
NODE-replay0.140170.12514
NODE-replay, projection + leakage0.157170.11013
NODE-CL0.0370.099

The two NODE-replay variants agree to within 12 percent at both noise levels. With clean data NODE-CL is about 4x below both; with noise, at the Table 1 settings, its margin over them is 1.1x to 1.3x, with a lower error on 13 to 14 of 20 seeds, and its margin over the observer-based methods is 5.4x to 7.7x.

Under noise, every method was then retuned on the same twenty seeds over a grid in its own gains, with the best median on the grid reported (Appendix L.1): the learning rate for NODE-replay (five values), kfk_f for CL and CL-LS (five values), and the gain cap, segment length, learning gain, and forgetting rate for NODE-CL (eighteen settings). The best medians are 0.125 for NODE-replay (its Table 1 learning rate remains the best at twenty seeds), 0.355 for CL, 0.572 for CL-LS, and 0.058 for NODE-CL (a slower forgetting rate, β0=0.1\beta_0 = 0.1), a margin of 2.2x over NODE-replay and 6x to 10x over the observer-based methods. On five separate tuning seeds the Table 1 setting of NODE-CL is the best of the eighteen, so its Table 1 numbers are not the product of selection on the evaluation seeds.

Constant-gain NODE-CL (Γ≡Γ0\Gamma \equiv \Gamma_0, the counterpart of the CL / CL-LS pair; the best Γ0\Gamma_0 on the grid {102,103,104,3×104,105}\{10^2,10^3,10^4,3\times10^4,10^5\} per domain, selected on an evaluation seed, so the variant is the one given access to evaluation data): medians 0.293, 0.841, 1.805, 0.505 (clean) and 0.308, 0.764, 1.437, 0.546 (noise), that is, 8.2, 2.2, 1.3, 6.8 and 7.2, 2.5, 1.0, 7.2 times the NODE-CL medians. The dynamic gain is the component that carries the method on the pendulum and reacher and matters least on the acrobot.

Section 4.1 now states the mechanism: for a fixed GSG_{\mathcal S} and an inactive gate, the gain dynamics have the equilibrium Γ=(λˉ−1I+GS/β0)−1\Gamma = (\bar\lambda^{-1}I + G_{\mathcal S}/\beta_0)^{-1}, that is, Γ≈β0GS−1\Gamma \approx \beta_0 G_{\mathcal S}^{-1} in the excited directions and the cap λˉ\bar\lambda elsewhere, a capped Gauss-Newton step, whose advantage over Adam's diagonal normalization is the off-diagonal and scale information in GSG_{\mathcal S}. The advantage is largest on the pendulum, where GSG_{\mathcal S} is worst conditioned (3×10153\times10^{15} against 10910^{9} to 101110^{11}).

AuthorsOpenReview ↗

Response to Reviewer 9s1h (3/3)

W3: excitation level of the stored data, against CL / CL-LS at stated memory budgets. Carried out as you describe (Appendix L.4, Table 8). Pendulum; multisine amplitude scaled by 0.25, 0.5, 1 during the first 10 s, then the same weak input (0.25) for all runs until 60 s; three seeds. CL and CL-LS store 100 pairs (400 scalars). NODE-CL is run at two budgets: 100 segments of four samples (the same number of stored items; 1200 scalars) and 33 segments of four samples (396 scalars; the same stored data, not counting the sensitivity cache and gain matrix). Excitation measure: number of eigenvalues of the stack regressor (ΨD\Psi_{\mathcal D} or GSG_{\mathcal S}) above 10−610^{-6} of the largest, out of 73, at 10 s / 60 s.

initial excitationCL dimCL errorCL-LS dimCL-LS errorNODE-CL (100) dimerrorNODE-CL (33) dimerror
low42 / 270.29630 / 210.17221 / 180.06020 / 60.190
medium41 / 390.67431 / 300.19423 / 180.07420 / 60.200
high41 / 280.40433 / 310.27024 / 190.03122 / 60.122

The table shows where NODE-CL is more and less competitive. With the same number of stored items it has the lowest median error at every level, by factors of 2.6 to 8.7 over CL-LS and 4.9 to 13 over CL. With the same stored scalars, one third as many segments, it stays below CL at every level (factors of 1.6 to 3.4), is level with CL-LS at the low and medium levels (0.190 and 0.200 against 0.172 and 0.194), and is 2.2x below CL-LS at the high level. By the eigenvalue-count measure, the segment stack spans fewer directions than either point stack at every level. The other regime where NODE-CL is less competitive is clean data on the cartpole (Table 1), where it is within a factor of 1.6 of the best method.

Q: how often ςΓ<1\varsigma_\Gamma < 1; per-eigendirection gating. Never in the benchmark: over all NODE-CL runs of Table 4, λmin⁡(Γ)\lambda_{\min}(\Gamma) stayed above 2λ‾2\underline\lambda at every update, so the gate was identically one. The gate is the mechanism that makes the lower gain bound of Lemma 4 hold for all time, and it acts only in the stress test of Figure 7(a). The active constraint in the benchmark is the cap λˉ\bar\lambda: it binds in 36 percent of the updates on the pendulum and reacher and in a median of 0 percent on the cartpole and acrobot (Appendix L.3). Per-direction gating was considered. With MM denoting the gated term, the Lyapunov step of Theorem 2 needs only 0⪯M⪯GS0 \preceq M \preceq G_{\mathcal S}, and Lemma 4 needs MM to vanish along eigenvectors of Γ\Gamma at the floor; the scalar gate satisfies both, and since it never activates at the benchmark settings, a matrix gate would not change any reported number. Section 4.1 now states the two requirements.

Reviewer_eMKaOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies online identification of smooth nonlinear dynamics using trajectory losses. It decomposes the negative Neural ODE trajectory gradient into a positive-semidefinite trajectory operator acting on parameter error plus an explicitly bounded perturbation. This enables Lyapunov analyses of drift-window and adjoint-window updates and of NODE-CL, which replays stored trajectory segments and uses forward sensitivities to form a Gauss-Newton gain without derivative estimates. Under compact-domain, containment, and gain conditions, the continuous-time parameter flow is uniformly ultimately bounded; excitation improves the bound. Four small DeepMind Control Suite benchmarks test prediction accuracy and velocity-noise robustness. NODE-CL has the lowest noisy median on three domains, while clean-data results are mixed

Critical strengths
  • The theoretical part of the paper assumes basic hypotheses that are in line with conventional adaptive control and system identification, and provides a theoretical proof of convergence.
  • By combining technologies such as Concurrent Learning and Gauss-Newton, a NODE-CL method without the need for state derivative estimation was proposed.
  • Its relative effectiveness was verified in the DeepMind Control Suite experiments. At the same time, comparative experiments were conducted under different time windows, different gains, and different holding methods, as well as different regularization intensities.
Critical weaknesses
  • The stability guarantees depend on restrictive, difficult-to-verify conditions, while experimental gains are tuned without ensuring these conditions are satisfied.
  • The analysis assumes bounded system states and only guarantees bounded parameter errors, without establishing closed-loop stability in practical control applications.
  • The paper lacks systematic ablations of NODE-CL’s key components, including historical trajectory replay, Gauss-Newton adaptation, and segment selection, making it difficult to assess their individual contributions to performance and stability.
Questions for authors
  • How can the restrictive theoretical conditions be verified in practice, and do the experimentally tuned gains satisfy these conditions?
  • Can the proposed framework provide closed-loop stability guarantees beyond bounded parameter estimation errors?
3 comments
AuthorsOpenReview ↗

Response to Reviewer eMKa

Thank you for the review. Direct answers first, then the details.

Q1 / W1: how the conditions are verified, and whether the tuned gains satisfy them. They are verified by bounding the first and second derivatives of the network over the parameter and state sets. Part B of the general response and the new Appendix L.5 do it in closed form on the teacher system S1 and exhibit gains that satisfy the hypotheses of Theorem 2, with the corresponding run. The tuned gains do not satisfy the sufficient gain inequality, Eq. (25); they are far below it, and every run is stable. A gap of that kind is expected, and Part B explains where it comes from: a worst case over a deliberately large parameter set, worst-case slopes that multiply, and norm bounds that assume adversarial alignment. The same holds for the certified baselines, CL and CL-LS. The relation is the one between the step-size bound η≤1/L\eta \le 1/L in convergence proofs for gradient methods and the learning rates used to train networks. At the tuned gains, two guarantees still hold unconditionally (the estimate stays in its set and the gain matrix stays bounded), and Part B lists what Theorem 2 adds and the experiment that tests each item.

Q2 / W2: closed-loop stability, and "only bounded parameter errors". Two points. First, the guarantees are stronger than boundedness: exponential convergence of the parameter error to a radius that shrinks with the excitation of the stored data (Theorem 2), and a bound on the time-averaged prediction error that needs no excitation (Proposition 4 for the drift law, new Proposition 7 for NODE-CL). Second, the problem solved is identification from a trajectory generated independently of the estimate, which is the setting of the Neural ODE literature the paper addresses and of system identification generally; a bounded state is the standard hypothesis there (Assumption 1). A closed-loop result requires a controller-specific analysis in which state boundedness and the model mismatch are handled jointly. In certainty-equivalence designs the identifier's error bound is the quantity that analysis consumes, as in the composite arguments for the concurrent-learning identifier (Patil et al., 2022; Hart et al., 2025a), and Theorem 2 supplies that bound for trajectory-based learning. The composition itself is future work, and the revised conclusion says so.

W3: ablations of the components.

  • Gauss-Newton adaptation (new, Section 4.1 and Table 4): NODE-CL with a constant gain, segments, projection, and leakage kept, with the best gain per domain from {102,103,104,3×104,105}\{10^2,10^3,10^4,3\times10^4,10^5\}, selected on an evaluation seed, so the ablated variant is the one given access to evaluation data. The median error changes by factors of 8.2, 2.2, 1.3, 6.8 (clean) and 7.2, 2.5, 1.0, 7.2 (noise) on pendulum, cartpole, acrobot, reacher.
  • Selection of the retained history (new, Appendix L.3): the same gradient, gain, segment length, and update schedule, with the most recent segments retained (first-in-first-out) in place of the segments selected by the admission rule. Selection lowers the error by factors of 2.1 to 2.8 on the pendulum and cartpole and makes no difference on the acrobot and reacher.
  • Replay against no replay (Table 2): the sliding-window adjoint law of Algorithm 2 retains no data beyond the current window; it also uses a constant gain and a longer window, so the comparison is not a clean isolation. NODE-CL lowers its functional error from 2.9×10−22.9\times10^{-2} to 3.4×10−33.4\times10^{-3} on S1 and from 8.9×10−28.9\times10^{-2} to 1.0×10−21.0\times10^{-2} on S2.

Every ablated variant remains bounded, as the theory predicts: boundedness does not rely on the Gauss-Newton gain or on the admission rule; the admission rule acts on the excitation margin φs\varphi_s of the stored segments and the gain on the scaling of the update, and both show their effect in the rate and the radius.

Thank you for the detailed response and additional experiments. Given that the gap between the sufficient theoretical gain conditions and the experimentally tuned gains remains substantial, I will maintain my original score.

AuthorsOpenReview ↗

The value of the stability analysis is in constructive synthesis and qualitative insights

Thank you for considering our response and additional experiments. As we understand it, the question underlying your comment is: what is the point of the theoretical analysis and gain conditions, given the gap with experimentally tuned gains?

The primary value of our Lyapunov analysis is constructive and qualitative. It was never meant or claimed to provide a numerical gain-tuning procedure. The analysis is not just an attempt to certify an existing algorithm. The trajectory-error decomposition and subsequent Lyapunov derivations directly enable the construction of NODE-CL, a new method. The resulting conditions also qualitatively characterize how excitation, horizon, noise, and adaptation gains influence convergence rates and ultimate bounds. Their numerical conservatism does not negate these contributions.

The gap between worst-case sufficient gains and performance-tuned gains is common to all Lyapunov-based adaptive methods, including the certified baselines CL and CL-LS, and no revision of any such method can remove it. We understand that the reviewer expects experimentally tuned gains to satisfy the sufficient theoretical conditions, but this is a standard that no existing work in this area meets. The revision shows that the conditions are satisfiable (Appendix L.5) and tests those dependencies experimentally. We ask that the paper be assessed on these contributions.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General response (1/3)

We thank the reviewers for careful and constructive reviews. Section, table, and equation numbers refer to the revised PDF, in which every change is typeset in blue. Each weakness asks for a specific experiment, derivation, or clarification, and the revision responds to each one.

Part A. What is new in the revision

Experiments (Appendix L.1 to L.5, summarized in Section 4). The new experiments were run on a second machine. Table 4 reruns all five methods of Table 1 there and lists the two ablated variants: the method with the lowest median is the same in all eight columns, and every comparison that involves the new experiments uses the same-machine values.

  1. Component ablations, four domains, five seeds, clean and noisy. Removing the Gauss-Newton gain from NODE-CL (constant gain, everything else kept) changes the median error by factors of 8.2, 2.2, 1.3, 6.8 (clean) and 7.2, 2.5, 1.0, 7.2 (noise) on pendulum, cartpole, acrobot, reacher. Replacing the segment-admission rule by first-in-first-out costs a factor of 2.1 to 2.8 on pendulum and cartpole and is neutral on acrobot and reacher. In NODE-replay with the clipping and leakage of NODE-CL, the clipping never activates.
  2. Twenty-seed study on the pendulum, all methods (Table 5). Clean: NODE-CL median 0.037 against 0.140 (NODE-replay), 0.171 (CL-LS), 0.347 (CL), lowest error of the five methods of Table 1 on 17 of 20 seeds. Noise: 0.099 against 0.125, 0.759, 0.536, lowest error of the five on 14 of 20 seeds. With every method retuned on the twenty seeds over a grid in its own gains: 0.058 against 0.125, 0.572, 0.355. Twenty-seed study on the acrobot, all methods (Table 6): NODE-CL and NODE-replay are level (0.953 against 0.977 clean, 1.253 against 1.108 noise), and both are ahead of CL-LS (1.365, 1.760), CL (1.841, 2.791), and the single-step law.
  3. Observer gain kfk_f retuned at the noisy level for CL and CL-LS over {50,100,200,400,800}\{50,100,200,400,800\}. With the best value on the grid per domain, CL-LS remains above NODE-CL by factors of 13, 2.7, 1.3, 1.8 and CL by 8.4, 3.0, 1.5, 13; NODE-CL has no such gain to retune.
  4. Learning-rate sweep for NODE-replay: the Table 1 value is the best on the grid on every domain with clean data.
  5. Excitation-level study (pendulum, three initial-excitation levels, common weak input afterwards) at two stated memory budgets.
  6. Gate, cap, and conditioning statistics of the NODE-CL runs.
  7. Closed-form evaluation of the conditions of Theorem 2 on S1 with a run that satisfies them, and a direct measurement of $\Vert G_{\mathcal S} - H_{\mathcal S}\Vert $ against the Lemma 11 bound (Appendix L.5).

Theory. Proposition 5 (Appendix I): with bounded measurement noise, the trajectory-error decomposition holds with an operator H⪰0H \succeq 0 of the same construction and the same b1b_1; b2b_2 becomes affine and b3b_3 quadratic in the noise bound, so Proposition 3 and Theorem 2 extend to bounded measurement noise. Proposition 7 (Appendix K): under the conditions of Theorem 2, the time-averaged prediction error on the stored segments is bounded with no excitation requirement.

Text. Mechanism behind the clean-data results (Section 4.1); how the bounds are to be read (Section 4.2); what is new in the proof, and the relation to integral concurrent learning (Section 1); relation to closed-loop guarantees (Section 5).

AuthorsOpenReview ↗

General response (2/3)

Part B. The gain conditions (eMKa W1/Q1, 9s1h W1, L111 W1)

A verified instance of feasible gains (new Appendix L.5). On S1 seed 0 (θ∗\theta^\ast known, εˉ=0\bar\varepsilon = 0) we derived closed-form upper bounds on the five derivative constants of the tanh network over Ω×Θϵ\Omega\times\Theta_\epsilon (each lies above the maximum found by sampling 10410^4 parameters). With margin ϖ=1\varpi = 1 and segment length Ts=2T_s = 2 ms, the containment condition Eq. (17) holds (left side 0.78), and Eq. (25) at α=2\alpha = 2 reads kσ>0.076k_\sigma > 0.076. NODE-CL run with kσ=0.084k_\sigma = 0.084 and Γ∈[10,100]\Gamma \in [10, 100] therefore satisfies the hypotheses of Theorem 2, and the run is stable. A leakage of that size holds the estimate at the origin, and Eq. (27) at φs=0\varphi_s = 0 evaluates to 94 against 13.4 from projection. Feasible gains exist and are computable from the constants of the theorem; they are not the gains one runs. Appendix L.5 reports the constants, the instance, and the run; the scripts and logs behind every number in Part B are in the supplementary material (cert_*.py).

The same condition at the tuned configuration (Ts=50T_s = 50 ms, kσ=10−9k_\sigma = 10^{-9}). With constants maximized over the sampled parameters, Eq. (25) requires kσ>1.3×105k_\sigma > 1.3\times10^{5} (more with the closed-form bounds, and containment over the whole of Θϵ\Theta_\epsilon needs shorter segments, as above). With the constants and the error bound taken along the realized run, the same formula gives 1.2×10−21.2\times10^{-2}. The run uses 10−910^{-9} and is stable, and on its stored segments the rollouts stay within 1.3×10−31.3\times10^{-3} of the measured trajectory.

Why that gap is expected. Eq. (25) asks how much leakage keeps the estimate bounded if the network is as steep as it can be anywhere in the admissible parameter set and the linearization error always points in the most harmful direction.

  1. The set is deliberately large. Θϵ\Theta_\epsilon is a ball of radius 6.9, more than twice the norm of θ∗\theta^\ast (3.2), so that it certainly contains θ∗\theta^\ast; the parameter error is then only known a priori to be below 13.4. The network is far steeper at the edge of that ball than near θ∗\theta^\ast, and a guarantee over the whole set pays for it even though the estimate never goes there.
  2. Worst cases multiply. The constants in Eq. (25) are products of slope constants and powers of the error bound. The slopes are 5 to 12 times larger over the set than along the run and the error bound is 12 times the realized error; replacing them by their values along the run removes seven orders of magnitude. The effect is the one familiar from Lipschitz bounds for deep networks obtained by multiplying layer norms.
  3. Norm inequalities take the perturbation to be aligned against the estimate at every instant, which adds further conservatism. One piece of it can be measured (Appendix L.5): the largest $\Vert G_{\mathcal S} - H_{\mathcal S}\Vert $ over 20 instants between 25 s and 120 s of the run is 9×10−99\times10^{-9} (a relative 4×10−54\times10^{-5}), with HSH_{\mathcal S} evaluated from Eqs. (51) and (52), against a Lemma 11 bound of 1.3×10−31.3\times10^{-3} evaluated along the run.

How gain conditions are read in nonlinear and adaptive control. In the nonlinear and adaptive control and estimation literature, a Lyapunov theorem and its gain conditions are typically not used as a numerical recipe for selecting the gains of a particular run. The constants are worst-case by construction, so the sufficient gains, although computable as in the instance above, are conservative for every method analyzed in that way, and the gains that are run are tuned. The value of the theorem lies in two other things. It proves that the update law is stable for the stated class of systems under an explicit sufficient condition. It also shows how each gain and design parameter moves the convergence rate and the ultimate bound, which is the information that guides tuning. In Theorem 2, the rate grows linearly with the decay coefficient ks(φs)k_s(\varphi_s), which increases with the excitation margin φs\varphi_s of the stored segments, and with the gain floor λ‾\underline\lambda, the radius shrinks as 1/ks(φs)1/\sqrt{k_s(\varphi_s)}, every perturbation constant shrinks with the segment length through TT and κT\kappa_T, and measurement noise enters integrated over the segment. The certificates of the certified baselines in Table 1, CL and CL-LS (Chowdhary and Johnson, 2011; Hart et al., 2025a; Proposition 2 here), are read in the same way: they use the same kind of worst-case constants and their gains are tuned. On S1 the drift-pathway perturbation is a factor of twelve inside its bound for a single inequality (Appendix L, check (vi)). NODE-replay and the single-step baseline have no certificate.

AuthorsOpenReview ↗

General response (3/3)

An analogy for readers outside control theory. For readers without a control-theoretic background, who may come to this public discussion from machine learning, the author(s) emphasize that the point of Lyapunov-based gain conditions for nonlinear systems is to give users a qualitative insight on gain tuning instead of a quantitative numerical procedure. As an analogy, consider the convergence proofs of gradient methods in machine learning or asymptotic complexity analysis for algorithms. Convergence proofs for gradient methods assume a step size η≤1/L\eta \le 1/L, where LL is a global smoothness constant. The learning rate of a deep network is tuned and not set from a certified LL, and the proof remains the result that establishes convergence under its condition and shows how the rate depends on the smoothness and conditioning of the problem. Similarly, asymptotic complexity bounds in algorithm analysis are also too conservative, e.g., an O(N2)O(N^2) time complexity bound does not predict the actual running time of a program on a given machine. Does that mean the big O complexity analysis is not useful? No, because it still gives users a qualitative understanding of how the cost scales. Eq. (25) and Eq. (27) are read in the same way.

What holds at the tuned gains, and what Theorem 2 contributes. Two guarantees do not depend on Eq. (25): the estimate stays in Θϵ\Theta_\epsilon (projection) and the gain matrix stays between its bounds (Lemma 4). Both were checked on every run: the projection is violated in no update, and the gain bounds hold at every logged instant. Under its gain condition, Theorem 2 adds the following; the experiments at the tuned gains test each dependence.

  • The Gauss-Newton gain is admissible. For every α>1/2\alpha > 1/2 the trajectory gradient dominates the HSH_{\mathcal S} part of the indefinite term that the gain dynamics add to the Lyapunov derivative, and the remaining mismatch GS−HSG_{\mathcal S} - H_{\mathcal S} is bounded by Lemma 11 and absorbed by the gain condition. The constant-gain ablation shows what the gain buys: up to 8.2x.
  • Convergence is exponential to a radius that scales as 1/ks(φs)1/\sqrt{k_s(\varphi_s)} with the excitation of the stored segments. On S1 the NODE-CL error component in the excited subspace contracts to 0.29 of its initial value while the full parameter error ratio stays at 0.79 (five seeds).
  • The bounds depend explicitly on the horizon, through factors of TT and the Gronwall factor κT\kappa_T, and every perturbation constant vanishes as the segment length goes to zero. The adjoint-window error grows 3.7-fold when the horizon doubles.
  • Measurement noise enters additively and integrated over the segment, not differentiated (Proposition 5). In Table 1 the CL-LS median rises by 3.3x to 8.4x under velocity noise on three domains while the NODE-CL median changes by at most 1.1x; over twenty pendulum seeds the factors are 4.4 and 2.7 (Table 5).

Radius against projection on S1. λmin⁡(HS(t))\lambda_{\min}(H_{\mathcal S}(t)), evaluated from Eqs. (51) and (52), and λmin⁡(GS(t))\lambda_{\min}(G_{\mathcal S}(t)) are both zero to machine precision at each of the 20 evaluated instants (Appendix L.5), with 13 to 14 of 25 directions excited. So φs=0\varphi_s = 0 and Eq. (27) does not improve on projection on this system. The theorem says as much: the radius improves on projection only through φs>0\varphi_s > 0, and parameters cannot be identified in directions the data do not excite. What Proposition 7 bounds without excitation, under the conditions of Theorem 2, is the prediction error on the stored segments: the time average of the trajectory-regressor energy satisfies lim sup⁡S→∞1S∫θ~⊤HSθ~ dr≤ca/(α−12)\limsup_{S\to\infty}\frac{1}{S}\int \tilde\theta^\top H_{\mathcal S}\tilde\theta\,dr \le c_a/(\alpha - \tfrac12) (Eq. (69)), and the time-averaged squared prediction error over the stored segments is bounded by 2ca/(α−12)2c_a/(\alpha - \tfrac12) plus a remainder proportional to TsT_s (Eq. (70)).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.