acceptodds
Under review as a conference paper at ICLR 2027

AugRelNet: Relation-Augmented Dynamics for Structure Discovery and Forecasting from Limited Data

Abstract

Complex dynamics often arise from persistent interactions whose effects vary with the system state. We introduce AugRelNet for short- and medium-horizon trajectory forecasting from limited data. A shared learnable relation matrix explicitly routes source signals to targets, while evolving local features and a persistent response state transform them into nonlinear, history-dependent responses. Feeding these responses back into the dynamics separates shared relational organization from its changing realization. The learned relations reflect dense coupling in Lotka–Volterra systems and develop structured, neighbour-preferential organization in locally interacting systems. On 40-dimensional Lorenz–96, AugRelNet uses about 4,000 training states, roughly an order of magnitude fewer than representative prior Lorenz–96 forecasting studies. Across five seeds, every true interaction ranks above every nonedge, with a mean absolute-weight contrast of approximately . Under the same data budget, eight-step RMSE is about 23–59× lower than MNO, DySLIM, DeepSkip, and AL-RNN, 14–55× lower than NRI and dense GraphODE, and 37% lower than PySINDy with a matched quadratic dictionary. The same relational representation extends to grids by treating sites as variables and sharing relations across spatial offsets: in discrete (Game of Life), continuous (FitzHugh–Nagumo), and stochastic (Ising) dynamics, learned weights rank physical neighbours above other candidate offsets while supporting multi-step prediction. AugRelNet learns these relations through prediction losses and regularization, without structural labels or system-specific physical losses.

Primary area:
learning on time series and dynamical systems
Keywords:
dynamical systems, neural forecasting, interaction structure learning, relation learning, multivariate dynamics, spatiotemporal dynamics, chaotic dynamics

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_Hsj4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces AugRelNet, a relation-structured neural dynamical model for forecasting and interaction discovery from limited observations. The framework learns a persistent relation matrix that represents interactions between variables, while augmented states capture nonlinear and history-dependent responses. The model is trained using prediction losses without requiring ground-truth interaction structures or system-specific physical constraints.

The authors evaluate AugRelNet on generalized Lotka–Volterra, Lorenz–96, FitzHugh–Nagumo, Game of Life, and stochastic Ising systems. On 40-dimensional Lorenz–96, the model recovers all 120 true directed interactions across five seeds using approximately 3,849 training states. It also substantially outperforms several forecasting and relation-learning baselines at short horizons. The spatial experiments further demonstrate recovery of local interaction structures across continuous, discrete, and stochastic dynamics.

Critical strengths
  • Clear separation between relational structure and dynamical responses. The proposed architecture explicitly separates persistent interaction pathways from their state-dependent realization. The use of augmented states, particularly the persistent response state, provides a flexible way to represent nonlinear and history-dependent interactions. The relation interventions and capacity-matched ablations help substantiate the importance of these components.

  • Strong forecasting and structure recovery under limited observations. The Lorenz–96 results are compelling. The model achieves exact recovery of all 120 directed interactions across five seeds without structural supervision, while substantially outperforming several baseline models under the same data budget. The additional experiments on Lotka–Volterra and spatial systems provide evidence that the relational representation can extend beyond a single dynamical system.

Critical weaknesses
  • Learned relations do not necessarily correspond to identifiable physical interactions. Although the recovery of Lorenz–96 interactions is convincing, the learned relation matrix represents predictive dependencies rather than uniquely identifiable physical or causal interactions. The nonlinear response functions and augmented states can potentially absorb or redistribute interaction effects. Moreover, the theoretical analysis of relation recovery relies on a frozen linearized model with unregularized gradient flow, rather than the actual nonlinear optimization procedure. I would like to see stronger evidence that the recovered structures remain meaningful under observational noise, partial observations, or different dynamical regimes.

  • The forecasting comparisons need stronger computational controls. The reported forecasting improvements are exceptionally large, particularly on Lorenz–96, where the model outperforms several established methods by more than an order of magnitude. However, the main AugRelNet checkpoints undergo 3,500 initial training epochs followed by approximately 13,000–16,000 additional epochs of optimization (Appendix D.2). While the baselines share the same training data, it is unclear whether they receive comparable optimization budgets, model capacities, or computational resources. A comparison under matched computational budgets would help establish how much of the improvement comes from the proposed architecture.

  • Strong short-term forecasting does not translate into accurate long-term dynamics. Despite substantial improvements in short-horizon RMSE, AugRelNet struggles to reproduce the invariant statistics of Lorenz–96. For example, its long-run mean energy is 12.68 compared with the reference value of 9.36, while DeepSkip remains considerably closer to the reference distribution (Appendix D.2.1, Table 9). The FitzHugh–Nagumo experiments also exhibit substantial degradation beyond the supervised forecast horizon. This raises concerns about whether the model learns the underlying dynamical system or primarily provides accurate finite-horizon approximations.

  • Limited evaluation of robustness and generalization. The experiments primarily involve synthetic dynamical systems with controlled interactions and fully observed states. Lorenz–96 is evaluated at a fixed forcing parameter and dimension, while the other systems also operate under relatively constrained settings. It is unclear whether the reported data efficiency and structure recovery extend to noisy, partially observed, or nonstationary systems. I would like to see experiments involving observation noise, missing variables, or parameter shifts, which are particularly relevant to real-world scientific dynamical forecasting.

  • The spatial experiments rely on relatively strong structural assumptions. While the method does not receive the exact interaction graph, its spatial extension assumes translation symmetry and restricts candidate interactions to predefined local neighborhoods. For example, FitzHugh–Nagumo and Game of Life use a radius-two stencil, while Ising considers only eight neighboring offsets. This substantially reduces the structure discovery problem. Moreover, the forecasting improvements are less consistent across these settings: PolyKAN performs better on Game of Life, while AugRelNet and CNN achieve nearly identical errors on Ising. Testing irregular interaction graphs or spatially heterogeneous systems would better establish the generality of the approach.

Questions for authors

See above.

5 comments
AuthorsOpenReview ↗

Identifiability Limits and Structural Selection

Q1, part 1: Identifiability and structural selection

Our revised theory separates what finite observations identify about physical interactions from why learning can reproducibly select benchmark structure. We will revise the original theory accordingly.

Identifiability. For n≥3n\ge3, every finite, uniformly sampled full-state dataset with unique observed successors and predecessors admits a smooth autonomous realization with any prescribed strongly connected dependency graph, fixed before observing the data. Self-dependence is allowed. The realization uses the original coordinates and no extra states; prescribed edges can have nonzero partial derivatives at every sampled state. No uniform function-complexity or Lipschitz bound is imposed.

Even exact prediction of finite noiseless data cannot certify the true physical graph or, without functional-form and normalization assumptions, GLV strength rankings.

Benchmark success may reflect a favorable alignment: inductive bias favors structures with properties common in benchmark generators, such as reusable interaction rules. When real systems share these properties and observations expose them, exploiting that bias can increase the probability of recovering their true physical structure, without guaranteeing recovery.

Structural selection. We develop a conditional principle for shared interaction representations under squared prediction loss. Under a product reference law, square-integrable channels are centered in each input and true dictionaries are orthonormal throughout learning. For Euclidean relation-gradient flow,

d˙e=Ck(t)be−Gk(t)de+ξe(t).\dot d_e=C_k(t)b_e-G_k(t)d_e+\xi_e(t).

Here ee indexes a candidate interaction channel of arity kk, beb_e its normalized generating coefficient, and ded_e its learned coefficient. GkG_k is the learned-feature Gram operator, CkC_k the learned/true-feature overlap, and ξe\xi_e the reference-gradient discrepancy. Relations of the same type share these evolving operators even as nonlinear features learn.

Present relations accumulate a common transformed signal, while absent relations lose initialization effects up to residuals. For nonempty true/non-edge sets and Gk⪰gIG_k\succeq gI along the path, g>0g>0, the score gap satisfies

Γ(t)≥a(t)bmin⁡−2r0e−gt−2δ(t).\Gamma(t)\ge a(t)b_{\min}-2r_0e^{-gt}-2\delta(t).

Scores group coefficient norms by participating variable; bmin⁡b_{\min} is the smallest true-edge group norm, r0r_0 the largest initial group norm, and a(t)a(t) the minimum gain of the shared accumulated transfer. The residual bound is

δ(t)=∫0te−g(t−u)∥ξ(u)∥2du,\delta(t)=\int_0^t e^{-g(t-u)}\|\xi(u)\|_2du,

with ξ\xi stacking channel discrepancies. A positive lower bound guarantees strict separation before accurate response fitting. The product law is a theoretical reference, not an independence claim for Lorenz trajectories.

With common positive scalar gain, strength rankings are preserved once initialization effects vanish; small residuals preserve rankings when strength gaps dominate. In a fixed-feature shared-rule subcase, sharing can increase the probability of jointly positive structural margins without changing expected prediction risk.

Connection to ARN. The identifiability result makes structural selection dependent on inductive bias; the gradient principle explains how shared representations can favor recurring interaction rules.

Lorenz-96, GLV and reaction-diffusion systems reuse low-order mechanisms despite different graph densities. ARN's DD organizes predicted dynamics through persistent named-variable routing. Shared source/receiver functions reuse mechanisms across relations; nonlinear responses and augmented states realize state- and history-dependent effects. Local-history initialization and a zero-aggregate response reference anchor this division.

ARN's contribution is a generative inductive bias: prediction is organized around reusable mechanisms and persistent relation coordinates. When this matches the generator and the data excite its mechanisms, shared learning can coordinate structural selection before detailed responses converge. The gradient principle motivates this relation–response–state design, while experiments test its realization in ARN. Reproducible selection does not certify unique physical coefficients.

Early structural selection. Across five independent runs, ARN fully separates true edges from non-edges at main epochs 10–20 (after conditioner warmup), while forecasting continues to improve. Five further runs without direct DD shrinkage also fully separate by epoch 300.

Full theorem statements and proofs are too lengthy for this response and will appear in the appendix of our revised manuscript in a subsequent update. Noise, partial observation and other experimental concerns will be addressed in our second reply.

Updated experimental overview. We acknowledge insufficient training in some original runs: GraphODE-Oracle improved with extended optimization. We reran Lorenz-96 with 4,000 training states and five independent datasets. All five ARN runs separate every true edge from non-edges at main epochs 10-20 after conditioner warmup, while forecasts continue improving. ARN substantially outperforms the tested models without supplied graph, locality or matched-library priors, except CLS at H1, and also locality-informed LocalRC. GraphODE-Oracle and Local DeepSkip forecast better, within a factor of a few of ARN.

Five-instance mean physical RMSE; lower is better:

\text{Method / prior}&\text{H1}&\text{H8}&\text{H40}\\\\ \hline \mathbf{ARN}\\ \text{/ none}&\mathbf{0.00729}&\mathbf{0.0489}&\mathbf{0.572}\\\\ \text{NODE / none}&0.348&2.306&3.453\\\\ \text{NRI / none}&0.202&1.164&3.567\\\\ \text{cMLP / none}&0.870&2.419&3.705\\\\ \text{Global DeepSkip / none}&0.106&0.791&3.923\\\\ \text{CLS / none}&0.00164&0.154&\text{NF (2/5)}\\\\ \text{LocalRC / locality}&0.00897&0.146&1.479\\\\ \text{Local DeepSkip / locality}&0.00177&0.0131&0.117\\\\ \text{GraphODE-Oracle / true graph}&0.00380&0.0253&0.183\\\\ \text{SINDy-Poly2 / quadratic library}&0.0000395&0.000300&0.00174 \end{array}$$ NF denotes two nonfinite runs; no finite-subset mean is used. (SINDy-Poly2: second-order differences originally; fourth-order now.) **Q1 (last part): Structure under noise and partial observation.** Gaussian noise corrupts training inputs/targets, scaled by each variable's clean training SD; validation stays clean. AUROC measures how often true edges outrank non-edges; AP summarizes precision as recall increases (1 is ideal for both). ARN uses one fixed recipe for 300 epochs (n=3 per level): AUROC/AP=1 at eight tested levels (3-75%); at 200%/300%, AUROC remains 0.9997/0.9499 and AP 0.9960/0.7881. SINDy-Poly2 is validation-tuned separately per noise level (54 configs plus confirmation): its AP is 1 at 3-50%, exceeds ARN at 100%, and is 0.9617/0.8246/0.4403 at 150/200/300%. With its clean-data settings fixed, SINDy AP falls to 0.2396 at 50% and about 0.083 at 200-300%. We also retrained with 36/40 and 30/40 variables observed (one instance each). AUROC/AP are 0.99998/0.99978 and 0.99795/0.97292, respectively. Both retain strong agreement with the visible-variable direct subgraph, although neither achieves strict separation. For 30/40 observation, H1 RMSE is 0.0155 versus 0.0077 with full inputs on the same 30 targets; H40 rises to 2.978 versus 0.590. Meaningful structural ranking thus survives missing variables while long-horizon prediction deteriorates. Hidden variables can induce effective predictive dependencies, so this is not a direct causal-identification claim. Parameter-shift and nonstationary regimes remain untested; these results support robustness within the stated observation conditions. **Q2: Computational controls and training fairness.** Gradient fits complete their declared optimization/LR stages, then require two consecutive 100-epoch windows with <0.2% improvement in historical best validation score. LR changes reset patience; no hard epoch/time cap applies. Testing uses validation-best checkpoints. cMLP, cLSTM and SyCo-AE follow this protocol; regression methods complete native solves. Validation-only tuning includes 60/70/108 configurations for Local/Global DeepSkip/LocalRC, further DeepSkip ridge refinement, 15 each for SINDy Poly2/Poly3, and 24 for CLS. Selection records are retained per method. Many baselines without supplied structural priors peak very early under our common stopping controls and can overfit with further training. For example, NODE, MNO and NRI reach validation optima at epochs 5, 10 and 15-40. NODE's mean H8 training-probe RMSE falls from 2.065 to 0.067 while validation worsens from 2.317 to 2.415. Comparisons use their earlier validation-best checkpoints. **300-epoch control.** Across three paired instances, ARN restricted to 300 main epochs achieves H8 RMSE 0.1077 versus 2.2801 (NODE), 2.3991 (cMLP), 1.1314 (NRI) and 2.1792 (DySLIM). Their full-run validation-best checkpoints already lie within 300 epochs. These gaps already appear without prolonged refinement. Equal epochs do not equal compute: a separate same-GPU, two-hour-budget comparison (n=3) gives H8 0.1112 for ARN and 0.0343 for true-support Oracle. Width/depth searches span 12.6k trainable parameters (ALRNN) to 10.5M fitted readout coefficients (Global DeepSkip, plus fixed random features); ARN has 1.29M and NRI 1.07M. Parameter counts are not forced equal. Over the next few days, we will present the remaining baselines in the revised paper and supplementary materials, together with a detailed reproducibility report covering code/data identities, seeds, configurations, searches, per-run training/validation curves, checkpoint identities and measured costs.

Q3: Long-horizon dynamics. ARN uses the same learned update to autonomously predict unseen test states over 40 steps, recovers the generating dependency pattern across independent datasets without structural supervision, and shows functional dependence on the learned relations in frozen-model interventions. Together, these results support learning reusable dynamics with meaningful interaction structure.

Five frozen 4k-data models give H40/H80/H160 RMSE of 0.581/2.985/21.597, versus Local RC's 1.498/3.331/4.773. RMSE is root mean squared prediction error; lower is better. H80 extends beyond ARN's H40 supervision, although GraphODE-Oracle and Local DeepSkip perform better throughout. Separately, the original 50k-data long-run experiment gives mean energy 12.68 versus the reference 9.36 and DeepSkip's 9.38.

ARN can learn where nonlinear interactions act and closely approximate finite-time evolution without enforcing Lorenz–96's exact energy balance:

E &= \frac{1}{2N}\sum_{i=1}^{N}x_i^2,\\\\ \frac{\mathrm{d}E}{\mathrm{d}t} &= -2E+\frac{F}{N}\sum_{i=1}^{N}x_i. \end{aligned}$$ The nonlinear terms redistribute energy and cancel in the total balance. Correct dependency locations alone do not enforce this quantitative cancellation. Systematic balance errors can shift long-run statistics, which our prediction loss does not explicitly constrain. For FHN, cross-site communication passes through the same offset weights $D$ across message channels. Making structure explicit thus constrains spatial mixing and may trade some predictive flexibility for a readable relation pattern. Correct neighbors do not fix reaction saturation, diffusion damping or the augmented states' behavior under repeated rollout. Errors in these quantities can amplify beyond the H16 training range; the new H32/H64 tests show substantial degradation, with ConvNODE performing better. The structural readout and finite-time accuracy therefore address different requirements from sustained, quantitatively faithful simulation. **Q4: Robustness and generalization.** We added Gaussian noise to training inputs and targets, scaled by each variable's clean training standard deviation, with clean validation. AUROC measures how consistently true interactions rank above noninteractions; AP (average precision) summarizes precision as more true edges are retrieved. Both are higher-is-better, with 1 ideal. With fixed hyperparameters and a 300-epoch budget, ARN achieves AUROC/AP=1 across three independent instances at each of eight noise levels from 3% to 75%. At 200%/300%, AUROC is 0.9997/0.9499 and AP is 0.9960/0.7881. Separately validation-tuned SINDy-Poly2 has AP 0.8246/0.4403 at these levels. We also retrained ARN under partial observation with the original validation-based stopping and checkpoint selection. With 36/40 and 30/40 variables observed, AUROC/AP are 0.99998/0.99978 and 0.99795/0.97292, respectively, with one instance per setting. Neither achieves strict separation. On the same 30 observed targets, H1 RMSE is 0.0155 versus 0.0077 with full inputs, while H40 is 2.978 versus 0.590. Strong agreement with the visible-variable direct subgraph persists despite larger forecasting errors: hidden variables remain dynamically active, adding influences that must be inferred from the observed history. These experiments address observation noise and missing variables; parameter shifts and nonstationarity remain untested.
Reviewer_h2rTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

AugRelNet is an augmented neural dynamical model that routes cross-variable signals through a shared learnable relation matrix D and realizes nonlinear, history-dependent responses through a freely evolving auxiliary state u and a persistent response state η driven only by the relational response, with the aim of separating where interactions are organized from how they are realized. The model is trained from prediction losses and regularization alone, without structural labels or system-specific physical losses. On Lorenz-96 in a low-data setting, it recovers the true interaction support without a topology prior and reports substantially lower short-horizon forecast error than existing forecasting and relation-learning baselines; on GLV it reproduces the dense interaction geometry.

Critical strengths
  1. On low-data Lorenz-96, every seed recovers exactly the true interaction support without any topology prior, and the opposite-sign pattern of the two neighbour terms in the Lorenz-96 equation is matched at every target.
  2. At short horizons, the mean RMSE is below that of GraphODE-Oracle, which receives the true graph, and PySINDy with the matched dictionary, so the model is competitive without strong priors.
  3. The intervention experiments (shuffling and zeroing D, resetting η, capacity-matched state ablations) are systematic and separate the functional roles of the components reasonably well.
  4. Experimental protocols and hyperparameters are documented in detail in the configuration table and the appendix.
Critical weaknesses
  1. There is no comparison against methods that perform structure inference and forecasting jointly, such as the cited higher-order Granger reservoir computing and causal local states approaches.
  2. Several baselines are worse than the paper’s own persistence reference, the PySINDy poly3 zero-field fit. MNO, AL-RNN, and Global ESN do not beat persistence at the shortest horizon, and DySLIM does not at the medium horizon, which suggests the baselines were not adequately tuned for this data regime.
  3. All benchmarks are fully observed synthetic systems without observation noise; there are no experiments with observation noise, partial observation, or real data, so the robustness of structure recovery is untested.
  4. Several short equations in the model section are typeset over two lines, and in the joint space-time spectrum figure the vertical axis extends far beyond the frequency range that carries any spectral mass, leaving most of each panel empty.
Questions for authors
  1. How do the support recovery rate and the separation margin change under moderate observation noise or partial observation?
  2. Why do several baselines fail to beat, or only barely beat, persistence, and how do the authors explain Neural ODE remaining nearly identical to persistence at the shortest horizon?
  3. For the low-data Lorenz-96 protocol, can the authors report the hyperparameter search range, parameter count, training epochs, and compute budget for each baseline, in particular Neural ODE, DeepSkip, and MNO, alongside those of AugRelNet?
  4. Does strict support separation persist when the no-shrinkage experiment, which removes both the explicit D regularizer and AdamW weight decay on D, is extended to all seeds?
3 comments
AuthorsOpenReview ↗

Robust Structure Recovery and Simple Inductive Biases

Q1. Structure recovery with noise and partial observation. We regenerated five independent 40-variable Lorenz-96 trajectories (dt=.02), each split into 4000/1000/1000 train/validation/test points. ARN and every baseline were trained/refitted entirely from scratch on the new trajectories, with independent data, initialization and training seeds. Methods share splits and train-only normalization within instances; no test selection. ARN receives neither the true graph nor a matched equation library.

Result: exact recovery from short, noisy chaotic trajectories. A single frozen ARN configuration, trained for only 300 epochs, achieves 24/24 exact support separations across 3/5/10/15/25/35/50/75% measurement noise (3 runs/level). AUROC=AP=1; the minimum separation margin is .03855: all 120 true directed edges rank above all 1440 false ones. Noise SD is scaled by each coordinate's clean training SD. Training inputs and targets are noisy; validation stays clean for LR scheduling.

High-noise results (n=3/level; mean ± sample SD). AP = average precision, summarizing precision–recall across score thresholds; exact = strict separation of all true/false edges.

\text{Noise}&\text{ARN AP}&\text{SINDy AP}&\text{Exact}\\\\ 150\\%&.999954\pm.000080&.961657\pm.021723&2/3\\\\ 200\\%&.996004\pm.004307&.824635\pm.005217&0/3\\\\ 300\\%&.788067\pm.051801&.440321\pm.022747&0/3 \end{array}$$ At 100% noise, exact recovery is 2/3 and tuned SINDy has higher AP. At matched recall (fraction of true edges recovered): $$\begin{array}{lrrrr} \text{Noise}&\text{Recall}&\text{TP}&\text{ARN FP}&\text{SINDy FP}\\\\ 100\\%&80\\%&96&13&0\\\\ 200\\%&80\\%&96&.33&27.67\\\\ 300\\%&40\\%&48&2&22 \end{array}$$ **TP/FP** = true/false positive edges (n=3 means). Thresholds are post-hoc evaluation cutoffs. SINDy uses a matched quadratic library with 54 development trials and three independent confirmations per noise level; ARN keeps its configuration and 300-epoch budget. With 36/40 variables observed (n=1), AUROC/AP=.999982/.999784; margin=-.00563. **AUROC** = area under the ROC curve; margin = lowest true-edge score minus highest false-edge score (excluding diagonals). **Critical Weakness 1. Joint structure/forecasting comparisons.** We reran higher-order Granger reservoir computing (HoGRC, adapted author code) and Causal Local States (CLS, paper reconstruction) for **both structure and forecasting**, using the same 4000-point protocol. LocalRC remains a separate forecasting comparator. Clean-data structure results (n=5 means): $$\begin{array}{lrrrr} &\text{AUROC}&\text{AP}&\text{Native F1}&\text{Native SHD}\\\\ \text{ARN}&1&1&-&-\\\\ \text{SINDy-Poly2}&1&1&1&0\\\\ \text{SINDy-Poly3}&.999984&.999809&.142857&1440\\\\ \text{Lasso-Poly2}&1&1&.144633&1419.4\\\\ \text{Group-VAR Granger}&.968535&.717082&.143627&1431\\\\ \text{CLS}&.975208&.751133&.859252&38.4\\\\ \text{HoGRC}&.5&.076923&.142857&1440 \end{array}$$ **F1** combines precision and recall; **SHD** (structural Hamming distance) counts graph edge errors. AUROC/AP rank edge scores; native F1/SHD assess each method's binary graph. ARN has no preset binary threshold (F1/SHD: —). SINDy-Poly2 also achieves exact recovery, using a library containing the true dynamics. CLS obtains AP=.751133, F1=.859252, SHD=38.4; its original L96 training set used 20000 points at dt=.05, compared with 4000 at dt=.02 here. HoGRC's selected simplices project to all 1560 directed variable edges, giving projected AP=120/1560=.076923; this metric evaluates variable-level edges rather than hyperedges. Forecast results for HoGRC/CLS appear in Part 2. **Q4. Removing D shrinkage.** Removing both the explicit D penalty and AdamW decay on D (other regularizers retained), **all five new from-scratch 300-epoch fits** achieve AUROC=AP=1, with margins .05521/.04049/.03282/.05575/.05349. Model, optimizer, scheduler and RNG states are saved. **More: Simplicity and Generality.** AugRelNet uses simple MLPs with shared relational routing and persistent-response biases, without system-specific physical priors. On 40-variable chaotic Lorenz-96, it recovers all 120 interactions from just 4000 consecutive observations, with exact separation up to 75% noise. Experiments also span dense-interaction systems and 2D spatial dynamics, highlighting the applicability of these general inductive biases across different dynamical regimes. Structure emerges early and supports prediction, while the current forecasting readout remains basic. We will clarify this motivation and add detailed results, code/configs/seeds, and formatting corrections in the revision.
AuthorsOpenReview ↗

Forecasting Performance and Baseline Fairness

Q2. Forecasting and persistence. On five new low-data Lorenz-96 instances, ARN achieves H1 RMSE 14.5–62.1× smaller than all six tested global neural baselines without supplied topology or a matched equation library, and remains better at H8/H24/H40. It also outperforms locality-informed RC throughout and comes within 2.4× of true-support GraphODE-Oracle at H1–H24. All methods were retrained/refitted from scratch on shared 4000/1000/1000 train/validation/test splits. Mean physical-state test RMSE (n=5); DS=DeepSkip; LocalRC=locality-informed reservoir computing:

&H1&H8&H24&H40\\\\ \text{Persistence}&.3746&2.747&5.209&5.585\\\\ \text{ARN}&.007289&.04891&.2183&.5724\\\\ \text{NODE}&.3476&2.306&3.435&3.453\\\\ \text{AL-RNN}&.4163&2.344&3.501&3.601\\\\ \text{MNO}&.4523&2.447&3.609&3.675\\\\ \text{GlobalDS}&.1059&.7908&2.531&3.923\\\\ \text{Dense GraphODE}&.3813&2.534&4.101&4.098\\\\ \text{DySLIM}&.3044&2.214&4.244&4.75\\\\ \text{LocalRC}&.008971&.1461&.6682&1.479\\\\ \text{LocalDS}&.00177&.01307&.04884&.117\\\\ \text{SINDy-Poly2}&.00003949&.0003&.000832&.001741\\\\ \text{SINDy-Poly3}&.3184&2.37&4.693&5.365\\\\ \text{GraphODE-Oracle}&.003795&.02531&.09106&.1834\\\\ \text{HoGRC}&.453&5.42&285&805.9\\\\ \text{CLS}&.001637&.1536&1.437&\text{undefined} \end{array}$$ At CLS H40, two runs are nonfinite and another reaches 5.40e113; thus no five-run mean. CLS outperforms ARN at H1. New independent runs and longer optimization put GraphODE-Oracle ahead of ARN at all four horizons, updating our earlier result. SINDy now uses fourth-order centered differences (archived Poly2 and Poly3 used second-order). LocalDS, true-support GraphODE-Oracle and matched-library SINDy-Poly2 outperform ARN. ARN jointly learns structure and dynamics without supplied topology or an equation library. NODE/MNO exhibit early overfitting. For all five instances, NODE's validation-best checkpoint is epoch 5 (training continues to 500): H8 training-probe RMSE falls 2.0653→.0670, while validation rises 2.3165→2.4155. MNO selects epoch 10 and trains to 600; training falls 2.2370→1.3053, validation rises 2.4614→2.5373. Tests use validation-best checkpoints. Overlapping windows reuse one short trajectory; dt=.02 makes persistence strong at H1, while multistep/dissipativity objectives optimize more than H1. ARN combines shared relational/node mechanisms with causal history. **Q3. Search, capacity and stopping.** Methods share train-only normalization and 960 test origins (t=5000..5959). ARN conditions on t-31:t, initially using 31 validation observations; rollouts use no future truth. NODE/MNO/GraphODE use only the current state. Development-only searches and independent confirmations precede frozen configurations and from-scratch formal fits. ARN uses fixed C01 following six partial trials. $$\begin{array}{lrrr} &\text{Search+confirm}&\text{Fitted P}&\text{Epochs/layers}\\\\ \text{ARN}&\text{fixed}&1292690&3000-4800\\\\ \text{NODE}&10+3&304168&500\\\\ \text{MNO}&16+3&125440&600\\\\ \text{LocalDS}&60+8+3&32768&8\text{L}\\\\ \text{GlobalDS}&70+8+3&10485760&16\text{L}\\\\ \text{GraphODE-Oracle}&20+3&166209&1000-9900 \end{array}$$ DS fixed-feature-inclusive totals are 393216/31719424 (local/global); the +8 terms denote ridge refinements. NODE explores width 256/512/1024, depth 2/3, LR .0003/.001, decay 0/.0001/.01; selected 512×2, LR .001, decay .0001. MNO explores width 150/300/512/2000, depth 3/6, horizon 1/8/40, LR .0001/.0005/.001, decay .0001/.01, dissipativity 0/.01/.1/1; selected 150×6/H40, LR .0005, decay .0001, dissipativity 1. DS explores width 1024–16384, depth 1/2/8/16/32, ridge 1e-10–1e-1; selected local 4096×8/ridge 1e-7, global 16384×16/ridge 1e-8. Searches cover specified subsets, not Cartesian grids. Ordinary ARN achieves exact separation by ~400 epochs on all five instances (margins .03639–.06368); subsequent 3000–4800 epochs follow forecasting validation. GraphODE-Oracle runs 1000–9900 epochs. Gradient models complete native LR plans, then stop after two successive 100-epoch windows with <0.2% best-validation improvement; phase/LR changes reset counters. No hard epoch/time caps. DS completes all layers. All gradient models use the same stopping rule after completing their native optimization plans. The remaining baselines also underwent development-only tuning; full search spaces, selected configurations and confirmation records will be provided in an attachment within the next few days. We will provide per-seed results, hyperparameter searches, configurations, stopping records, code, and seeds within the next few days. Equation wrapping and spectral axes will be corrected in the revised manuscript. Our focus is short- and medium-horizon point forecasting; some specialized models achieve better long-term statistical fidelity.

Q1 follow-up — Strict separation under partial observation. We now have three instances per fixed observation mask. With 36 and 30 of 40 variables observed, mean AP on the visible direct subgraph is 0.9921 and 0.9773. The mean gap between the weakest edge score and strongest nonedge score is −0.0167 and −0.0951, respectively. Strict separation occurs in 0 of 3 instances under each mask (Figure 3).

Q2 follow-up — SINDy implementation. The revised SINDy Poly2 and Poly3 comparisons use fourth-order differences. The archive confirms second-order differences for the original Poly3 run. The original Poly2 derivative order remains unverified.

Q3 follow-up — The 300-epoch comparison. Table 5 and Appendix I.1 report a 300-epoch comparison on the same three Lorenz–96 instances. It includes five dedicated capped fits and six methods whose archived validation-best checkpoints fall within the cap. Mean 40-step test RMSE is 0.8687 for ARN, 3.6796 for cMLP, 3.4290 for NODE, and 3.6565 for MNO. Each method retains its native update schedule. Appendix G and the supplement document tuning and checkpoint selection.

Reviewer_JpW3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper proposes AugRelNet, a neural dynamical model for short- and medium-horizon forecasting from limited data. Each observed variable has two augmented states: a freely evolving local state uu and a response state η\eta. All cross-variable dependence passes through a learned, dense, signed relation matrix DD. Shared networks emit messages, targets aggregate them with DD weights, and nonlinear receivers process the result. η\eta relaxes at learned rates toward a zero-anchored relational drive. Training uses prediction losses with light regularization on DD.

The theory gives a walk expansion of the model Jacobian (Prop. 3.1), a characterization of the relation coordinates selected by gradient flow in a frozen linearized model (Prop. 3.2, Cor. 4.1), and a relaxation-filtered bound on response errors (Prop. 3.3).

The experiments cover five systems:

  • On GLV (6 species, 5 systems), learned relations correlate at about 0.9960.996 with the true interactions in four of five systems.
  • On 40-dimensional Lorenz–96 with 3,849 training states, the top 120 entries of ∣D∣|D| match the true support in all five seeds, and h=8h=8 RMSE is 1414–59×59\times below the neural baselines.
  • On FitzHugh–Nagumo, Game of Life, and Ising lattices, relations are shared across spatial offsets, and the physical neighbours are ranked highest.
Critical strengths
  1. The design is clear. Routing all cross-variable information through one matrix, with per-node encoders that cannot see other nodes, gives an interpretable relational object while leaving the dynamics flexible.
  2. The Lorenz–96 results are promising. Support recovery is exact across seeds, signs are correct at every target, separation persists without shrinkage on DD, and forecast errors are well below the tested baselines.
  3. Zeroing or shuffling DD, resetting η\eta, capacity-matched state ablations, and neighbour-group swaps together suggest the learned components are used as described.
  4. The coarse-sampling experiment is informative. Off-support weight shifts toward two-hop neighbours as Δ\Delta grows. This indicates DD reflects predictive structure at the observation scale.
  5. The paper reports its limitations openly, including the cases where baselines perform better.
Critical weaknesses
  1. AugRelNet received extended training, with about 3,500 epochs plus 13,000–16,000 continuation epochs. Training budgets, tuning, and parameter counts for the Lorenz–96 baselines are not reported. Several baselines are global variants, while AugRelNet shares networks across nodes, which may be an advantage on a homogeneous system. The localized variants from the cited works (Platt et al., 2022; Mandal & Gottwald, 2025) would be natural comparisons.
  2. Noise-free, fully observed Lorenz–96 is a setting where sparse regression or Granger-type methods often succeed. GLV is linear in its interaction coefficients in log coordinates. The support recovered by PySINDy, a Lasso or Granger baseline, and CNN kernel magnitudes on the lattices would help show whether the recovery goes beyond existing tools.
  3. In Table 6, the zz-only model already outperforms all non-oracle baselines (0.3670.367 at h=8h=8). Augmentation adds roughly 4×4\times. The η\eta-specific gain over capacity-matched free augmentation appears to be about 20% (0.1070.107 vs. 0.1310.131). Most of the improvement may come from the shared, DD-routed architecture.
    • The Lorenz–96 seeds share one dataset, and the scaling study uses one seed per budget.
    • GLV has six species and a single baseline.
    • The largest unstructured system has N=40N=40.
    • All systems are synthetic, noise-free, and fully observed.
  4. Prop. 3.2 assumes a frozen linear model, zero initialization, and no regularization. Training uses AdamW with weight decay. CC is not estimated empirically. Lemma B.2 is asymptotic in δ\delta, while the implementation uses unit steps in model time. The theory seems to motivate the design more than it explains the results.
    • The comparison with GraphODE-Oracle at short horizons is within about one standard deviation.
    • Ising results roughly match the CNN.
    • Life accuracy on fresh initial conditions varies across seeds (95.7±8.0%95.7 \pm 8.0\%).
    • FHN at h=32h=32 is about 11×11\times worse than U-Net.
    • The long-rollout Lorenz–96 model shows a mean-energy bias (12.6812.68 vs. 9.369.36).
  5. Presentation. The text uses much specialized terminology, and the main claims depend heavily on the appendix. The FHN model uses an unanchored η\eta drive (Appendix D.3), which differs from Section 3.2.
Questions for authors
  1. What training budgets, tuning, and parameter counts were used for the Lorenz–96 baselines? How do localized DeepSkip and localized reservoir computing perform at 3,849 states?
  2. What support do PySINDy, Lasso, or Granger-type methods recover on Lorenz–96 and GLV?
  3. How does recovery change with observation noise or partial observation?
  4. Can CC, or a local analogue, be estimated for the trained models?
  5. Do the results hold across independently generated datasets?
  6. What are the training costs relative to the baselines?
  7. Please clarify how the Table 8 values at 3,849 states (margin 0.11100.1110, contrast 108.33108.33) relate to the main five-seed runs.
5 comments

Q1: Training budgets, tuning and localized baselines. We have rerun all five benchmark systems—GLV, Lorenz–96, FHN, GoL and Ising—on new independent datasets, retraining/refitting all compared models from scratch. The Lorenz–96 comparison includes the requested local baselines.

Lorenz–96 uses five independent datasets, each with 4,000 training, 1,000 validation and 1,000 test states. Methods share data, train-only normalization and 960 test origins per instance. All rollouts are autonomous; test values never enter tuning, stopping or checkpoint selection. Checkpoints are frozen before testing; all five instances are retained. Five-instance mean physical RMSE follows; lower is better:

\text{Method} & H1 & H8 & H24 & H40 \\\\ \hline \text{Persistence} & 0.3746 & 2.747 & 5.209 & 5.585 \\\\ \mathbf{ARN} & 0.007289 & 0.04891 & 0.2183 & 0.5724 \\\\ \text{NODE} & 0.3476 & 2.306 & 3.435 & 3.453 \\\\ \text{AL-RNN} & 0.4163 & 2.344 & 3.501 & 3.601 \\\\ \text{MNO} & 0.4523 & 2.447 & 3.609 & 3.675 \\\\ \text{Global DeepSkip} & 0.1059 & 0.7908 & 2.531 & 3.923 \\\\ \text{GraphODE-Dense} & 0.3813 & 2.534 & 4.101 & 4.098 \\\\ \text{DySLIM} & 0.3044 & 2.214 & 4.244 & 4.75 \\\\ \text{NRI} & 0.2023 & 1.164 & 3.271 & 3.567 \\\\ \text{cMLP} & 0.8696 & 2.419 & 3.474 & 3.705 \\\\ \text{cLSTM} & 1.989 & 3.049 & 3.546 & 3.707 \\\\ \text{SyCo-AE} & 1.507 & 2.478 & 3.907 & 4.121 \\\\ \text{HoGRC} & 0.453 & 5.42 & 285 & 805.9 \\\\ \text{CLS} & 0.001637 & 0.1536 & 1.437 & \text{NF} \\\\ \text{LocalRC} & 0.008971 & 0.1461 & 0.6682 & 1.479 \\\\ \text{Local DeepSkip} & 0.00177 & 0.01307 & 0.04884 & 0.117 \\\\ \text{GraphODE-Oracle} & 0.003795 & 0.02531 & 0.09106 & 0.1834 \\\\ \text{SINDy-Poly2} & 0.00003949 & 0.0003003 & 0.0008321 & 0.001741 \\\\ \text{SINDy-Poly3} & 0.3184 & 2.37 & 4.693 & 5.365 \\\\ \end{array}$$ NF: CLS has two nonfinite H40 runs; no finite-subset mean is reported. (SINDy originally used second-order differences; the rerun uses fourth-order differences.) LocalRC/Local DeepSkip receive locality, Oracle the true graph, and SINDy polynomial dictionaries; ARN receives none of these supplied structures. ARN beats LocalRC at every horizon and leads H8/H24/H40 among configurations without these supplied structures. CLS is better at H1; Local DeepSkip, Oracle and SINDy-Poly2 have lower errors throughout. Tuning was completed on development data, leading recipes independently confirmed, then frozen before five formal fits. NODE searched 10 candidates plus 3 confirmations, covering widths 256–1024 and depths 2–3; MNO used 16+3, widths 150–2000 and depths 3–6. Local/Global DeepSkip searched 60/70 configurations, followed by ridge refinement and independent confirmation; LocalRC searched 108. DeepSkip widths span 1024–16384, depths 1–32 and ridge 1e-10–1e-1. These ranges describe tested subsets. ARN keeps C01 after six partial trials, without per-instance retuning. Native losses and inputs, including ARN's 32-observation history, are documented. All gradient methods use the same validation-based stopping logic after their declared optimizer/LR stages: two consecutive 100-epoch windows with less than 0.2% improvement in historical best validation score. LR or training-stage changes reset patience; there is no hard epoch/time cap. Tests use validation-best checkpoints. Regression methods complete native solves. ARN terminates at 3,000–4,800 main epochs; Oracle at 1,000–9,900. Parameter counts are 1,292,690 for ARN, 304,168 for NODE and 166,209 for Oracle. Local/Global DeepSkip fit 32,768/10,485,760 coefficients, with fixed features separate. In three paired instances, a 300-main-epoch ARN control gives H8 RMSE 0.1077 versus NODE 2.2801, cMLP 2.3991, NRI 1.1314 and DySLIM 2.1792. The gaps precede prolonged refinement. Several baselines reach validation optima early: NODE at epoch 5, MNO at 10, NRI at 15–40. NODE's H8 training-probe error falls from 2.065 to 0.067 while validation worsens from 2.317 to 2.415. We evaluate the earlier validation-best checkpoints. **Q6: Training costs.** Equal epochs are not equal compute. On one GPU, complete-entry timing gives 18.56 s/epoch for ARN and 13.67 for Oracle over 25 main epochs. Three repeated full Local DeepSkip fits average 8.48 s each. A separate same-GPU, two-hour-per-run experiment, including initialization, warmup, validation and saving, gives H8 0.1112 for ARN and 0.0343 for Oracle across three independent pairs. Shared-hardware full-run elapsed times are recorded separately. Because these records exceed the comment limit, we will provide a comprehensive reproducibility PDF covering data/code identities, seeds, complete searches and selected recipes, parameters and fixed features, losses and histories, update counts, curves, stopping/checkpoint decisions, hardware/concurrency, costs, individual outcomes and sample SDs, linked to archived code and raw records.

Q2-Q3: Structure recovery and robustness. We added sparse-regression and Granger comparisons, and experiments with noisy and partially observed data. The clean Lorenz-96 comparison uses five independently generated datasets with 4000 training states each. Hyperparameters are selected by validation prediction error, without the true graph or test data.

AUROC measures how consistently true edges rank above nonedges; AP (average precision) summarizes precision as recall increases. Both are higher-is-better, with 1 ideal. F1 balances edge precision and recall (higher is better); directed SHD counts edge additions/deletions needed to match the reference (lower is better). Five-instance means are:

\text{Method}&\text{AUROC}&\text{AP}&\text{native F1}&\text{SHD}&\text{density}\\\\ \hline \text{ARN}&1&1&-&-&-\\\\ \text{SINDy-Poly2}&1&1&1&0&7.7\\%\\\\ \text{Lasso-Poly2}&1&1&.14463&1419.4&98.7\\%\\\\ \text{Group-VAR Granger}&.96853&.71708&.14363&1431&99.4\\% \end{array}$$ SINDy-Poly2 recovers all 120 true edges without false edges, using a quadratic library containing the true dynamics. The validation-selected Lasso and Group-VAR graphs remain almost fully connected: 1539.4 and 1551 of 1560 possible edges (98.7% and 99.4%), versus 120 (7.7%) in the true graph. Both include every true edge but also 1419.4 and 1431 false edges, respectively. Lasso's perfect AUROC/AP describes its ranking, while its native L1 solution fails to produce a sparse graph; Group-VAR is similarly dense despite group regularization. ARN supplies continuous D scores without a preset binary threshold, so its native F1/SHD remain undefined. Lasso/Group-VAR used 15/30 development candidates, top-three confirmation on a second dataset, then five formal fits. For GLV, we added full quadratic regression and DenseNODE to the same five independent systems evaluated with ARN and NGM. Mean absolute off-diagonal Pearson correlations with the interaction coefficients are .992719 for ARN, .999999363 for NGM, .999999076 for DenseNODE and .999999998 for quadratic regression. The readouts are ARN's native D, abundance-adjusted log-RHS Jacobians for NGM/DenseNODE, and polynomial coefficients for regression. The supplied quadratic library matches GLV's algebraic form and achieves especially precise recovery. ARN learns explicit relations without a prescribed equation library. For lattice CNNs, the added structural diagnostic measures end-to-end input influence. This accounts for nonlinear layers and is distinct from treating raw convolution-kernel magnitudes as an interaction graph. **Observation noise.** Gaussian noise is added to both training inputs and targets, with its standard deviation expressed as a percentage of each variable's clean training standard deviation; validation remains clean. ARN keeps the same configuration and a fixed 300-epoch budget across noise levels. At 3/5/10/15/25/35/50/75%, all 24 fits (three independent instances per level) achieve AUROC=AP=1: every true edge ranks above every nonedge. Higher-noise AP (mean +/- sample SD; n=3): $$\begin{array}{lrr} \text{Noise SD}&\text{ARN AP}&\text{tuned SINDy-Poly2 AP}\\\\ \hline 100\\%&.953580\pm.080402&.996970\pm.004804\\\\ 150\\%&.999954\pm.000080&.961657\pm.021723\\\\ 200\\%&.996004\pm.004307&.824635\pm.005217\\\\ 300\\%&.788067\pm.051801&.440321\pm.022747 \end{array}$$ SINDy is separately validation-tuned at each noise level with 54 development configurations and top-three confirmation, including training-data preprocessing choices. ARN retains its fixed recipe. SINDy performs better at 100%, while ARN retains higher AP at the three larger noise levels. Near-perfect average ranking and exact separation remain distinct: ARN achieves strict separation in 2/3 instances at 150%, and 0/3 at 200% and 300%. **Partial observation.** We retrained ARN with only 36 or 30 of the 40 variables available, using the original validation-based stopping and checkpoint selection. Hidden variables enter neither model inputs nor normalization or training targets. Scores against the direct interaction subgraph among visible variables are: $$\begin{array}{lrr} \text{Observed variables}&\text{AUROC}&\text{AP}\\\\ \hline 36/40&.999982&.999784\\\\ 30/40&.997946&.972920 \end{array}$$ These are one data/training instance per setting (additional independent instances are being completed for both fixed observation masks). Neither setting achieves strict all-edge separation. For the same 30 observed targets and 960 test origins, H1 RMSE is .0155 with partial inputs versus .0077 with full inputs; H40 RMSE is 2.978 versus .590 (lower is better). Strong agreement with visible-variable direct interactions thus persists despite substantial forecasting degradation. The hidden variables remain dynamically active, so predicting future states also requires accounting for influences absent from the observed state.

Q5: Independently generated datasets. We reran the five benchmark systems with newly generated data and from-scratch fits under frozen, validation-selected protocols. In the new Lorenz-96 main comparison, formal00-formal04 use five independently generated trajectories, each with 4000 training, 1000 validation and 1000 test states. Data seeds, model initialization and sampling streams differ across instances; methods share the same data within each instance. This replaces the original design in which five model seeds shared one trajectory.

The new ARN models achieve AUROC=AP=1 on all five Lorenz-96 datasets: all 120 true directed interactions rank above all 1440 noninteractions in every instance. Prediction results and the expanded baseline comparisons are reported in Q1/Q6; structure, noise and partial-observation comparisons are in Q2/Q3. The unit of replication is the dataset/model instance, rather than its test origins or individual coordinates.

For GLV, we generated five independent interaction systems and trajectories and evaluated ARN and NGM on all five. DenseNODE and full quadratic regression were then added on these same systems. ARN's mean absolute off-diagonal correlation with the interaction coefficients is .992719, with sample SD .005673, including every system. The added baselines' results and their different structural readouts are given in Q2/Q3. FHN, Game of Life and Ising were also rerun with five new data/initial-condition instances per system and matched data across methods. These repeat the spatial dynamics under new realizations while retaining each benchmark's specified interaction rule.

The scaling extension additionally targets three independent data/training instances at each of 1000, 10000 and 20000 training observations. Local RC has completed all these fits and frozen tests; remaining ARN fits are running, while ARN/DySLIM prediction tests await cohort freezing. The retained 50000-observation neural experiment is a single instance and is labelled separately. We also completed a separate 128-variable irregular-mesh Allen-Cahn case, reported as one instance. Dataset repetition, changes of the physical interaction system, and increased dimension are identified separately in the experimental record.

Q7: Table 8 and the main five-seed results. Table 8's margin .1110 and edge/nonedge contrast 108.33 belong to the original single-model scaling branch at 3849 training states. Appendix D.2.1 specifies nested prefixes of one trajectory and one model seed per budget. These are not averages of the main five model seeds.

The original main cohort has strict separation in all five model seeds, with margins ranging from .06988 to .10817 and mean contrast 146.13 (Appendix D.2). The scaling row and the main results therefore refer to different trained models; the common observation count does not make their weight summaries identical.

To make the correspondence between structural and predictive evidence explicit, we rebuilt the main comparison in full: all five ARN instances and the baseline fits were trained from scratch on the new independent datasets described in Q5. Every instance is retained in the comparison. The revised main results thus come from a complete new five-instance evaluation, extending beyond the original single-model scaling diagnostic.

For each new instance, we will present forecasting and structure results together with the dataset, initialization/sampling seeds and the checkpoint identity underlying each metric. This makes the structure-to-prediction correspondence traceable per instance. The original 3849-state scaling row and the new 4000-state comparison will be labelled as separate cohorts, with these records provided in the detailed reproducibility PDF.

Comments on the paper

1 comment

The revision keeps ARN's explicit relation matrix and the five original benchmark families. It follows a question raised by the original Lorenz–96 result and the reviews. When prediction training recovers the generating edge ranking, what does that agreement tell us, and do the learned weights also help the predictor fit the dynamics?

  • Theory and a new test of the learned weights (§§3 and 5.2, Appendices A–B and I.4). Theorem 3.1 shows that distinct dependence graphs can realize the same finite record, which changes how we interpret agreement with the generating graph. Proposition 3.2 replaces the frozen-Jacobian analysis with a controlled example where the choice of weights affects how quickly a fit is reached. The original permutation experiment perturbed a fitted ARN. We now refit the other parameters from matched initializations, holding either the learned or permuted weights fixed. The learned weights give lower 40-step validation error in all three pairs at the same update budget.

  • Independent evaluations on the original benchmarks (§§5–6, Appendix G). We repeated the Lorenz–96 comparison on five independent trajectories with 4,000 training observations each. Without a supplied graph or stencil, ARN ranks all 120 generating edges above all 1,440 nonedges in each instance. We also updated the GLV held-out forecasting comparison and reran FitzHugh–Nagumo with the corrected memory update.

  • Stronger comparisons and documented budgets (§5.2, Appendices G and I.1). We added structural and forecasting baselines, an eleven-method 300-epoch comparison, and a same-GPU comparison with GraphODE–Oracle. ARN has the lowest 40-step error among the tested methods given no graph, local stencil, or equation library. Several baselines given such information achieve lower errors. Appendix G and the supplement document tuning and checkpoint selection.

  • Tests of the scope of the results (§§5–7, Appendices E–F and I). We added noise and partial-observation tests, paired memory and response-anchoring ablations, and an irregular-mesh case. In Kuramoto, ARN has low short-horizon error alongside weak agreement with the physical graph. In a separate Lorenz–96 run, later training improves 40-step forecasts while worsening longer-horizon errors. Both checkpoints fail during long-rollout burn-in.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.