acceptodds
Under review as a conference paper at ICLR 2027

LLN: Learnable Lens Networks for Parameter-Efficient Long-Horizon Dynamical Prediction

Abstract

Explicit residual connections of the form (x+f(x)), often combined with normalization layers, have become a standard strategy for training very deep neural networks. However, residual addition primarily provides an algebraic shortcut for gradient propagation, while leaving the evolution of feature geometry across layers largely unconstrained. We introduce Learnable Lens Networks (LLNs), a physics-inspired architecture that replaces direct feature-space residual accumulation with learnable optical transport in an augmented position–angle phase space. Each layer alternates between free propagation, which provides an implicit transport path, and a learnable lens field that performs nonlinear trajectory transformation and focusing. Theoretically, we establish that LLN transport is globally invertible and volume-preserving for any differentiable lens field, with the implemented coordinate-wise Gaussian transport further satisfying symplecticity. Importantly, these structural constraints do not limit expressivity: with unrestricted embeddings and readouts, LLNs retain universal approximation of continuous end-to-end maps. Experiments across diverse dynamical systems demonstrate that LLN improves long-horizon prediction while using substantially fewer parameters than same-depth comparators. Further analysis reveals stable depth-wise gradient transport and interpretable learned dynamics under the coupled propagation and refraction design.

Primary area:
learning theory
Keywords:
LLN, phase‑space transport, symplectic constraint, residual network

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_SzQqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2(+1)
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces Learnable Lens Networks (LLN), an architecture that alternates linear propagation and coordinate-wise Gaussian refraction in an augmented position–angle space. The paper characterizes the invertibility, volume preservation, and symplecticity of the implemented latent transport and evaluates prediction error and costs on simulated dynamical systems.

Critical strengths
  1. The propagate–refract construction assigns computational roles to the position and angle states, making the architecture straightforward to understand and analyze.

  2. The paper provides an explicit inverse and analyzes volume preservation, the conditions for symplecticity, and the resulting reciprocal singular-value pairing.

Critical weaknesses
  1. The core transport mechanism of LLN is closely related to established reversible additive coupling and symplectic shear constructions (e.g., RevNets and SympNets). The coordinate-wise Gaussian lens appears to provide an incremental parameterization of this existing framework.

  2. Table 1 reports accuracy for selected deployments but FLOP ratios for a separate matched-width configuration. These counts also exclude exponential/tanh costs, leaving the practical efficiency of the evaluated models unclear.

  3. The provided manuscript does not specify the Gaussian center locations, bandwidth, or lens-coefficient initialization sufficiently to reproduce the transport. This also makes it difficult to assess the sensitivity of the results to the fixed dictionary.

Questions for authors
  1. How does LLN’s end-to-end inference latency compare with the baselines?

  2. Could the authors specify these settings and report sensitivity to the center range and bandwidth?

5 comments
AuthorsOpenReview ↗

W1: Relation to reversible coupling and symplectic networks

Thank you for raising this distinction. We agree that additive coupling, invertibility, and symplectic shear composition are established ideas; our contribution is the particular parameter-efficient and inspectable transport architecture, not a new general principle of reversibility or symplecticity. LLN combines fixed linear propagation with a learned coordinate-wise refraction field:

\theta_{l+1}=\theta_l+\phi_l(y_{l+1}),\qquad L=T/D.$$ The implemented ordering is a drift–kick symplectic-Euler-type composition. This connects LLN to reversible coupling and SympNets [1]. The design fixes the propagation map, separates displacement from directional correction, and uses a learned initial direction. Its structural properties should not be presented as uniquely Gaussian. The Gaussian parameterization makes each directional correction explicitly decomposable. For coordinate j, $$\phi_{l,j}(y)=\sum_{k=1}^{K}a_{l,j,k} \exp\left[-\left(\frac{y-c_k}{b}\right)^2\right].$$ Here cₖ specifies "where" a basis response is concentrated, b specifies "how" broadly it acts, and $a_{l,j,k}$ specifies its signed strength. For a given trajectory, each term's contribution to the angle update can be evaluated separately, then related to subsequent displacement. This provides a concrete way to inspect local refraction and layerwise transport, rather than relying on optical terminology alone. The precise settings are given in W3/Q2 below. With fixed centers and bandwidth, each layer learns HK coefficients rather than a dense H×H transformation, yielding an O(DHK) rather than O(DH²) depth-dependent parameter term. This is a compact parameterization when K is small relative to H. 1. Jin et al. (2020). [*SympNets: Intrinsic Structure-Preserving Symplectic Networks for Identifying Hamiltonian Systems*.](https://arxiv.org/abs/2001.03750) Neural Networks, 132, 166–179.
AuthorsOpenReview ↗

W2 and Q1: Practical efficiency and end-to-end inference latency

Thank you for identifying this presentation issue. To complement those counts, we provide measured end-to-end inference latency, which includes actual execution of the Gaussian exponential and angle-initialization tanh operations.

To further assess LLN's practical efficiency relative to closely related structure-preserving architectures, we extend the latency comparison beyond MLP baselines to include G-SympNet [1] and R-VPNet [3]. Comparing these implementations on the same reported hardware and inference settings directly tests whether LLN offers a latency advantage over related symplectic and volume-preserving models, rather than only over generic dense networks.

Table R1. End-to-end inference latency (ms; lower is better). D=40, H=512, batch size 1, FP32, and three initialization seeds. Bold marks the lowest reported latency in each hardware/thread column. This is a separate runtime benchmark, not a timing measurement of every manuscript Table 1 deployment.

ModelL20 GPU0 (ms)L20 GPU1 (ms)CPU, 1 thread (ms)CPU, 8 threads (ms)
LLN0.2000.2030.8960.566
Plain MLP0.4340.4395.1595.128
Scaled ResMLP (Res-S)0.4370.4395.1115.086
G-SympNet [1]0.4100.4150.9370.951
R-VPNet [3]0.5960.5991.1821.201
HNN-RK41.2031.20924.69025.045
NeuralODE-RK4 [2]0.7590.76117.41817.365
FNO-1D [4]4.2814.287652.930221.861

LLN has the lowest reported latency in all four settings. On L20 GPU0, it takes 0.200 ms, compared with 0.434 ms for Plain MLP, 0.437 ms for Scaled ResMLP, 0.410 ms for G-SympNet, and 0.596 ms for R-VPNet. These correspond to approximately 2.17×, 2.19×, 2.05×, and 2.98× comparator-to-LLN latency ratios. The related-family comparators are referenced to SympNets [1], VPNets [3], NeuralODEs [2], and FNO [4]; the labels identify the implementations evaluated here.

These measurements support practical efficiency for the tested implementations and configuration. We hope the above response addresses your concerns.

  1. Jin et al. (2020). SympNets: Intrinsic Structure-Preserving Symplectic Networks for Identifying Hamiltonian Systems. Neural Networks, 132, 166–179.
  2. Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. (2018). Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems (NeurIPS).
  3. Zhu, A., Zhu, B., Zhang, J., Tang, Y., and Liu, J. (2022). VPNets: Volume-preserving neural networks for learning source-free dynamics. Journal of Computational and Applied Mathematics, 416, 114523. DOI: 10.1016/j.cam.2022.114523.
  4. Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. International Conference on Learning Representations (ICLR).
AuthorsOpenReview ↗

W3 and Q2: Gaussian-dictionary settings and sensitivity

We thank the reviewer for this suggestion. We now report the dictionary settings and provide an experiment that varies the center range and bandwidth independently to assess their effects on prediction accuracy.

Dictionary settings. Each lens uses Gaussian basis functions exp⁡[−((y−ck)/b)2]\exp[-((y-c_k)/b)^2]. The default dictionary places KK centers uniformly in [−5,5][-5,5], including both endpoints, and uses bandwidth b=20/Kb=20/K. Thus, b=5/3b=5/3 for K=12K=12 and b=2.5b=2.5 for K=8K=8. The centers and bandwidth remain fixed during training. The coefficients al,j,ka_{l,j,k} are initialized independently from N(0,0.052)\mathcal{N}(0,0.05^2) and learned separately for each layer, coordinate, and basis function.

The center range determines coverage in latent position coordinates, rather than the physical input domain, while the bandwidth controls the response width and overlap between neighboring basis functions. With the default range, the center spacing is Δc=10/(K−1)\Delta c=10/(K-1), giving b/Δc=2(K−1)/Kb/\Delta c=2(K-1)/K. This rule reduces the bandwidth as more centers are introduced while maintaining overlapping responses. We use it as a default configuration, without assuming that it is optimal for every system.

Experimental setup. We fixed D=40D=40, H=128H=128, and K=12K=12, and evaluated all nine combinations of

R∈{2.5,5,10},b∈{5/6,5/3,10/3},R\in\{2.5,5,10\},\qquad b\in\{5/6,5/3,10/3\},

with centers uniformly spaced in [−R,R][-R,R]. Both factors therefore cover half, the default, and twice their default values. Bandwidth was varied independently of the center range, allowing us to examine their individual effects and interaction.

Within each system, the parameter count, training procedure, and optical total step scale were unchanged. All runs used 20 epochs of Adam, batch size 1024, zero weight decay, no learning-rate schedule, and one-step MSE training. Evaluation used 500-step autoregressive cumulative normalized MSE.

The results show that the preferred dictionary depends on the system. For example, with bandwidth fixed at b=5/3b=5/3, increasing RR from 2.5 to 5 to 10 reduces the mean error on ReactionDiff-32 from 0.00705 to 0.00374 to 0.00268. The same change does not benefit ReactionDiff-48, whose corresponding errors are 0.00574, 0.01592, and 0.01371. A larger center range is therefore not uniformly better.

The results also show an interaction between range and bandwidth. On Burgers-48, at R=5R=5, reducing bb from 5/35/3 to 5/65/6 increases the mean error from 0.20251 to 2.41695. However, the same narrow bandwidth paired with R=2.5R=2.5 gives the lowest mean in the grid, 0.10865. This illustrates why the two parameters should be examined jointly.

The default configuration is within a factor of two of the lowest grid mean on eight of the ten systems; the exceptions are Lorenz-28 and ReactionDiff-48, with ratios of approximately 2.51 and 3.94.

We will include these settings and the full sensitivity table in the revision. The results suggest that LLN has further room for improvement through validation-based tuning of the dictionary settings. This does not change our main conclusions, which are supported by the reported configurations and do not rely on the dictionary being optimal or insensitive to these choices.

Reviewer_VXduOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Learnable Lens Networks (LLN), a parameter-efficient architecture that alternates free propagation and learnable Gaussian refraction in an augmented position–angle phase space. The internal transport is invertible, volume-preserving, and symplectic, while unrestricted embeddings and readouts retain universal approximation capability.

Experiments on several dynamical systems demonstrate promising long-horizon prediction performance with fewer parameters, particularly on dissipative PDEs. Additional evaluations investigate gradient propagation, component ablations, and the interpretability of learned trajectories.

Critical strengths
  1. Elegant architecture with sound theoretical properties. The propagate–refract construction is simple and mathematically coherent. The analysis of invertibility, volume preservation, symplecticity, and universal approximation provides a solid theoretical foundation for the proposed architecture.

  2. Promising empirical results. LLN achieves substantial improvements on several dissipative dynamical systems with considerably fewer parameters. The additional multi-trajectory evaluation provides meaningful confirmation of these improvements, while the gradient analysis offers useful evidence of stable depth-wise signal propagation at initialization.

Critical weaknesses
  1. Limited cross-channel interaction. The coordinate-wise lens formulation makes the latent transport separable across channels. While parameter-efficient, this may restrict feature interactions and make expressivity more dependent on network width, particularly for strongly coupled dynamics (see Q1).

  2. Limited comparison with related symplectic architectures. The proposed construction is closely related to existing symplectic networks. Its incremental advantage over comparable structure-preserving architectures remains insufficiently established (see Q2).

  3. Practical computational efficiency. Although parameter savings and arithmetic operation counts are reported, actual inference latency is not evaluated (see Q3).

Questions for authors

Q1. Could the authors investigate the width–depth trade-off under comparable parameter budgets? In particular, can increasing depth compensate for limited width, and how does LLN compare with dense MLPs on strongly coupled systems?

Q2. Could the authors clarify the distinction from existing symplectic networks and consider an encoder–symplectic network–decoder baseline with comparable parameter budgets?

Q3. Could the authors report measured inference latency on representative hardware to complement the parameter and arithmetic efficiency analysis?

5 comments
AuthorsOpenReview ↗

W1 and Q1: Feature interaction and the width–depth trade-off

Thank you for raising this question. LLN's coordinate-wise transport operates on learned projections of the input, not on isolated original variables. Its position and direction are initialized as y0=Wyx+by,θ0=tanh⁡(Wθx+bθ).y_0=W_yx+b_y,\qquad \theta_0=\tanh(W_\theta x+b_\theta). Each latent channel can therefore depend jointly on multiple observed variables, and the learned readout combines the transported channels. This allows LLN to represent coupled input dependencies while keeping the internal transport parameter-efficient. Additional depth composes further nonlinear transformations of these learned projections; it does not introduce new cross-channel mixing inside the stack.

The depth-width results in Table R1 show that additional depth can be useful at reduced width in the tested settings. The LLN configuration (D,H)=(40,256) has a lower reported mean error than (20,512) on 9/10 systems. On the coupled SpringChain-32 and SpringChain-48 systems, the deeper, narrower model obtains 0.075669 ± 0.001820 and 0.015367 ± 0.000921, compared with 1.065 ± 0.030 for each wider, shallower model. On ReactionDiff-48, the corresponding errors are 0.241831 ± 0.012370 and 2.575 ± 0.051.

These two configurations have the same DH product and, if K is held fixed, the same DHK lens-coefficient budget. The embedding and readout still contribute width-dependent parameters, so this is a depth-width comparison rather than an exact total-parameter match. The approximately parameter-matched cross-model results are reported separately in Table R2.

For the comparison with dense MLPs on coupled systems, Table R1 gives a direct answer. The spring chains contain interacting neighboring masses, while Lorenz has nonlinear cross-variable coupling. At the common setting D=40 and H=512, LLN obtains 0.074686 ± 0.000600 on SpringChain-32 and 0.013302 ± 0.000762 on SpringChain-48, compared with 1.122 ± 0.001 and 1.105 ± 0.001 for Plain MLP. On Lorenz-28, the corresponding values are 0.041560 ± 0.015171 for LLN and 1.192 ± 0.001 for Plain MLP. Plain MLP is also the lowest-mean finite MLP-family comparator in each of these rows. Thus, in these tested coupled systems, deep LLN outperforms the dense MLP comparator despite its coordinate-wise internal transport.

In the D=40 subset of Table R1, LLN has lower reported means than the selected MLP-family comparator on 9/10 systems at H=256 and 10/10 at H=512; the corresponding PDE counts are 5/6 and 6/6. Thus, wider dense baselines do not generally remove the observed deep-LLN advantage. These results support the practical value of composing compact transport layers, including on coupled systems, while the full sweep in Table R1 shows where the depth-width trade-off is less favorable.

  1. Jin et al. (2020). SympNets: Intrinsic Structure-Preserving Symplectic Networks for Identifying Hamiltonian Systems. Neural Networks, 132, 166–179.
  2. Chen, R. T. Q., Rubanova, Y., Bettencourt, J., and Duvenaud, D. (2018). Neural Ordinary Differential Equations. Advances in Neural Information Processing Systems (NeurIPS).
  3. Zhu, A., Zhu, B., Zhang, J., Tang, Y., and Liu, J. (2022). VPNets: Volume-preserving neural networks for learning source-free dynamics. Journal of Computational and Applied Mathematics, 416, 114523. DOI: 10.1016/j.cam.2022.114523.
  4. Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., and Anandkumar, A. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. International Conference on Learning Representations (ICLR).
AuthorsOpenReview ↗

Table R1. Depth–width comparison

DSystemLLN H256MLP H256LLN H512MLP H512
5L241.607 ± 0.3751.080 ± 1.142 (R)1.350 ± 0.7341.426 ± 0.475 (R)
5L281.298 ± 1.1541.379 ± 1.045 (P)1.593 ± 1.0040.169 ± 0.150 (S)
5SC321.055 ± 0.4841.140 ± 0.200 (S)0.373 ± 0.2211.118 ± 0.014 (P)
5SC480.207 ± 0.1501.209 ± 0.196 (R)0.138 ± 0.1081.097 ± 0.014 (P)
5B322.106 ± 0.3261.572 ± 0.264 (B)2.211 ± 0.5531.067 ± 0.070 (P)
5B483.603 ± 0.6812172.325 ± 4852.544 (S)2.876 ± 0.8132.111 ± 0.102 (P)
5AD323.416 ± 1.1641901.313 ± 2445.435 (S)4.245 ± 0.60335.660 ± 72.005 (S)
5AD483.484 ± 1.3158.090 ± 4.832 (P)4.217 ± 0.32756.427 ± 120.885 (R)
5RD323.387 ± 2.0293.085 ± 0.701 (B)2.662 ± 1.5581.988 ± 0.766 (R)
5RD485.619 ± 0.2193.386 ± 1.090 (B)3.154 ± 0.9383.183 ± 1.116 (L)
10L241.442 ± 0.9891.913 ± 0.563 (R)2.308 ± 0.7440.715 ± 0.901 (S)
10L281.236 ± 0.0620.810 ± 0.910 (S)1.324 ± 0.1631.857 ± 1.532 (S)
10SC321.258 ± 0.5981.274 ± 0.189 (S)0.459 ± 0.4321.050 ± 0.197 (L)
10SC481.068 ± 0.0281.142 ± 0.229 (S)0.154 ± 0.0470.688 ± 0.507 (L)
10B321.858 ± 0.6641.038 ± 0.001 (P)1.190 ± 0.1201.116 ± 0.103 (S)
10B483.039 ± 0.2092.100 ± 0.017 (P)2.301 ± 0.3462.064 ± 0.045 (S)
10AD323.485 ± 0.3173.908 ± 1.195 (P)1.996 ± 0.0222.343 ± 1.710 (R)
10AD482.743 ± 0.9876.909 ± 2.220 (P)2.078 ± 0.0393.071 ± 2.297 (S)
10RD326.045 ± 0.8485.887 ± 3.799 (R)3.175 ± 0.5933.023 ± 0.833 (R)
10RD485.798 ± 0.2363.766 ± 1.470 (R)2.587 ± 0.0486.242 ± 1.887 (P)
20L240.000855 ± 0.00020.681 ± 0.556 (S)2.019 ± 1.3941.030 ± 0.014 (L)
20L280.403421 ± 0.63690.731 ± 0.570 (S)1.232 ± 0.0100.519 ± 0.453 (S)
20SC320.075811 ± 0.00241.122 ± 0.002 (P)1.065 ± 0.0301.120 ± 0.001 (P)
20SC480.022379 ± 0.00511.105 ± 0.001 (P)1.065 ± 0.0301.103 ± 0.001 (P)
20B321.098 ± 0.0451.036 ± 0.005 (P)1.088 ± 0.0261.038 ± 0.001 (P)
20B480.907525 ± 0.17082.104 ± 0.008 (P)2.153 ± 0.1581.881 ± 0.291 (R)
20AD322.022 ± 0.0382.001 ± 0.008 (P)1.797 ± 0.3781.571 ± 0.458 (S)
20AD480.570678 ± 0.14032.107 ± 0.006 (P)2.113 ± 0.0372.104 ± 0.003 (P)
20RD321.082844 ± 0.78792.941 ± 0.005 (P)2.875 ± 0.0292.576 ± 1.257 (R)
20RD480.212349 ± 0.01052.622 ± 0.004 (P)2.575 ± 0.0511.493 ± 1.227 (R)
40L240.001047 ± 0.00021.017 ± 0.002 (P)0.000817 ± 0.00021.015 ± 0.001 (P)
40L280.048338 ± 0.02511.193 ± 0.001 (P)0.041560 ± 0.01511.192 ± 0.001 (P)
40SC320.075669 ± 0.00181.121 ± 0.002 (L)0.074686 ± 0.00061.122 ± 0.001 (P)
40SC480.015367 ± 0.00091.105 ± 0.002 (P)0.013302 ± 0.00081.105 ± 0.001 (P)
40B321.371810 ± 0.70671.040 ± 0.011 (L)0.963365 ± 0.32641.042 ± 0.002 (P)
40B481.402734 ± 1.31882.095 ± 0.013 (P)1.415068 ± 0.83712.102 ± 0.005 (P)
40AD320.289476 ± 0.08881.996 ± 0.014 (L)0.366195 ± 0.16052.008 ± 0.008 (P)
40AD480.294315 ± 0.06502.108 ± 0.010 (P)0.263846 ± 0.00582.110 ± 0.006 (L)
40RD320.990504 ± 0.46022.948 ± 0.022 (L)0.487427 ± 0.10252.947 ± 0.006 (P)
40RD480.241831 ± 0.01232.619 ± 0.008 (P)0.159100 ± 0.00122.620 ± 0.031 (S)
AuthorsOpenReview ↗

Table R2. Approximately parameter-matched comparisons

NODE=NeuralODE-RK4 [2], G-Symp=G-SympNet [1], VP=VPNet [3], FNO [4], Koopman=Linear Koopman.

DSystemLLNNODEG-SympVPFNOKoopman
5L246.78±2.1610.83±0.53N/A12.36±5.22N/A7.39±0.22
5L281.69±1.302.04±0.44N/AN/AN/A1.41±0.59
5SC321.16±0.031.92±0.090.82±0.053.68±2.16N/A1.52±0.30
5SC481.15±0.110.59±0.841.49±0.631.29±0.531.70±1.5e-41.28±0.11
5B320.97±0.320.09±0.030.02±1.3e-30.61±0.290.04±2e-34.34±5.97
5B480.80±0.281.31±0.51N/A1.39±0.080.85±0.981.6e5±1.5e5
5AD320.81±0.341.42±0.850.76±0.410.32±0.085.02±0.029.82±14.13
5AD481.12±0.832.05±1.082.26±2.110.71±0.481.92±1.295.82±5.58
5RD321.15±0.618.10±6.170.06±6.5e-30.25±0.161.37±0.6230.29±12.56
5RD482.24±0.956.15±1.003.30±0.224.59±4.540.34±0.020.14±0.04
10L247.48±1.8210.67±1.65N/A0.04±0.04N/A7.39±0.22
10L281.47±1.091.46±0.39N/A0.06±0.02N/A1.41±0.59
10SC321.16±0.062.62±1.160.50±0.020.02±1.2e-3N/A1.52±0.30
10SC481.26±0.241.98±1.140.04±3.2e-30.61±0.23N/A1.28±0.11
10B320.85±0.313.60±5.201.03±0.580.92±0.971.70±6e-64.34±5.97
10B480.63±0.179.56±4.000.12±0.080.70±0.442.25±2.8e-41.6e5±1.5e5
10AD320.64±0.230.30±0.040.74±0.020.91±1.065.04±9.3e-69.82±14.13
10AD480.89±0.560.11±0.010.97±0.080.71±0.375.32±9.9e-65.82±5.58
10RD320.80±0.540.10±0.010.06±6.5e-30.07±3.7e-36.29±1.4e-530.29±12.56
10RD480.67±0.570.73±0.131.43±1.040.73±0.746.20±6.6e-60.14±0.04
20L246.65±0.9515.49±4.29N/A18.75±4.01N/A7.39±0.22
20L281.24±0.622.42±2.21N/AN/AN/A1.41±0.59
20SC321.21±0.151.29±1.270.50±5e-319.79±6.28N/A1.52±0.30
20SC481.18±0.340.25±0.040.38±3e-31.03±0.41N/A1.28±0.11
20B320.64±0.180.94±0.801.30±0.97N/A1.70±3.1e-64.34±5.97
20B480.63±0.201.16±0.870.76±0.92N/A2.25±5e-41.6e5±1.5e5
20AD321.04±0.961.52±1.820.48±0.080.44±0.255.04±6.1e-69.82±14.13
20AD480.78±0.520.18±0.021.34±0.800.36±0.175.32±9.9e-65.82±5.58
20RD320.13±0.121.21±0.880.06±2.2e-4N/A6.29±1.5e-530.29±12.56
20RD480.23±0.360.57±0.300.05±2.3e-30.30±0.436.20±9.9e-60.14±0.04
40L246.17±0.9511.06±1.62N/A15.39±3.33N/A7.39±0.22
40L280.92±0.551.10±0.79N/AN/AN/A1.41±0.59
40SC321.07±0.381.81±9.3e-30.09±0.020.02±1.3e-3N/A1.52±0.30
40SC481.09±0.021.34±1.260.06±7.9e-30.04±3e-3N/A1.28±0.11
40B320.54±0.169.55±0.091.00±0.070.04±4.1e-31.70±4e-64.34±5.97
40B480.55±0.1210.85±5.170.75±0.510.66±0.472.25±4.4e-41.6e5±1.5e5
40AD320.96±0.601.06±0.561.94±1.601.18±0.975.04±1.8e-59.82±14.13
40AD480.56±0.320.83±0.260.32±0.111.49±0.945.32±6.7e-65.82±5.58
40RD320.07±0.040.49±0.370.09±0.010.16±0.056.29±2.6e-630.29±12.56
40RD480.04±0.030.11±0.140.07±4.9e-30.10±0.166.20±4.7e-60.14±0.04
Reviewer_g5tCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper replaces residual addition with transport in an augmented position-angle phase space. The hidden state is a pair (y, theta); each layer does free propagation y <- y + L*theta followed by refraction theta <- theta + phi(y), where phi is a coordinate-wise Gaussian basis field with K fixed control points, and L = T/D so that depth adds refraction stages without changing the total propagation scale. The readout is linear in y. The theory establishes that each transport layer is a global diffeomorphism with det J = 1 (Schur complement, with an explicit inverse), that the implemented lens — diagonal Jacobian, hence symmetric H — makes each layer symplectic with reciprocal singular values, exact parameter and arithmetic counts (depth-dominant ratio K/(h+1), about 9.3% at h=128, K=12; FLOP ratio about 29%), and a constructive universal-approximation result via Gaussian ridge expansions. Experiments learn one-step transition maps on ten synthetic systems (Lorenz-24/28, SpringChain-32/48, Burgers/AdvecDiff/ReactionDiff at two resolutions) and evaluate 500-step rollouts, against MLP/ResMLP/scaled-ResMLP/Res-BN/LayerScale baselines at fixed width 128. The strongest results are on dissipative PDEs: Burgers rollout MSE at 25.6%/27.9% of the best baseline with 5.5%/5.9% of its parameters; gradient transport at initialization is near-neutral (log10(g1/g40) between 0.000 and 0.008).

Critical strengths

The theory is simple, correct, and honestly scoped. I checked Proposition 1, Corollaries 1-2, and the counts in Proposition 2; the Schur-complement argument and the symplectic shear factorization are elementary and right, and the authors are careful to say that volume preservation is a bias, not a conditioning guarantee, and that the embedding/readout are left free so the end-to-end map is not constrained to be Hamiltonian. Closing the expressivity question with a constructive universal-approximation argument is the right move.

The empirical bookkeeping is more transparent than most architecture papers: a protocol index mapping every table to the exact protocol it came from (Table 3), a frozen multi-trajectory confirmation with disjoint seeds and perturbed initial conditions analyzed with seed-cluster bootstrap (Table 5), per-system configurations disclosed in full (Table 4), a matched-arithmetic audit, and component ablations. Whatever one thinks of the architecture, the evidence trail is auditable.

The gradient-transport experiment (Table 2) is clean and directly supports the motivation: LLN sits at g1/g40 between 1.00 and 1.02 on every system, where scaled ResMLP is at 0.087-0.333 in log10 and plain MLP/ResMLP are orders of magnitude away in both directions.

The efficiency story has exact numbers behind it rather than adjectives: O(DhK) versus O(Dh^2) depth-dominant parameters, and the measured parameter ratios in Table 1 follow from the formula.

Critical weaknesses

Three issues limit the reliability of the headline claims and the paper's ability to place itself in context; the rest I class as lesser concerns.

The stricter protocol reverses the chaotic-system results, and the abstract does not say so. In Table 1, Lorenz-24 favors LLN (ratio 0.821). In the frozen multi-trajectory confirmation (Table 5), LLN loses Lorenz-24 by 3.66x [1.18, 13.44], Lorenz-28 by 6.17x [4.77, 7.95], and SpringChain-32 by 2.05x. The body text reports this honestly, but the abstract's "LLN improves long-horizon prediction" carries no trace of it. There is also no analysis of why volume-preserving transport hurts precisely on chaotic dissipative flows.

The comparator protocol favors LLN. LLN's (T, h, K, s) are tuned per system (widths up to 512, Table 4) while every baseline runs at fixed h=128 and is never given a parameter-matched width. Learning rates are described only as "method-specific recorded settings", and the main-table rows use six-epoch runs. A wider baseline at LLN's parameter count is the obvious missing control.

No established baseline from the area appears in the experiments. The paper cites SympNets, VPNets, Hamiltonian networks, FNO and KAN as related families, yet the only comparators are generic MLP variants on 3-to-48-dimensional synthetic systems with no real data. For a paper titled around long-horizon dynamical prediction, this leaves the contribution unplaced: the reader cannot tell whether the phase-space prior beats the existing structure-preserving toolkits or merely beats plain MLPs.

Lesser concerns, listed briefly: in numerical-integration terms, propagate-then-refract is a kick-drift splitting — symplectic Euler with a learnable kick — and the paper never states this equivalence, which weakens the novelty positioning (the genuinely new pieces are the coordinate-wise Gaussian lens, the L = T/D depth-scale decoupling, and the learned initial angle); on the spring-chain systems the machinery appears inert (ablation deltas within noise, and on SC-32 the no-propagation ablation is best, Table 6), so the effective domain is dissipative spatial PDEs, narrower than the title suggests; there are no wall-clock numbers despite DhK exponential evaluations per forward pass; and Table 1 vs Table 5 use different aggregation conventions without a flag in the main text.

Questions for authors
  • Why does LLN fail on the Lorenz systems under the multi-trajectory protocol, and how should the abstract-level claim be qualified given Table 5?
  • Can you add parameter-matched baselines — a wider MLP / ResMLP / LayerScale at LLN's parameter count — at least for the PDE systems?
  • Can you compare against at least one structure-preserving baseline (SympNets, VPNets) on these systems, and state the relation to kick-drift symplectic Euler explicitly?
  • How were baseline learning rates and the six-epoch budget chosen, and do the conclusions survive longer training?
10 comments
AuthorsOpenReview ↗

W1 and Q1: Multi-trajectory generalization and the prediction claim

We thank the reviewer for the careful assessment. We have added capacity-controlled comparisons, established dynamical modeling baselines, extended benchmarks, and wall-clock measurements. We also explain how the multi-trajectory results define the scope of LLN's prediction benefits.

The multi-trajectory findings provide a more stringent assessment of generalization to unseen initial conditions. In Table 5, LLN has favorable mean error ratios on seven of ten systems, with five 95% seed-cluster-bootstrap intervals entirely below one. However, the ratios are 3.656 [1.180, 13.444] on Lorenz-24, 6.165 [4.769, 7.947] on Lorenz-28, and 2.051 [1.560, 2.741] on SpringChain-32. The results therefore support generalization benefits on a subset of systems, with the clearest overall gains on the evaluated dissipative spatial problems.

Tables 1 and 5 answer different questions. Table 1 evaluates a temporal split of a reference trajectory and reports arithmetic mean ± SD over training seeds. Table 5 uses four training trajectories and four disjoint test trajectories, frozen configurations, six confirmatory training seeds, and two 500-step windows per test trajectory. It reports geometric-mean MSE and seed-cluster-bootstrap ratio intervals. The two tables therefore differ in both their evaluation task and aggregation convention.

The Lorenz results indicate a limitation in transferring the learned transition map to new trajectories. Locally, rollout error evolves approximately as

δk+1≈DF(xk)δk+εk,\delta_{k+1}\approx DF(x_k)\delta_k+\varepsilon_k,

where εk\varepsilon_k is the local prediction error. Expansion along a trajectory can amplify these errors during rollout. A plausible architecture-specific contributor is LLN's restricted cross-coordinate interaction: the embedding and readout mix variables, while the internal lenses act coordinate-wise. This may make coupled transitions over a broader visited region harder to fit.

We also distinguish latent volume preservation from the dynamics of the observed prediction. The embedding and readout are unrestricted, so the observed-state map can be dissipative even when the latent transport has unit determinant. Volume preservation alone neither controls directional error amplification nor guarantees transfer to unseen trajectories. The favorable results on several dissipative PDEs also show that dissipation alone cannot explain the Lorenz outcome. Similarly, gradient transport across network depth and error growth across prediction time are separate properties.

The horizon sweep in Table R1 provides complementary evidence for LLN's rollout behavior under the reference-trajectory protocol. LLN has lower means than Plain MLP in all 50 system–horizon combinations and than all available finite MLP-family comparators in 45/50. On ReactionDiff-48, its error changes from 0.037 at 100 steps to 0.042 at 500 steps. These results show that the favorable comparison persists over the tested horizons; they do not replace the unseen-initial-condition evaluation.

The spring-chain ablations likewise indicate that component benefits depend on the task: removing propagation gives the lowest mean error on SpringChain-32.

Appendix Table 5 explicitly documents the limits of LLN’s generalization to unseen initial conditions. These results qualify the scope of our claims without overturning the main-text findings under the reported evaluation protocols: LLN achieves parameter-efficient long-horizon prediction, with the clearest gains on the evaluated dissipative spatial systems. The appendix therefore clarifies where these benefits extend and where they remain limited; our conclusions do not require uniform improvement across all dynamical systems.

AuthorsOpenReview ↗

W2 and Q2: Width and parameter-budget controls

We would like to clarify that the hyperparameters listed in Table 4 apply to the multi-trajectory evaluation in Table 5 and were not used to obtain the results in Table 1. In Table 1, LLN and the baseline models are compared at the same fixed depth DD and hidden width HH, with matched data splits and evaluation protocols. The per-system configurations in Table 4 therefore do not give LLN a width advantage in the Table 1 comparison. The additional parameter-matched comparisons further address capacity differences that remain even at equal depth and width.

We now provide two complementary controls. Table R2 holds width fixed at H=256H=256 or H=512H=512 while varying depth. At D=40D=40, LLN has lower mean errors than the best reported MLP-family comparator on 9/10 systems at H=256H=256 and 10/10 at H=512H=512, including 5/6 and 6/6 PDE settings. On ReactionDiff-48, error decreases from 5.619 to 0.241831 at fixed H=256H=256, and from 3.154 to 0.159100 at fixed H=512H=512, between depths 5 and 40. Thus, the deep-LLN advantage persists without increasing width across depths.

Table R3 directly addresses total parameter count: the MLP-family baselines use 98.2–102.4% of the corresponding LLN parameter budget. LLN has lower reported mean errors than every finite comparator on all ten systems. Across the six PDE settings, Plain MLP/LLN error ratios range from 2.01 to 7.00. This provides evidence that the accuracy advantage over dense MLP controls persists at approximately matched capacity.

AuthorsOpenReview ↗

W3 and Q3: Established baselines, architectural relation, and runtime

We have expanded the comparison to G-SympNet [1], VPNet [3], NeuralODE-RK4 [2], Linear Koopman, and FNO [4], with HNN-RK4 additionally evaluated on FPU-128. Table R4 compares the neural baselines at approximately matched LLN parameter budgets across reference depths; Linear Koopman is a separate fixed reference.

Within the tested LLN configurations, nine of ten systems reach their lowest mean error at D=40D=40, and eight have a smaller SD than at D=5D=5. At the D=40D=40 reference, LLN has the lowest mean among the evaluated methods on six of ten systems. These results extend the evidence for effective deep prediction beyond the original MLP comparisons. They also support empirical consistency across several system families, while leaving system-specific differences visible in the full table.

The numerical-integration connection follows directly from LLN's updates:

\theta_{l+1}=\theta_l+s\phi_l(y_{l+1}),\qquad L=T/D.$$ Under the usual position–direction convention, propagation followed by refraction is the drift–kick ordering of a symplectic-Euler-type composition. LLN's architectural choices are fixed linear propagation, coordinate-wise Gaussian refraction, $L=T/D$ depth–scale decoupling, and a learned initial direction. The Gaussian expansion makes local correction contributions directly examinable and limits the learned coefficients to $hK$ per layer. These choices define our contribution within established reversible and symplectic constructions. Table R5 adds FPU-128, RD2D-32, and CMU-35 walking. LLN has lower means than all evaluated MLP-family baselines on these three tasks and the lowest mean among all reported methods on CMU-35 walking, at $0.52030\pm0.01158$. The results broaden the evidence to coupled dynamics, two-dimensional spatial evolution, and real motion data. The reported uncertainties for LLN are small on all three tasks, supporting repeatability under these protocols. Table R6 addresses the cost of Gaussian evaluations through measured end-to-end inference, including the exponential and initial-angle nonlinearities. At the reported $D=40$, $H=512$, batch-size-1 FP32 setting, LLN has the lowest latency in all four hardware/thread configurations. It takes 0.200 and 0.203 ms on the two L20 GPUs, and 0.896 and 0.566 ms on CPU with one and eight threads. This shows that the compact parameterization also yields practical execution benefits in the tested implementations.
Reviewer_y78TOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes Learnable Lens Networks (LLN), a physics-inspired architecture that transforms features in an augmented position-angle phase space. Each layer combines linear propagation with nonlinear refraction through learnable Gaussian lens fields. The authors analyze invertibility, volume preservation, symplecticity, and approximation capability. Experiments on several synthetic dynamical systems show improved long-horizon prediction over MLP-based baselines, particularly for deeper networks.

Critical strengths

The main strengths of this paper are its clear presentation and the simplicity of the proposed architecture. The geometric analysis is complemented by experiments across several dynamical systems, including depth sweeps, gradient diagnostics, and architectural ablations. The results on some dissipative PDE systems are promising, with LLN achieving lower prediction errors than the compared MLP architectures.

Critical weaknesses
  1. Methodological novelty and relation to existing architectures.

The proposed architecture appears closely related to existing reversible and structure-preserving neural networks. The alternating position and angle updates have essentially the same form as the additive coupling layers in reversible residual networks (Gomez et al., 2017). The symplectic construction is also closely related to SympNets (Jin et al., 2020), which compose structured update modules to obtain symplectic transformations.

Moreover, the auxiliary angle variable and the resulting second-order dynamics resemble ideas in augmented and second-order neural ODEs (Norcliffe et al., 2020). Although the paper cites some of these works, it does not sufficiently explain what is fundamentally new beyond the optical interpretation and the Gaussian lens parameterization. The invertibility and volume-preservation results largely follow from the additive coupling structure. Likewise, the universal approximation result does not establish why the lens mechanism is necessary or advantageous. Overall, the methodological contribution appears relatively incremental. Direct comparisons with these closely related architectures are needed to clarify both the novelty and the practical value of the proposed design.

  1. Experimental scope and adequacy of the comparisons.

The experiments cover several dynamical systems and include useful ablations, but the settings remain relatively small and simple. The Lorenz systems are three-dimensional, and the spatial PDE experiments use only 32 or 48 discretization points. These settings do not sufficiently establish scalability to more challenging dynamical systems.

More importantly, all compared methods are MLP-based architectures. Such comparisons are useful for examining depth-wise training behavior, but are insufficient to establish LLN as a general dynamical prediction model. Comparisons with Koopman learning and neural ODE-based models would be informative, as would neural operators or convolutional architectures for spatial PDEs. Reversible, symplectic, and second-order networks are particularly relevant given the architectural similarities discussed above. The depth-dependent results in Figure 2 also require clarification. On ReactionDiff-48, the LLN error decreases from 2.244 at depth 5 to 0.042 at depth 40, whereas the plain MLP error remains around 0.29. However, the selected LLN configurations also change in width and substantially increase in parameter count. It is therefore unclear how much of the improvement is attributable to the architecture rather than increased capacity. Although LLN uses fewer parameters than the MLP baselines at matched depths, accuracy comparisons under matched parameter or computational budgets would be more convincing.

Finally, generalization to unseen initial conditions remains insufficiently established. The multi-trajectory results in Appendix Table 5 do not show consistent improvements over the baselines. Together with the limited scale of the main benchmarks, this leaves the general applicability of LLN unclear. Evaluation on more challenging, higher-dimensional systems would help address this concern.

References

Gomez et al. (2017). The Reversible Residual Network: Backpropagation Without Storing Activations. NeurIPS 2017.

Jin et al. (2020). SympNets: Intrinsic Structure-Preserving Symplectic Networks for Identifying Hamiltonian Systems. Neural Networks, 132, 166–179.

Norcliffe et al. (2020). On Second Order Behaviour in Augmented Neural ODEs. NeurIPS 2020.

Questions for authors
  1. Could the authors clarify the architectural differences between LLN and existing reversible or symplectic networks, particularly RevNet and SympNets? In particular, what advantages does the Gaussian lens parameterization provide beyond the general properties of additive coupling and symplectic transformations?

  2. Could the authors provide comparisons with stronger dynamical modeling approaches, including Koopman learning, neural ODEs, and neural operators where appropriate? Accuracy-versus-parameter and accuracy-versus-cost comparisons across depths, together with larger-scale experiments and evaluation on unseen initial conditions, would help assess the practical advantages and generality of LLN.

7 comments
AuthorsOpenReview ↗

W1 and Q1: Relation to reversible and symplectic networks, and the role of the Gaussian lens

We thank the reviewer for the careful assessment. We address these concerns by clarifying LLN’s architectural choices and their relation to prior work, followed by the supplementary comparisons, depth–width tests, and extended benchmarks.

We agree that LLN shares important structural foundations with existing methods. RevNet uses additive coupling to construct reversible transformations, SympNets compose structured modules to preserve symplecticity, and augmented and second-order neural ODEs introduce auxiliary states to enrich the dynamics. LLN builds on these ideas. Its contribution lies in a specific architecture that combines fixed linear propagation with learned, localized directional corrections, making the internal computation interpretable while limiting the number of parameters learned at each layer.

In a RevNet block, both coupling functions are learned. LLN fixes one of these functions to linear propagation and learns the other as a refraction field:

\theta_{l+1}=\theta_l+\phi_l(y_{l+1}),\qquad L=T/D.$$ The current direction determines the displacement, while the lens changes the direction of subsequent propagation. This constraint gives position and angle distinct computational roles: the optical interpretation corresponds to how the updates operate. The connection to SympNets is also direct: LLN’s update is a composition of symplectic shears, related to a drift–kick discretization. SympNets provide broader constructions using linear, activation, and gradient modules; LLN adopts a fixed linear propagation step and a coordinate-wise Gaussian expansion for the learned correction. Its invertibility and volume preservation follow from the coupling structure, and its symplecticity follows from the symmetric lens Jacobian. These properties are shared mathematical foundations, not properties unique to Gaussian lenses. The Gaussian parameterization contributes a concrete way to examine the learned correction: $$\phi_{l,j}(y_j)=\sum_{k=1}^{K}a_{l,j,k} \exp\ \left[-\left(\frac{y_j-c_k}{b}\right)^2\right].$$ With fixed centers and bandwidth, each learned coefficient controls the sign and strength of a response concentrated around a known location in the latent coordinate. Plotting the resulting lens function reveals where and how it changes the propagation direction. For an individual sample, the correction can also be separated into its basis contributions. This connects the learned parameters to a specific operation on the evolving representation. That connection extends across layers. The displacement $L\theta_l$ and correction $s\phi_l(y_{l+1})$ can be examined separately, allowing a change in direction to be associated with a particular layer and lens response. Figures 4–5 illustrate this analysis in the frozen two-moons model: we follow a misclassified boundary sample and nearby correctly classified samples through their position paths, layerwise turns, and final readout. The interpretation therefore links the learned lens functions to feature evolution and the classification outcome. It concerns computation in latent space; the case study does not establish causal explanations or greater interpretability than SympNets. The same parameterization also controls model size. Each layer learns $hK$ lens coefficients, yielding an $O(DhK)$ depth-dependent parameter term, compared with $O(Dh^2)$ for dense hidden transformations. For a small fixed dictionary, this allows depth to increase with a limited parameter cost. The saving comes from the coordinate-wise dictionary expansion, rather than Gaussian functions alone. The corresponding trade-off is restricted cross-coordinate interaction within the stack, making the embedding, initial directional state, and width important for representing coupled dynamics. Finally, our universal approximation result establishes the expressivity of the full architecture with its embedding and readout. Likewise, the lens-removal ablation evaluates the usefulness of learned refraction within LLN. Our central contribution is the combination of an explicit propagation–refraction computation, directly examinable lens functions, and a compact parameterization. The prediction experiments and component analyses assess the practical value of this design beyond its shared structural properties.
AuthorsOpenReview ↗

W2 and Q2: Baseline selection, approximately matched capacity, depth–width tests, and extended benchmarks

We thank the reviewer for these suggestions. The original MLP-family comparisons were designed to examine feature transport and training behavior as depth increases. We agree that these comparisons alone are insufficient to assess LLN as a dynamical prediction model. We have therefore added comparisons with related modeling approaches, fixed-width depth sweeps, extended benchmarks, and inference-time measurements. We also clarify the evidence for generalization to unseen initial conditions.

Comparisons with dynamical modeling approaches. Table R1 adds Linear Koopman, NeuralODE-RK4 [2], G-SympNet [1], VPNet [3], and FNO [4].

For the neural baselines in Table R1, model sizes were chosen to approximately match the total parameter count of the corresponding LLN configuration. The first column identifies the LLN reference depth, not a common architectural depth across model families. Linear Koopman is a separate fixed reference.

Table R1 provides evidence that LLN can sustain effective prediction as the stack becomes deeper. Within the tested LLN configurations, nine of ten systems reach their lowest mean error at D=40D=40, and eight show a smaller SD at D=40D=40 than at D=5D=5. At the D=40D=40 reference, LLN also has the lowest reported mean among the evaluated methods on six of ten systems. For example, ReactionDiff-48 improves from 2.244±0.9502.244 \pm 0.950 at D=5D=5 to 0.042±0.0310.042 \pm 0.031 at D=40D=40, combining lower prediction error with reduced variation across runs. These observations support the empirical reliability of the deeper LLN configurations across several system families.

Separating width changes from depth effects. We agree that Figure 2 compares selected configurations and therefore cannot isolate depth from changes in width and capacity. Table R2 adds depth sweeps with width held at H=256H=256 or H=512H=512. Each LLN result is compared with the lowest reported finite MLP-family mean at the same depth and width.

The deep-LLN advantage persists at both tested widths: at D=40D=40, LLN has lower means than the best reported MLP-family comparator on 9/10 systems at H=256H=256 and 10/10 at H=512H=512. On ReactionDiff-48, its error decreases from 5.619 to 0.241831 as depth increases from 5 to 40 at H=256H=256, and from 3.154 to 0.159100 at H=512H=512. The corresponding LLN standard deviations at D=40D=40 are 0.012370 and 0.001181, while the best MLP-family mean errors are 2.619 and 2.620. Thus, the deeper LLN configurations combine improved accuracy with small run-to-run variation on this example, and the gains persist when width is held fixed.

The fixed-width sweeps and the approximately parameter-matched comparisons address complementary aspects of the capacity concern. The former show that the gains are not explained solely by increasing width; the latter compare architectures at similar parameter budgets.

Extended benchmarks. Table R3 reports results on FPU-128, RD2D-32, and CMU-35 walking, extending the evaluation to coupled dynamics, two-dimensional reaction–diffusion, and real motion data beyond the original systems.

LLN maintains consistent performance across the three added task types, with lower mean errors than all evaluated MLP-family baselines and small reported variation across runs: 0.78434±2.34×10−60.78434 \pm 2.34\times10^{-6} on FPU-128, 0.78715±0.003800.78715 \pm 0.00380 on RD2D-32, and 0.52030±0.011580.52030 \pm 0.01158 on CMU-35 walking. On the real motion benchmark, it also achieves the lowest mean among all reported methods. The improvement over Plain MLP on RD2D-32 is modest, but the combined results extend the evidence for repeatable performance from the original systems to coupled dynamics, two-dimensional spatial evolution, and real observations.

Inference cost. To complement the parameter comparisons, Table R4 reports end-to-end inference latency, including the nonlinear operations in each implementation, on two L20 GPUs and CPU configurations with one and eight threads. The reported settings are D=40D=40, H=512H=512, batch size 1, FP32, and three initialization seeds.

LLN has the lowest reported latency in all four hardware configurations: approximately 0.20 ms on either GPU, 0.896 ms on the single-thread CPU, and 0.566 ms with eight CPU threads. This complements the accuracy and parameter comparisons with measured execution cost. The measurements apply to the reported configuration and do not establish an accuracy–cost frontier across depths.

Generalization to unseen initial conditions. Appendix Table 5 evaluates disjoint training and test trajectories using six confirmatory training seeds and seed-cluster bootstrap analysis. LLN achieves a lower mean error ratio on seven of ten systems, with five intervals entirely below one. This supports the persistence of its prediction benefits beyond the training trajectories on several systems.

Together, these experiments strengthen the empirical case for LLN's ability to sustain prediction performance at larger depth, with repeatable results on the extended benchmarks and low inference cost across the tested hardware. The main evidence is the consistency of its deep-model advantages across both fixed-width settings, supplemented by favorable unseen-trajectory results on several systems. This supports LLN as a practical, parameter-efficient architecture for the evaluated tasks, with further scope for tuning and broader validation.

We thank the reviewer again for the thoughtful comments, which have helped us clarify LLN’s contributions and strengthen the experimental evaluation. We hope that the explanations and additional results address your concerns.

References

[1] Jin et al. (2020). SympNets: Intrinsic structure-preserving symplectic networks for identifying Hamiltonian systems. https://arxiv.org/abs/2001.03750

[2] Chen et al. (2018). Neural Ordinary Differential Equations. https://arxiv.org/abs/1806.07366

[3] Zhu et al. (2022). VPNets: Volume-preserving neural networks for learning source-free dynamics. https://doi.org/10.1016/j.cam.2022.114523

[4] Li et al. (2021). Fourier Neural Operator for Parametric Partial Differential Equations. https://arxiv.org/abs/2010.08895

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.