- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Higher-Order-KANDy for discovering higher-order ordinary and partial differential equations from observed data. The work addresses two challenges in conventional equation discovery methods. First, converting higher-order differential equations into first-order state-space systems may introduce additional state variables and considerably expand the candidate function library. Second, estimating higher-order derivatives directly from noisy observations can significantly amplify measurement noise.
To address these challenges, the authors combine delay embeddings, derivative (jet) embeddings, and derivative-ordered KANDy layers. The proposed framework first reconstructs derivative coordinates from delayed observations or obtains them through smooth function fitting. KANDy layers with learnable spline functions are then used to identify the right-hand side of the governing equation. The authors argue that this formulation allows higher-order equations to be identified directly while reducing the need to specify fixed nonlinear candidate functions in advance.
The theoretical analysis draws on delay embedding theory and the Kolmogorov–Arnold representation theorem to establish properties of the delay-to-jet mapping. The method is evaluated on several synthetic ODEs, PDEs, and systems with hidden states. The experiments examine equation recovery, coarse sampling, and noise robustness, with comparisons against SINDy, WSINDy, and other KAN-based methods.
Critical strengths
- A potentially useful formulation for higher-order dynamics discovery
The paper attempts to identify governing equations directly in their native higher-order form rather than relying entirely on first-order state-space reformulations. This may help avoid unnecessary candidate terms and preserve a representation that is more closely related to the original physical equation.
The theoretical discussion of the delay-to-jet mapping also provides some justification for the proposed architecture. In particular, the analysis of additive and compositional representations helps explain why a more expressive lifting map may be needed under certain sampling conditions.
This formulation may be useful for problems where preserving the original differential order is important.
- Promising results on selected nonlinear PDEs and coarsely sampled systems
The proposed method shows encouraging results on several synthetic benchmarks. For instance, it recovers the nonlinear restoring force in the sine-Gordon equation without explicitly including in the candidate functions. Table 1 reports good support recovery under the authors' evaluation criteria.
The learned delay-to-jet mapping also performs better than conventional differentiation or smoothing approaches in some coarsely sampled settings. These results suggest that the proposed framework may be useful for certain additively separable nonlinear equations and undersampled dynamical systems.
Critical weaknesses
- The library-free claim is overstated, and the benefits in interpretability and performance are not sufficiently established
The authors emphasize that Higher-Order-KANDy can discover governing equations without a candidate library, as illustrated by recovering the nonlinear function without explicitly providing it in advance.
However, this capability largely follows from the flexible function approximation provided by KAN spline edges. Such flexibility also comes at a cost to interpretability. A learned spline may have an irregular shape that does not correspond directly to a recognizable analytical expression. Additional symbolic fitting or regression may still be needed to determine the underlying functional relationship. This process may itself require candidate function families or other assumptions about the symbolic structure.
Therefore, avoiding a predefined symbolic library during training does not necessarily mean that the method directly produces interpretable governing equations.
Furthermore, Eq. (3) in Section 2.1 still introduces a feature library. Although the authors use free-form edges in some experiments, composite features must still be specified for equations involving multiplicative or non-additively separable interactions. The library-free capability is therefore restricted to particular equation structures and does not directly extend to general nonlinear dynamical systems.
In my view, the method mainly replaces fixed nonlinear basis functions with more flexible learnable splines. This may improve representation flexibility, but it can also introduce additional costs in symbolic interpretation, model complexity, and training. These trade-offs are not adequately evaluated.
More importantly, the experiments in Supplementary Section S6 show that PySINDy-STLSQ achieves better functional-form estimation accuracy when the correct term is included in its library and the same derivative data are used. Although Higher-Order-KANDy shows better exact-support recovery under the authors' evaluation protocol, trigonometric functions are common candidate functions in SINDy and can be included without much difficulty. The advantage over a polynomial library that excludes is therefore not sufficient to demonstrate a general performance improvement over a reasonably configured SINDy method.
There is also a concern about the support evaluation itself. Higher-Order-KANDy identifies symbolic terms through post-hoc shape matching of learned spline functions, whereas WSINDy selects explicit candidate functions through their coefficients. These are not fully equivalent criteria. A small residual component within a learned spline may not be counted in the same way as an explicitly selected spurious term in WSINDy. A common symbolic read-out and support evaluation procedure would make the comparison more convincing.
Overall, I find the claims regarding library-free discovery, interpretability, and performance improvement stronger than what the current evidence supports.
- The treatment of derivative noise relies substantially on existing techniques, while the necessity and practical relevance of the proposed architecture remain unclear
The amplification of noise during higher-order derivative estimation is presented as one of the main motivations for this work. The authors propose a delay-to-jet embedding strategy to address this problem.
However, the proposed method does not eliminate the need for higher-order derivative estimation. Instead, it appears to move a substantial part of this difficulty to the construction of derivative supervision during training or calibration.
According to Eq. (9), the training objective still requires first-order and highest-order derivative targets, which are obtained using conventional techniques such as smoothed finite differentiation and Savitzky–Golay filtering. In the PDE experiments, the derivative coordinates are similarly obtained by fitting smooth functions to the observed data.
Consequently, it is not clear how much of the reported improvement in noise robustness comes from the proposed Higher-Order-KANDy architecture rather than from established preprocessing and derivative estimation techniques.
Some coarse-sampling experiments also rely on additional densely sampled calibration trajectories or high-quality derivative targets. This raises concerns about how well the method would perform when only noisy or sparsely sampled observations are available throughout both training and deployment.
Another important limitation is the absence of a direct KAN baseline. Specifically, the paper does not compare the proposed architecture against a standard KAN that directly learns the mapping from the delay embedding to the target highest-order derivative , without an explicit jet embedding stage.
Such an ablation is important for establishing whether the derivative-ordered architecture itself improves modeling accuracy and robustness. Although the authors compare linear, additive KAN, and compositional KAN lifting maps, these experiments do not directly establish the benefit of the complete Higher-Order-KANDy framework over a capacity-matched direct KAN.
The general applicability of the method is also insufficiently demonstrated. The main formulation focuses on scalar observations, and most experiments involve second- or third-order systems and selected synthetic PDEs. It remains unclear how the method would scale to high-dimensional vector-valued states or dynamical systems with strong interactions among multiple variables.
Finally, the practical advantage of the compositional delay-to-jet mapping appears to be restricted to a particular sampling regime. The observations must be too coarse for conventional differentiation to remain accurate, but still sufficiently informative for the delay mapping to preserve the information needed for state reconstruction.
The paper's own theoretical analysis and experiments show that conventional linear differentiation or smoothing splines can be competitive or even superior when sufficiently dense observations are available.
If practical measurement systems can readily provide samples that are sufficiently dense relative to their characteristic dynamical timescales, the benefit of the proposed nonlinear lifting strategy may be considerably reduced.
Although the experimental study covers several systems, many of the governing equations have relatively simple or special structures. The current results do not convincingly establish the practical value of the proposed approach for more general and complex dynamical systems.
Questions for authors
- Candidate-library-free discovery and interpretability
Could the authors clarify the distinction between avoiding a predefined library of analytical functions and avoiding predefined candidate feature structures?
For systems involving non-additively separable interactions, how are the feature library and composite terms determined?
When a learned KANDy spline has a complicated or irregular shape, how is it reliably converted into an explicit symbolic expression? Does this post-hoc procedure require a candidate function family, and what are its computational and interpretability costs?
- Derivative noise and supervision requirements
To what extent does the proposed jet embedding actually address the high-order differentiation problem, rather than transferring it to the construction of derivative supervision during calibration?
In particular, how would the method perform if only the noisy or coarsely sampled observations available at deployment were also available during training, without additional densely sampled calibration trajectories or high-quality derivative targets?
It would be helpful if the authors could clarify the sources of derivative supervision and calibration data used in the main experiments, as well as the conditions under which the learned mappings generalize to new observations.
- Necessity of the explicit jet embedding architecture
Why is a standard KAN that directly maps the delay embedding to the target highest-order derivative not included as a baseline?
I suggest comparing a direct KAN against Higher-Order-KANDy using identical input data, derivative targets, comparable model capacity, and similar training budgets.
This comparison would help establish whether the explicit jet representation provides measurable benefits in prediction accuracy, noise robustness, or training efficiency beyond making the model structure more interpretable.
- Fairness of the SINDy comparisons and the practical performance advantage
Supplementary Section S6 shows that STLSQ achieves better functional-form estimation accuracy when the correct term is provided in the candidate library.
Since trigonometric functions are commonly used in SINDy libraries, could the authors clarify under which conditions Higher-Order-KANDy offers a meaningful advantage over a reasonably configured SINDy or WSINDy model?
Could a common post-hoc symbolic decomposition and support evaluation procedure also be applied to both the free-form KANDy functions and the explicit SINDy candidate terms?
- Model dimensionality and practical application scenarios
The formulation in Section 2 mainly considers a scalar observable . How would the framework extend to high-dimensional vector-valued states or systems with strong interactions among multiple state variables?
More importantly, the main advantage of the compositional delay-to-jet mapping appears to arise within an intermediate sampling regime where conventional differentiation becomes inaccurate but the delay observations remain sufficiently informative.
Could the authors clarify how commonly this regime occurs in realistic scientific measurement settings? Is obtaining denser observations typically difficult or costly in the applications targeted by this work?
If sufficiently dense observations are readily available, the practical value of the learned lifting approach over conventional smoothing and derivative estimation methods may be limited.
- Minor Comment
Please check the figure references throughout the main text and supplementary material. Every figure should be explicitly referenced and discussed in the corresponding text.
2 comments
Thanks for the clarifications and the additional experiments (including the direct KAN ablation, the improved derivative estimates and the extension to the coupled systems). That all helps, but my fundamental concerns are still there.
- Architecture and interpretability (Q1, Q3). The direct KAN achieves similar prediction accuracy so the jet architecture's advantage is largely confined to interpretability. Composite features are still required for non-additive dynamics and symbolic interpretation is still dependent on domain informed fitting. The reliability and additional value of this interpretation is not yet established.
- SINDy comparisons (Q4). The authors do admit that the threshold choices may have biased the benchmark and that sparse methods will in fact do a better job of estimating the coefficients if the correct terms are available. A corrected comparison using consistent symbolic support evaluation is still to be carried out and so the claimed advantage is not settled.
- Derivative supervision (Q2). The denoising results are quite promising, but exceeding the training targets doesn't prove anything in the absence of dense calibration trajectories or any real derivative supervision during training.
These responses do clarify the approach in terms of its scope but still don't provide enough to allay my concerns about its further contribution and practical benefit. I therefore remain at my current rating.
Thank you for your review!
Dear Reviewer TWy4,
Thank you for the thorough review. We agree with several of your points.
Q1. Candidate-library-free discovery and interpretability.
In the Genesio–Tesi and sine-Gordon experiments, the equation layer sees only the raw coordinates, and terms such as x2 and sin(u) are read from the edges. We supplied uux for KdV and ∣x˙∣z3, x˙∣z∣3 for the Bouc–Wen memory law. A suggested default for an unknown oscillatory system is a Duffing-style oscillator library as shown in Equation (4). Applying a function approximation to a learned spline requires practitioner domain knowledge, and it is computationally efficient to try many combinations. When applied to data, the splines can be irregular in shape. For example, we tested our method on biomedical data where ground truth was unknown, and the existence of a dynamical system is not necessarily guaranteed. This testbed motivated many of our algorithm’s design principles, including the jet for denoising and reintroducing the library for non-additive dynamics.
Q2. Derivative noise and supervision requirements.
The proposed jet embedding addressed the high-order differentiation problem surprisingly well. The jet embedding is consistently more accurate than the training targets: 0.981 vs. 0.964 (Δ= 0.05, 5% noise), 0.918 vs. 0.884 (Δ = 0.2, 5% noise), and 0.987 vs. 0.970 (Δ = 0.4, 1% noise). This experiment will appear in the final version.
Q3. Necessity of the explicit jet embedding architecture.
We benchmarked a KAN without the intermediate layers, and the performance was roughly the same. The additional architecture delivers extra interpretability. Accuracy is the same: held-out R2 of 0.9993 for our model against 0.9991 to 0.9998 for the direct models. The jet stage added equational interpretability. Our model returns x’‘’=−ax¨−bx˙−cx+qx2 with coefficients within 7%.
Q5. Fairness of the SINDy comparisons and the practical performance advantage.
Given the correct column, sparse regression and WSINDy estimate coefficients better. The method helps when the law has no sparse representation in the library. Two cases are in the paper. A polynomial library cannot span sin(u) over the full period of the collision (forming RMS 0.707 at every noise level, including zero). A damping term κtanh(νut) with unknown ν does not lie in a finite set of columns, and the free-form edges recover it correctly.
In Messenger & Bortz, “Weak SINDy for partial differential equations,” Journal of Computational Physics 443:110525 (2021), doi: 10.1016/j.jcp.2021.110525 (not the arXiv version), there are circumstances where WSINDy can return spurious terms.
In Appendix B.6, Eq. (B.9), p. 24 of the published JCP version, it is described that with standard STLS thresholding instead of their modified MSTLS, WSINDy returns a mixture of the true term and its Taylor expansion, roughly:
α(u−61u3+⋯)+(1−α)sin(u)
So the polynomial terms partly stand in for sin(u), even though sin(u) is in the library and can be resolved through a choice of thresholding methods.
In our benchmark, we didn’t use the MSTLS thresholds reported in their paper; we used some of PySINDy’s threshold settings, which may have skewed this benchmark. Likewise, “re-sparsifying” both KANDy and Higher-Order-KANDy models by applying MSLS or STLSQ changes the results and often returns machine-precision coefficients. We will re-evaluate this benchmark in the final version of the paper. As KAN-based models and data-driven dynamical systems methods converge, we believe a more coherent comparison will contribute positively to the literature.
We will tone down our claim in the abstract and introduction regardless. We additionally added trig libraries to the STQLS benchmarks. Although these remain accurate estimates, the trig terms do not always return the correct function.
Our core thesis is that Higher-Order-KANDy performs under noise with fewer hyperparameters (in the data-driven dynamical systems sense, e.g., thresholds and dictionary terms), and this benchmark shows that. And, per your comment, unknowns remain, such as spline interpretability and architecture choice. This work attempts to apply proofs to motivate the choice of architecture, an underutilized property of KANs: constraints on the spline functions translate directly into hypotheses used to derive theoretical proofs about architectural capabilities.
Q5. Model dimensionality and practical application scenarios.
Our framework extends naturally to higher-dimensional oscillators and coupled systems. We tested it on up to 16 coupled Duffing oscillators and still recovered the equations without specifying the coupling term in advance. This additional benchmark will be in the final version of the paper.