- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes LoKAN, a single-layer functional-edge model for equation discovery. Given a task-specified candidate feature map, the method constructs fixed anchored piecewise-linear spline bases and jointly estimates the coefficients of all active incoming edges at each output node through ridge regression. It then iteratively prunes edges according to their normalized output contributions on the training data and performs a final refit on the selected support. Since the model is linear in its edge coefficients once the bases are fixed, coefficient estimation does not require backpropagation. The paper establishes the uniqueness of the solution for a fixed representation, support, and regularization, and characterizes the conditions under which the system can be partitioned into exact component-wise solves based on its cross-Gram structure. The model’s expressivity depends on the candidate features: raw-coordinate candidates yield additive nonlinear models, explicit interactions require corresponding features in the candidate map, and two-knot linear edges allow coefficient estimation for explicit candidate equation terms. The experiments cover noisy dynamical systems, measured system-identification benchmarks, correlated and missing candidate terms, nonlinear function fitting, and joint-estimation ablations. The paper reports the lowest test NRMSE on all six noisy-derivative tasks and all four candidate-term fitting tasks, and demonstrates faster fitting than BP-KAN and Spline Group Lasso in a separate efficiency evaluation.
Critical strengths
-
The paper presents a clear and implementable pipeline for joint estimation and structural selection. The fixed spline representation reduces coefficient estimation to a linear-system solve; node-level joint estimation preserves cross terms among correlated edges; and contribution-based pruning followed by final refitting connects coefficient estimation with structural selection. This design is practically useful for scientific modeling tasks that require repeated fitting of candidate equations. The joint-estimation ablations also show that independent edge-wise fitting can lead to substantially larger errors.
-
The experiments cover multiple scientific modeling settings and clearly demonstrate the role of the candidate representation. Experiments on dynamical systems, measured system-identification sequences, correlated candidate terms, and nonlinear function fitting provide evidence that the method is applicable across several settings. In particular, the comparison between raw inputs and inputs augmented with interaction features clearly illustrates how the candidate map determines the expressive capacity of the single-layer model. This explicit discussion of the method’s applicability and limitations helps clarify how LoKAN should be interpreted and used.
Critical weaknesses
-
The paper does not clearly establish the methodological gap that motivates LoKAN. In the single-layer setting with a fixed candidate map and fixed basis functions, the model is linear in its learnable coefficients. Both direct coefficient estimation without backpropagation and joint estimation that preserves coupling between correlated features can therefore be achieved through standard least squares or ridge regression. In particular, with two-knot linear edges, the method is closely related to sequentially thresholded ridge regression over a candidate library; with multiple knots, it is directly related to additive regression with fixed spline bases. Although the paper discusses these related approaches, it does not sufficiently explain their remaining limitations or how LoKAN’s specific design addresses them. The algorithmic integration is clear, but the substantive advance beyond existing methods remains insufficiently established.
-
The connection between the structure-selection mechanism and reliable equation identification with correlated candidates is insufficiently justified. Joint ridge regression can stabilize coefficient estimation on a fixed support, but uniqueness of the regularized solution does not imply correct attribution to candidate terms or recovery of the correct equation structure. LoKAN prunes edges according to the magnitude of their fitted outputs. This score reflects the contribution allocated by the current joint fit, rather than whether a candidate remains necessary after the other edges are refitted. Consequently, correlated redundant terms may be retained, while weak but necessary true terms may be removed. The paper does not provide sufficient justification for why this selection rule should improve structural identification under the candidate correlations emphasized in its motivation.
-
The result-selection and reporting protocol is insufficient to establish a consistent performance advantage. Section 4 states that the lowest test NRMSE among the recorded runs is reported, but it does not clearly specify whether the minimum is selected across random seeds, hyperparameter configurations, or training checkpoints. Reporting only the best-performing run makes it difficult to determine whether LoKAN’s advantage is consistent and may make the comparison sensitive to differences in the number of trials or variability across methods. The relationship between this statement and the validation-based configuration selection followed by final test evaluation described in Section 3.3 also requires clarification. A clearly specified selection protocol and statistics across repeated runs are needed to support the empirical conclusions.
Questions for authors
-
What specific shortcomings of existing joint ridge regression and candidate-library sparse regression methods remain unresolved in the single-layer, fixed-basis setting considered here? Please identify LoKAN’s key additional mechanism relative to these approaches and explain how it provides a substantive benefit for the equation-discovery tasks studied in the paper.
-
Why is the magnitude of an edge’s fitted output an appropriate structure-selection criterion when candidate terms are correlated? Please explain how this criterion distinguishes necessary true terms from redundant terms that receive substantial contributions in the current fit, and how it translates stable coefficient estimation on a fixed support into more reliable equation identification.
-
How is the “lowest test NRMSE” in Section 4 selected, and is the test set used to select configurations or checkpoints? Please specify the selection scope and number of runs for each method, and explain how this protocol corresponds to the validation-based selection procedure in Section 3.3. Could the authors report test means together with standard deviations or confidence intervals across repeated runs after selecting and fixing configurations using the validation set, so that the consistency of the reported advantages can be assessed?
4 comments
W2 / Q2: Contribution scoring and structure identification with correlated candidates
The contribution score evaluates each entire edge using its actual function output. LoKAN performs selection after joint fitting and refits the retained edges. On the correlated-distractor and weak-true-term tasks from the original Section 4.2, we compare models with and without pruning using the same data, basis functions, and joint ridge solver. We retain the original validation configurations frozen in advance and report all five paired seeds.
Table 2. Five-seed structure selection ablation under matched conditions.
| Task | Method | Support F1 ↑ | Exact support ↑ | False positives ↓ | Weak-term retention ↑ | Test NRMSE ↓ |
|---|---|---|---|---|---|---|
| Correlated distractors | Without pruning | 0.75 ± 0.00 | 0/5 | 2.0 ± 0.0 | — | 0.02134 ± 0.00030 |
| Correlated distractors | Contribution-based pruning | 1.00 ± 0.00 | 5/5 | 0.0 ± 0.0 | — | 0.02132 ± 0.00030 |
| Weak true terms | Without pruning | 0.60 ± 0.00 | 0/5 | 4.0 ± 0.0 | 5/5 | 0.02340 ± 0.00057 |
| Weak true terms | Contribution-based pruning | 1.00 ± 0.00 | 5/5 | 0.0 ± 0.0 | 5/5 | 0.02334 ± 0.00063 |
Values are means ± sample standard deviations; counts indicate successes out of five, and “—” indicates that no weak true term is present. Both strategies fix , use the same base regularization parameter, and refit on the combined training and validation sets. Support is measured using the terms actually retained, excluding the intercept.
Pruning exactly recovers the support on both tasks (5/5), removes all false positives, and retains the weak true terms (5/5). Test NRMSE remains essentially unchanged, so the benefit is directly reflected in structure recovery.
To examine whether a term remains necessary after the other terms are refitted, we perform 20 edge-deletion-and-refitting evaluations from a common initial model, holding the basis functions, standardization, and ridge scale fixed. Deleting a true interaction term increases validation MSE by an average of 0.0213 and 0.00128 on the two tasks, respectively; deleting the most strongly correlated distractor produces an average change of approximately on each task. Across all five seeds, the contribution scores rank true terms and distractors consistently with the post-refitting loss rankings, supporting term identification in this setting.
Contribution magnitude and the loss after deletion and refitting have different meanings, and the results above apply to the candidate libraries tested. When , the contribution score is proportional to the coefficient norm, and the retained sets are identical after mapping the thresholds accordingly.
W3 / Q3: Selection procedure and statistics across runs
For each method and task, we conduct runs with five seeds using the same data splits and predefined, comparable search budgets. For each seed, hyperparameters and, where applicable, training checkpoints are selected using the validation set. Once these choices are frozen, the test set is used only for final evaluation. LoKAN's model is determined by the pruning stopping rule and final refitting.
The “lowest test NRMSE” in the original paper refers to the minimum across the five seed runs, rather than selection across hyperparameters or checkpoints based on test error. One-step prediction and rollout are summarized as separate tasks, and their respective minima may correspond to different seeds. We now additionally report the mean and sample standard deviation across all five runs to show typical performance and run-to-run variability.
Based on unrounded means, LoKAN achieves the lowest mean error on 11 of the 17 tasks, with an average rank of 1.59. Equation term recovery fixes , whereas nonlinear function learning selects using the validation set; structural metrics for these two types of tasks are interpreted separately.
Table 3. Test NRMSE (×10⁻³) across five runs, reported as mean ± sample standard deviation. Bold indicates the lowest unrounded mean for each task.
| Task / summary metric | LoKAN | SR3 | Spline GL | BP-KAN | HKAN | MatrixKAN |
|---|---|---|---|---|---|---|
| LV, direct | 0.8274 ± 0.15 | 0.9264 ± 0.076 | 1.558 ± 0.35 | 13.24 ± 0.97 | 164.2 ± 15 | 13.02 ± 2.0 |
| Duffing, direct | 1.951 ± 0.56 | 1.794 ± 0.27 | 2.910 ± 0.18 | 7.708 ± 1.4 | 77.14 ± 26 | 20.97 ± 2.1 |
| VdP, direct | 0.8636 ± 0.32 | 1.173 ± 0.17 | 1.158 ± 0.18 | 5.664 ± 0.88 | 263.7 ± 11 | 12.14 ± 0.40 |
| Lorenz-63, direct | 0.9559 ± 0.10 | 1.418 ± 0.20 | 2.076 ± 0.30 | 21.02 ± 1.0 | 145.9 ± 6.9 | 37.60 ± 0.57 |
| FHN, direct | 0.9941 ± 0.26 | 0.9927 ± 0.25 | 1.885 ± 0.055 | 4.019 ± 0.37 | 389.3 ± 6.1 | 25.96 ± 0.55 |
| Rossler, direct | 1.036 ± 0.061 | 1.969 ± 0.13 | 2.063 ± 0.18 | 18.93 ± 0.68 | 167.1 ± 21 | 38.72 ± 0.50 |
| LV, state | 30.28 ± 1.5 | 32.59 ± 2.5 | 31.40 ± 1.7 | 35.71 ± 1.4 | 188.6 ± 9.4 | 41.09 ± 5.5 |
| Duffing, state | 16.90 ± 0.53 | 24.92 ± 2.9 | 18.15 ± 0.38 | 25.89 ± 1.1 | 125.1 ± 52 | 17.39 ± 1.5 |
| VdP, state | 134.5 ± 2.3 | 136.5 ± 0.58 | 137.0 ± 0.57 | 136.0 ± 0.86 | 340.6 ± 8.2 | 140.6 ± 0.30 |
| Lorenz-63, state | 753.3 ± 3.0 | 753.3 ± 3.1 | 753.2 ± 3.1 | 750.8 ± 1.4 | 759.5 ± 1.2 | 760.2 ± 2.2 |
| FHN, state | 84.83 ± 1.3 | 86.87 ± 0.74 | 90.49 ± 1.9 | 87.10 ± 1.8 | 579.9 ± 3.9 | 127.2 ± 0.86 |
| Rossler, state | 435.1 ± 71 | 475.5 ± 28 | 476.3 ± 28 | 470.9 ± 30 | 482.9 ± 26 | 486.9 ± 24 |
| N-body | 55.39 ± 11 | 61.19 ± 5.6 | 50.42 ± 5.6 | 50.80 ± 5.6 | 83.26 ± 11 | 51.74 ± 6.2 |
| Silverbox, 1-step | 37.13 ± 0.050 | 20.73 ± 0.00082 | 41.45 ± 0.027 | 41.13 ± 0.028 | 143.6 ± 9.9 | 15.44 ± 0.085 |
| Silverbox, rollout | 88.50 ± 1.6 | 169.1 ± 0.089 | 113.1 ± 0.097 | 106.0 ± 0.73 | 600.1 ± 5.9 | 70.69 ± 0.66 |
| Tanks, 1-step | 24.64 ± 0.14 | 30.70 ± 4.9 | 24.89 ± 0.065 | 63.65 ± 0.64 | 58.94 ± 1.3 | 84.02 ± 0.44 |
| Tanks, rollout | 229.5 ± 4.6 | 487.4 ± 4.6 | 270.7 ± 2.0 | 447.1 ± 16 | 1663 ± 500 | 852.6 ± 4.9 |
| Average rank ↓ | 1.59 | 2.88 | 2.94 | 3.47 | 5.76 | 4.35 |
| Number of task wins ↑ | 11 | 2 | 1 | 1 | 0 | 2 |
I thank the authors for the detailed response and for conducting the additional five-seed evaluations and pruning ablations.
Acknowledgment of W2 & W3: The newly provided matched-condition ablations (Rebuttal Table 2), edge-deletion analyses, and full five-seed mean ± std statistics across all 17 tasks (Rebuttal Table 3) satisfactorily address my previous concerns regarding experimental reporting and the empirical behavior of the pruning heuristic. Accordingly, I am happy to acknowledge the improved empirical soundness of the paper.
Remaining Core Concern on W1 (Algorithmic Novelty & Necessity of the KAN Framing): Despite the stronger empirical validation, my fundamental concern regarding the methodological contribution remains unresolved. By restricting the model to a single layer with fixed, uniform spline bases, the architecture removes both the compositional depth and the adaptive basis learning that distinguish KANs from classical function approximators. Mathematically, the resulting formulation reduces to standard ridge regression over a fixed spline basis (i.e., a classical generalized additive model) paired with iterative backward elimination. In the regime, it further collapses into standard sequentially thresholded linear ridge regression, where classic STLSQ remains slightly more accurate and faster (Appendix Table 5).
While decoupling joint ridge estimation from output-norm pruning is a clean engineering pipeline, framing fixed-basis ridge regression as a "backprop-free KAN" feels largely like a repackaging of well-established statistical regression techniques rather than a substantive algorithmic advance for equation discovery.
Conclusion: While I appreciate the authors' thorough empirical follow-up, I believe the algorithmic novelty falls short of the bar for acceptance at ICLR, and therefore I maintain my rating of 2: Weak rejection.
W1 / Q1: Additional mechanisms and their practical benefits
LoKAN's additional mechanisms address how to learn functional effects and select sparse structures simultaneously under a fixed representation. Ridge regression on a fixed support provides joint coefficient estimation. Building on this, LoKAN adds contribution scoring at the functional edge level, iterative pruning, and final refitting to determine a separate edge support for each output.
Compared with sparse regression over a candidate library, when K=2, LoKAN adjusts the ridge scale according to the active design and combines this with threshold-based selection to recover explicit terms and coefficients. When K>2, each candidate edge contains multiple spline coefficients, and the unit of selection becomes an entire nonlinear effect function. Compared with Spline Group Lasso, LoKAN separates regularized coefficient estimation from explicit structure selection: the ridge parameter controls estimation regularization, the contribution threshold determines edge retention, and estimation in each round uses a direct joint solve.
We evaluate the practical benefits of this design through both search cost and prediction accuracy. In the additional Duffing and Pendulum experiments, BP-KAN and LoKAN use matched forward basis functions, allowing comparison of the two learning procedures under matched representations. HKAN and MatrixKAN provide architectural comparisons.
Table 1. Search-stage runtime and test accuracy across five random seeds.
Values are medians across five seeds, with interquartile ranges (IQRs) in parentheses. Search time excludes shared preparation and final retraining. BP-KAN and LoKAN use matched forward basis functions; HKAN and MatrixKAN are independent implementations in the project used as architectural comparisons. Pendulum uses a common nine-column candidate library.
On both tasks, LoKAN achieves the lowest median search time and test NRMSE among the four methods, demonstrating the cost and accuracy benefits of this procedure. Fixed-library recovery with K=2 is separately evaluated against STLSQ and SR3 as direct comparisons. When K>2, scalar weighting and functional edge learning use different representation spaces, and computational comparisons are distinguished accordingly.