- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2(+1)
- Soundness
- 2
Read review
Paper summary
C-KAN grows Kolmogorov–Arnold networks by neuroevolution, which is a replacement of training a fixed template and pruning it. Individuals are typed sparse graphs with spline edges. A NEAT-style loop applies five Adam steps per generation and writes the adapted weights back (Lamarckian inheritance). The authors name five problems that arise when NEAT meets spline edges and add a fix for each: growth at the identity function, gradual fan-in normalization with geometric-mean multiplicative nodes (Proposition 1 bounds activation scale), crossover inside the fitter parent’s template, safe mutation with the spline amplitude divided out, and a bounded behavioral distance for speciation. For symbolic regression, five champions spread across species are pruned by replacing edges with their mean output. They are then converted per edge (“vanilla”) or by greedy in-context symbolic regression (GSR), and the simplest valid equation within a fidelity band is returned.
Experiments use SRBench and 30 clean SRSD equations. Baselines are PyKAN with one or two inner layers (vanilla extraction only, sparsity weight 0), KAN-SR, PySR, Operon and TPSR. C-KAN-GSR returns 12 of 30 equations with test R^2 above 0.999, where neither PyKAN template returns any, and its black-box AUC is 0.332 against 0.249 and 0.149. No KAN pipeline recovers an exact skeleton, while PySR recovers 43.3% and KAN-SR 15.8%. A constant refit added after seeing the results lifts PyKAN-1’s high-accuracy rate to 54%, above C-KAN-GSR’s 45%, though C-KAN’s expressions stay smaller (median size ratio 2.0 against 7.4). A stage breakdown places 17 of 30 losses in the search, 3 in conversion and 7 in export. Ablations on 12 and 6 equations show zero recovery in every variant.
Critical strengths
-
I think, the main strength is the stage-level diagnosis that the paper reports even where it cuts against the headline. Tables 2 and 21 trace each equation to the first stage where it is lost. The finding that the dense template fits more equations at first (22 against 9) but loses them later, while the grown graph keeps what it finds, points to where KAN pipelines break. That is useful beyond this method, even though the comparison is not yet like for like (see the first weakness below).
-
The second strength is the good scoping together with a concrete account of why plain NEAT fails on spline edges. The authors name five specific mismatches and mark each mechanism as new, adapted or borrowed (Table 4). The amplitude correction to Safe Mutation addresses a real problem for low-amplitude splines. The paper labels its comparisons as exploratory and its clean-up analyses as post hoc, and it reports the PyKAN refit divergence and the failure on the 50-input dataset. I did not check the proofs.
Critical weaknesses
-
The first weakness is that the central claim, that growing beats pruning, rests on a weak, partly diverged and unevenly instrumented baseline, and the paper’s own refit result undercuts it. PyKAN ran with sparsity weight 0, although that penalty is part of the standard recipe before pruning. Only vanilla extraction was completed, so C-KAN-GSR is compared with a pipeline that never received GSR. PyKAN’s post-pruning refit diverged in 85 of 150 Track B runs, and PyKAN-2’s median expression has 1,178 nodes, which suggests its pruning did not work as intended. The stage comparison in Table 2 is also not like for like, because C-KAN prunes and refits in one step while PyKAN’s symbolic model is not logged before its refit. The roster was revised after results were seen, so every test is exploratory. After Holm correction, no Track A paired comparison passes 0.05, and the NED comparison with PyKAN is not detectable either. With constants refit, PyKAN-1 beats C-KAN on accuracy (54% against 45%), and both recover 5.8% of skeletons. What remains is smaller expressions, but C-KAN’s own pruning and selection rules also favour small size, so the size advantage is not tied to growth. GSR and vanilla outputs also share one evolution, with GSR run first, so the GSR gain is not isolated.
-
The method misses its stated task, and the ablations cannot link results to the proposed mechanisms. No exact skeleton is recovered, and 17 of 30 equations have no accurate prepared network, so the main bottleneck is the very search the paper proposes. The mechanism ablations use 12 and 6 equations where recovery is zero everywhere, so only validity and runtime can differ, and even those comparisons are descriptive because hardware and budgets differ. No control separates growth from minimal networks, which is the paper’s central design choice, from evolutionary search over fixed topologies at equal compute. Black-box AUC and expression size also stay behind PySR and Operon, and 10 of C-KAN’s 16 invalid Track A outcomes come from one 50-input dataset.
Questions for authors
1 - What are the wall-clock and compute budgets for each method, including PyKAN’s training, pruning and refit, on the same hardware? C-KAN uses 100 generations of 150 individuals within one hour on one CPU thread, so how do PyKAN, PySR and KAN-SR compare on that budget?
2 - How does PyKAN do with a tuned sparsity weight, with its refit divergence fixed, and with GSR extraction applied to its networks, and does the vanilla-versus-vanilla gap survive that?
3 - What happens if C-KAN’s own pruning (mean replacement with the tolerance rule), refit and selection are applied to the trained PyKAN-1 network, and can PyKAN’s symbolic model be logged before its refit so that the stage columns in Table 2 match?
4- Can you add an equal-budget control that runs the same evolutionary loop over fixed-size topologies, without growth from minimal networks? Without it, I cannot attribute the result to constructive growth.
5 - Please report per-seed high-accuracy and recovery rates with intervals, not only best-of-five by validation loss.
5 comments
Thank you for the new experiments. Tuned PyKAN with GSR beats C-KAN on Track B accuracy (73/120 against 54/120), so it looks like the result is a trade-off with expression size. The pruning and fixed-topology controls are unfinished, so growth is still not separated from C-KAN’s own pruning and selection rules. I could not check the expanded benchmark in this period. I am keeping my score for now. The paper would be stronger as a resubmission with the finished controls and claims stated as a trade-off.
Extra ablation runs completed
Thank you for the clarification. We agree that the results should be presented as trade-offs, rather than general superiority over tuned PyKAN.
The fixed-topology, pruning-transfer and longer-fitting controls are now complete on the same 12-equation, five-seed panel. The following equation counts include common elimination/rounding/refitting and retain failures in the denominator:
- Growth: Keeping C-KAN’s minimal graph fixed, with the same recovery procedure, yields 5/60 highly accurate GSR equations versus 11/60 with growth. However, the fixed minimal graph is more often fully evaluable (54/60 versus 36/60). The fixed-[10] follow-up now completes all 100 generations under a six-hour allowance, yielding 10/60 highly accurate GSR equations.
- Pruning transfer: Normally trained tuned PyKAN with C-KAN-style pruning yields 9/60 highly accurate GSR equations versus 22/60 with native pruning. Thus transferring the pruner does not reproduce C-KAN’s delivery behavior.
- Additional fitting: Post-pruning L-BFGS increases numerical high accuracy from 33/60 to 37/60. Cleaned symbolic accuracy improves for Vanilla (15→19/60), while GSR remains at 22/60. We are incorporating these completed controls and their limitations into the revision. They sharpen the fitting–delivery trade-off, but do not establish that growth alone explains every difference.
Response to Reviewer sCAy
Thank you for the detailed experimental and presentation concerns. We have expanded the collection to 24 SRBench datasets × 30 seeds and all 120 SRSD equations × 10 seeds, reporting unresolved execution cases explicitly. We are strengthening the comparisons and rewriting the methodology/results to clarify stage definitions, evidence and limitations.
1. Budgets and runtime reporting. Each main pipeline has a 3,600-second training-to-equation allowance, plus 120 seconds for evaluation. C-KAN uses one CPU thread; PyKAN uses CUDA with one host thread.
The experiments were collected across heterogeneous machines with concurrent workers. We will report hardware configurations, recorded stage timings, hardware-specific medians/IQRs and timeout rates, with concurrency disclosed where known. Published SRBench records will be distinguished from locally measured runs.
These measurements describe computational costs under our collection conditions; they do not establish equal compute or hardware-independent speedups. We will make this limitation explicit and avoid pooled cross-machine runtime rankings. Would this budget/resource disclosure and machine-specific timing analysis address your concern? Clarification on whether your principal concern is resource allocation or comparative execution speed would help us focus the reporting.
2. Stronger PyKAN and matched extraction. We added staged sparsity-regularized training, pruning, grid refinement and refitting, with a two-depth regularization pilot (λ = 0, .0001, .001, .01; selected .01), and both vanilla/GSR extraction. The original 85/150 divergence count concerns the unregularized recipe; failures of the new recipe will be reported separately, distinguishing numerical outcomes from execution failures.
The vanilla-versus-vanilla advantage survives on Track A but reverses on Track B: original validation-selected high-accuracy counts are 37/120 for C-KAN versus 54/120 for tuned PyKAN-1. For GSR, Track A’s 12 complete first-principles datasets favor C-KAN throughout: failure-aware median dataset R² .802 versus .238 (exploratory Holm-adjusted p = .00244).
After common Track B backward elimination, rounding and constant refitting, C-KAN-GSR versus PyKAN-1-GSR gives:
This is a prediction–structure trade-off, not general superiority. Original exact recovery remains zero for both KAN pipelines. Cleanup adds computation and remains post hoc; fixing the original selected seed narrows recovery to 8 versus 7. Two PyKAN-GSR execution-failure cells remain under verification, making its figures provisional.
3. C-KAN-style pruning and matched stages. The requested control is implemented: one extended-training PyKAN checkpoint feeds native versus mean-preserving C-KAN-style pruning, each followed by vanilla/GSR extraction. Dataset-specific training targets derive from recorded C-KAN evolution times; its 5,400-second per-output cap is deliberately more generous, not equal compute.
New logging records pruning/refitting stages and symbolic conversion before final refitting. We distinguish the numerical champion from the source of the reported equation. This control is unfinished, so no pruning conclusion is claimed. We will also apply common post-hoc cleanup to Track A and the ablations.
4. Fixed-topology evolution and attribution. A ten-hidden-node control retains the evolutionary/adaptation/extraction settings with node/link growth disabled and the main time cap. Results are incomplete, so compactness remains a pipeline-level finding rather than proof of a causal growth benefit.
Further mechanism/candidate-count ablations are completed. The full SM/behavior configuration yields 28/60 fully evaluable GSR outputs versus 19–24/60 for removals, but recovery remains zero. Hardware/timing qualifications and the 50-input Track A scalability failure remain explicit.
5. Individual-seed results. Original Track B high-accuracy rates across individual runs, with 95% intervals, are:
Intervals resample whole equations, keeping their seeds together. These complement validation-selected results. Revised-roster tests and cleanup remain explicitly exploratory/post hoc.