- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper propose DGKAN, which dynamically reallocates fixed grid point budgets to high complexity regions through a layerwise adaptation scheme that ensures stable relocation and function preservation.
Critical strengths
By formulating grid adaptation as a budget-constrained allocation problem, DGKAN dynamically reallocates fixed grid point budgets to high complexity regions, improving representational efficiency while preserving network composition.
Critical weaknesses
Contribution 2 represents the main methodological innovation proposed by the authors; however, several key design elements within it have not been individually validated.
There is a gap in the experimental design regarding "layerwise adaptation": the authors do not justify why hidden layers must be guided by the empirical density, while the output layer must be guided by the prediction residual.
The authors constrain the magnitude of each individual movement of grid points using a "relocation rate" and a "maximum displacement". This is a reasonable design choice for numerical stabilization. However, the paper does not discuss how training stability varies under different displacement limits.
Questions for authors
Could the authors explain why empirical feature distributions are used to guide hidden-layer adaptation while prediction residuals are used for output-layer adaptation, and provide evidence that this design is more effective than alternative configurations?
Rationale and Stability Analysis of Layerwise Dynamic Grid Adaptation
1. Why use density in hidden layers and residuals at the output?
Response. We thank the reviewer for raising this point. Our asymmetric layerwise adaptation strategy is motivated by the different supervision signals available at the output and hidden layers.
At the output layer, the input representation is directly connected to the final prediction. Therefore, we can compute interval-wise prediction residuals and use them to construct a local error monitor for grid distribution.
Backpropagating the final prediction error to a hidden layer yields an error signal, not the true approximation residual of the hidden univariate function. It depends on downstream parameters, nonlinearities, and channel-wise scaling, and may therefore mix credit assignment with the local resolution requirement of the hidden basis.
We consequently use the empirical density of hidden features as an observable and lower-variance proxy for where representation resolution is needed. This density is indirectly shaped by the final prediction loss because the hidden representations are learned under that loss. Thus, the output-layer residual provides a direct task-aware signal, while hidden-layer density provides a stable representation-resolution signal. Density alone is insufficient to characterize prediction error, which is why we retain residual-driven adaptation at the output layer.
Due to the limited time available, we conducted a preliminary experiment on an NVIDIA RTX 3090 using a subset of the data (fpacket) and the original experimental settings. For each configuration, we used five different random seeds and repeated the experiment five times. The table reports the mean MSE and standard deviation across these five runs.
2. Relocation rate, maximum displacement, and stability
Response. We thank the reviewer for pointing out this omission. We use the relocation rate, denoted by λ, and the maximum per-update displacement, denoted by δmax, to limit the magnitude of each grid update. These two parameters affect the trajectory of grid movement and should be distinguished from the final predictive performance.
We conducted controlled layerwise sensitivity experiments. With λ=0.2 fixed, changing δmax from 0.25 to 2.0 resulted in relatively stable selected frozen-stage test errors across the 20 paired configurations, without an evident systematic trend. This suggests that, under the current experimental settings, the maximum displacement is not the primary factor determining the selected final error. With δmax=0.25, changing λ from 0.2 to 1.0 yielded 9 paired improvements and 11 degradations, and increased the mean error in the local-uniform G=5 condition. Thus, our current evidence does not support a universal optimum for either parameter.
We recorded the minimum grid spacing, number of grid updates, cumulative grid movement, immediate change in training-anchor loss, and checkpoint-level performance for every run. For the proposed density/residual guidance, we observed 720 grid-update events in total. Only 174 events (24.2%) immediately reduced the training-anchor error, whereas 546 events (75.8%) temporarily increased it. Nevertheless, after the subsequent frozen-grid continuation, all runs achieved lower final test MSE than their corresponding pre-update states. This pattern supports our interpretation that a grid relocation should not be judged solely by its immediate loss change. Because coefficient projection is approximate, an individual relocation can temporarily perturb the local representation, while subsequent retraining can compensate for this perturbation and yield a better final solution.
In the revised manuscript, we will add the complete sensitivity results and grid-movement trajectories. Our results support a conditional-effect interpretation: the relocation constraints are useful for controlling representation perturbations, but their predictive impact depends on the data distribution, grid budget, and update schedule.