- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes SPAWN (Self-Partitioning Adaptive Widening Network), an adaptive neural network framework that dynamically adjusts its architecture during training. SPAWN organizes affine experts into a soft-gated binary tree, determines when to expand based on training loss plateaus, selects experts for splitting using within-region loss heterogeneity, and employs output-preserving inheritance to maintain the model's predictions during expansion. Experiments across multiple classification and regression benchmarks demonstrate that SPAWN achieves competitive predictive performance with substantially fewer parameters.
Critical strengths
-
SPAWN combines dynamic network growth with input-space partitioning, adaptively allocating model capacity through data-driven expansion criteria. Output-preserving inheritance allows optimization to continue smoothly after structural changes. The overall framework is well motivated and has practical potential.
-
The paper evaluates SPAWN on multiple synthetic and real-world datasets, comparing it with MLPs, XGBoost, and existing network-growing methods. Results across multiple random seeds demonstrate the stability of the approach and its promising parameter efficiency.
Critical weaknesses
-
The paper proposes dynamically partitioning the input space to adaptively grow model capacity. However, prior studies, such as Adaptive Neural Trees (ICML 2019)[1], have also explored dynamically expanding tree structures during training. Although SPAWN introduces distinctive expansion criteria and parameter inheritance mechanisms, the paper lacks direct experimental comparisons with these approaches, making it difficult to fully assess its advantages over existing adaptive tree-based models.
-
The evaluation of practical computational efficiency remains limited.The paper primarily demonstrates the efficiency of SPAWN through parameter counts and predictive performance. However, the method requires multiple expansion and optimization cycles during training, potentially introducing additional computational overhead. A more comprehensive comparison of end-to-end training time and actual inference efficiency would help substantiate its overall computational efficiency advantages.
[1] Tanno, R., Arulkumaran, K., Alexander, D., Criminisi, A., & Nori, A. (2019). Adaptive Neural Trees. Proceedings of the 36th International Conference on Machine Learning (ICML), PMLR 97, 6166–6175.
Questions for authors
-
Can the authors further clarify the key advantages of SPAWN over existing methods such as Adaptive Neural Trees and provide more direct experimental comparisons?
-
Given that SPAWN involves multiple expansion and optimization cycles, can the authors provide a more detailed analysis of its end-to-end training cost and inference efficiency?
Thank you again for your review. As the discussion period ends soon, we would like to ask whether our response has addressed your concerns. If any point is still unclear, please let us know which one, and we will clarify it. If your concerns have been resolved, we would be grateful if you could consider raising your score.