- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes Linear Reusable Neural Bases (LRNB), a parameter-sharing framework where neuron weights are represented as linear combinations of a small set of shared basis functions. The authors argue that neuron-level reuse can improve parameter efficiency while maintaining model expressiveness, and evaluate the approach on synthetic regression and WikiText-2 language modeling tasks.
Critical strengths
- The core idea is simple, intuitive, and easy to implement within existing architectures.
- LRNB provides substantial parameter reduction while preserving reasonable performance on the presented benchmarks.
- The paper offers an interesting geometric interpretation of neural networks through reusable bases and vector-field representations.
Critical weaknesses
- The proposed approach appears closely related to existing cross-layer parameter-sharing methods, particularly Basis Sharing [1], which represents layer weights using shared basis vectors and layer-specific coefficients. The paper does not sufficiently establish the methodological differences or demonstrate a clear advantage over this prior work.
- The evaluation lacks comparisons with relevant parameter-sharing and model-compression methods, such as [1,2]. Without comparisons against these methods under comparable parameter budgets and evaluation settings, it is difficult to assess the novelty and practical competitiveness of the proposed approach.
- Key design choices, particularly the gradient rescaling strategy for shared parameters and the parameter-sharing configurations, are not adequately validated. Ablations against alternative optimization strategies are needed to establish the contribution of each component.
- Although LRNB reduces the parameter count from 34.40M to 19.72M, its reported training time per epoch is approximately 189.5 seconds, compared with 185.4 seconds for the classical Transformer. The paper should provide a more thorough analysis of training and inference latency, memory consumption, and throughput.
Reference
- Wang, et al. "Basis Sharing: Cross-Layer Parameter Sharing for Large Language Model Compression." ICLR (2025).
- Zhang, Boyang, Daning Cheng, Yunquan Zhang, and Fangming Liu. "Rethinking parameter sharing as graph coloring for structured compression." ICML (2026).
Questions for authors
Check weakness
Participate in the discussion period
Dear reviewers,
The authors have replied to your comments and the discussion is close to the end. We encourage you to participate in the discussion to see whether your questions and concerns have been answered by the authors. Thank you!
Best, Your AC