- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper investigates under what conditions residual networks maintain finite forward and backward costs within a given depth, without requiring global Lipschitz continuity. The author decomposes the growth behavior of each residual block into two exponents for discussion.
Critical strengths
1.The paper explains that q=1 is a sharp threshold. 2.The paper establishes a set of primitive level exponential arithmetic, which enables researchers to symbolically derive the upper bounds (q, σ) of each residual block from the computational graph of the model before training, and locate potential unstable primitives based on this.
Critical weaknesses
1.The core theoretical prediction of the paper is the authentication depth, but the experiment has never directly calculated or reported specific values, nor has it demonstrated the correspondence between the authentication depth predicted by the theory and the actual observed explosion depth. 2.The boundary that the theory relies on in the most relevant scenario (attention) is acknowledged by the author as potentially loose, weakening the credibility of quantitative predictions.
Questions for authors
1.Why does the paper choose "layer-by-layer uniform boundary" instead of "trajectory adaptive boundary", and what impact does this choice have on the universality of the conclusion? 2.Compared with spectral normalization, ReZero/LayerScale, and stability analysis of neural ODE, in what sense does the "authentication" in this article provide something they do not have?
Reminder to reviwers to read author's responses
Dear Reviewers,
Thank you for your efforts in providing thorough reviews.
As the discussion period is underway, I would like to kindly remind you to read the authors' responses to your reviews and provide follow-up comments where appropriate or leave a comment. In particular, please consider whether the responses address your questions and concerns, and update your assessment if the clarifications or additional information affect your initial evaluation.
Your engagement in this discussion is important for ensuring a fair and thorough review process.
Thank you again for your time and contributions!
Best regards, Area Chair