- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper investigates the sensitivity of pretrained Llama-3.2-1B and 3B to training-free layer skipping, separating the choice of which layers to skip from the choice of which token positions to skip them at. Using a common evaluation harness, it compares prefix bypass, trailing early exit, interior-block skipping, and uniformly spaced layer removal, alongside random selection, confidence and residual-norm gates, and a cost-based oracle heuristic. Evaluation primarily uses teacher-forced NLL on WikiText-2 and Alpaca-format prompts. The authors report that sensitivity varies substantially across depth, whereas the tested token-adaptive policies offer little consistent benefit over random selection. They attribute this behavior to interactions among skipped layers and weak correlations between gate signals and skip costs. Separate decoding experiments measure speedups from static layer omission.
Critical strengths
The problem is relevant to efficient inference, and the layer-wise sensitivity measurements provide some useful observations.
Critical weaknesses
-
The main conclusion is not adequately supported. The oracle is defined inconsistently across sections and is explicitly acknowledged to be a heuristic rather than an upper bound. Its failure to outperform random selection therefore does not establish that token adaptivity has negligible headroom. Sub-additivity across layers alone does not justify this inference.
-
The presentation is poorly balanced, and the empirical scope is too limited for the conclusions. The paper devotes substantial space to related work and repeatedly restates its conclusions, while key experimental definitions and table labels remain inconsistent. Evaluation covers only two small models from the same family, three seeds, and teacher-forced NLL rather than downstream generation performance. Although many of these limitations are acknowledged, the broad conclusions about training-free adaptive skipping remain insufficiently qualified.
Questions for authors
What exact intervention and ranking score produced the oracle results? What evidence supports the claimed lack of adaptive headroom independently of this heuristic’s failure?
Thank you for the author's response. In my view, this work does not meet the acceptance criteria for ICLR; I stand by my original assessment.