- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies early exit for transformer classifiers with a guarantee that the early prediction stays close to the full-depth prediction. It treats the layer-wise softmax outputs of a shared classifier as a stochastic process, adds a regularizer that penalizes the batch-mean change of these outputs across layers, and exits when the relative quadratic variation of the outputs falls below a threshold. A time-uniform deviation bound via Ville's inequality (Proposition 3) is claimed for the regularized model, together with a non-martingality result (Proposition 1) and an expressivity result (Proposition 2). Validation covers six GLUE tasks with BERT-base and BERT-large against selected early exit baselines, Pareto sweeps, an empirical check of the bound, and an extension to TinyLlama.
Critical strengths
- Connecting early exit to martingale theory and anytime-valid inference is an interesting direction.
- The method is simple and adds no parameters.
Critical weaknesses
- The theory is not established
Proposition 3 has only an invalid proof sketch whose premise the regularizer does not satisfy, and Propositions 1 and 2 are not proved.
- The experiments do not support the motivation of the paper
The described stopping rule collapses in the paper's own appendix, and the baselines are out-dated and not run as in their original papers.
Questions for authors
- Where is the full proof of Proposition 3? Line 269 and the Reproducibility Statement promise one in the appendix, but Appendix A contains only results. The sketch does not hold: contains the terminal , so it is not adapted to ; , so fails; the supermartingale property is only asserted; and the optimized depends on the random , while Ville's inequality needs fixed in advance. The assumption always holds with , and never enters the bound.
- Why is the regularized sequence a martingale? Eq. 3 only drives the unconditional batch mean of the increments to zero, which Remark 1 calls "precisely the martingale condition", but a martingale needs a zero mean conditional on . With exactly, two mirrored groups of inputs that reach one vertex by layer 7 and flip to the other by layer 24 violate Eq. 9 for every input at . Moreover, is the same vector for every input, so under the paper's probability space a martingale forces a constant classifier.
- How are Propositions 1 and 2 proved? The argument for Proposition 1 asserts the drift without derivation. The proof of Proposition 2 uses a path that it admits is not a martingale, and as stated the proposition is false: with , the batch mean of must equal the constant on every batch, which an accurate classifier cannot satisfy on both an all-positive and an all-negative batch.
- Is the bound informative where it is checked? By Cauchy-Schwarz, Eq. 9 holds for any sequence whenever , that is, for every exit on BERT-base and every exit at layer 15 or later on BERT-large. It also needs , which is unknown at exit time. Figure 3 and Table 13 check KL rather than the squared L2 of Proposition 3, and the check fails at on BERT-large CoLA (14.8%).
- Proposition 1 concerns a model trained with cross-entropy, but Table 1 uses pretrained BERT-base. Which classifier produces the beliefs there, and why does Table 1 report 12/12 significant layers when Table 14 marks three layers as non-significant?
- Table 19 reports standard QV at with MCC .461 on BERT-large SST-2, while Table 9 reports .870 for QV at the same . Which numbers reflect the stopping rule of Algorithm 1, and which rule produces Tables 2 and 3?
- CALM [2] calibrates its threshold to guarantee consistency with the full model, but here it exits at the same layer for every threshold, at about on SST-2 (Appendix A.12). How was CALM implemented and tuned, and how were the thresholds of DeeBERT, PABEE, and Entropy chosen?
- The paper states that existing early exit methods offer no guarantee (lines 12 to 13), but CATs [1], CALM [2], and [3] give guarantees with respect to the full model, and [4] and [9] already use martingale arguments across exits or layers. How does the method compare with [1] to [4]?
- The newest compared baseline is from 2022. How does QV Exit compare with ConsistentEE, LayerSkip, RAEE, and [5] to [8], which report 1.5 to 2.2 times speedup on GLUE, while QV Exit saves at most 9% of FLOPs?
- What is the agreement between the exit and full-depth predictions of the same model? This is the property the paper promises, but only MCC against labels is reported.
- In Tables 9 and 10, entropy stopping reaches higher MCC than QV at matched FLOPs (for example .814 vs .732 at about 78% on QNLI), contrary to lines 363 to 365. What does QV stopping add over entropy, and what does it give without ?
- On TinyLlama, SST-2 drops from .874 to .492 at 80% FLOPs (Table 4). How does this fit "degrades gracefully", and why is no baseline run there?
- Tables 2 and 3 differ from Tables 5 and 6 for the same five seeds, and the ΔMCC column of Table 15 does not equal Full minus Exit. Which numbers are correct?
[1] Tal Schuster, Adam Fisch, Tommi Jaakkola, and Regina Barzilay. Consistent accelerated inference via confident adaptive transformers. In EMNLP, 2021.
[2] Tal Schuster, Adam Fisch, Jai Gupta, et al. Confident adaptive language modeling. In NeurIPS, 2022.
[3] Metod Jazbec, Alexander Timans, Tin Hadži Veljković, Kaspar Sakmann, Dan Zhang, Christian A. Naesseth, and Eric Nalisnick. Fast yet safe: Early-exiting with risk control. In NeurIPS, 2024.
[4] Metod Jazbec, Patrick Forré, Stephan Mandt, Dan Zhang, and Eric Nalisnick. Early-exit neural networks with nested prediction sets. In UAI, 2024.
[5] D. J. Bajpai and M. K. Hanawal. BEEM: Boosting performance of early exit DNNs using multi-exit classifiers as experts. In ICLR, 2025.
[6] D. J. Bajpai and M. K. Hanawal. Beyond greedy exits: Improved early exit decisions for risk control and reliability. In NeurIPS, 2025.
[7] J. He, Q. Zhang, D. Miao, K. Yi, S. Hao, H. Zhang, and Z. Wei. Improving prediction certainty estimation for reliable early exiting via null space projection. In IJCAI, 2025.
[8] J. He, Q. Zhang, H. Zhang, X. Huang, U. Naseem, and D. Miao. COSEE: Consistency-oriented signal-based early exiting via calibrated sample weighting mechanism. In AAAI, 2025.
[9] Michele Caprio and Sayan Mukherjee. Concentration inequalities and optimal number of layers for stochastic deep neural networks. IEEE Access, 11:38458-38470, 2023.