- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This is a good paper that brings together several findings in the field and empirically demonstrates their effectiveness in real applications. The paper shows that applying pre-determined yet dynamic budget control to sparse attention can improve inference efficiency while maintaining quality. The experiments provide good results. Although the overall experimental design does not analyze every step or component in an extremely complete and fine-grained manner, it is already sufficient to support the paper’s claims.
Critical strengths
As a paper that focuses more on methodological and empirical progress, it demonstrates solid improvements and shows a certain degree of generality. I believe some readers will find ideas in this paper that can be applied to their own work. Therefore, I lean toward acceptance.
Critical weaknesses
The main incremental contribution of this paper lies in the specific budget prediction and control scheme, together with the corresponding empirical validation, rather than in a new principle for computation allocation. I think the paper makes limited conceptual progress and does not bring enough new knowledge. This limits me from giving a rating of 4.
There are also some presentation issues. For example, the contrast in Figure 4(b) and Figure 5 seems somewhat too high/too low, and Table 3 itself is placed quite far from where it is actually discussed. However, I can see that the authors have already put effort into the design and layout, so this is acceptable if there is no clearly better way to present them. In addition, some terms in the paper are a little confusing. For example, “panel head” (854–855) and “panel distillation” (858) somewhat increase the cost of understanding. Also, “posterior pruning” (013) may be interpreted by readers with a theoretical background as something related to probability, so this could be clarified. This point will affect my Presentation score. Since these issues are easy to fix, I am giving a Presentation score of 3 in advance.
Questions for authors
I found that some works I know that are related to this paper in terms of method or idea were not included in the comparison. This is understandable, because under the mainstream ideas in this area, there can be many specific implementations, and it is difficult to compare against all of them. Therefore, I only hope that the authors can conceptually clarify the novelty of SABER relative to Fluxion [1], Elastic Attention [2], FBS [3], and ACM [4], as a supplement.
[1] Yao, F., Niu, Z., Li, X., Xiong, Y., Fang, J., & Wang, Q. (2026). An Efficient Hybrid Sparse Attention with CPU-GPU Parallelism for Long-Context Inference. arXiv preprint arXiv:2605.07719.
[2] Tang, Z., Qiu, Q., Yang, Y., Hong, Z., Xiang, H., Liu, K., Dang, Q., Li, J. & Zhang, M.. (2026). Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers. Proceedings of the 43rd International Conference on Machine Learning, in Proceedings of Machine Learning Research 306:119115-119142.
[3] Wang, T. (2026). Fbs: Modeling native parallel reading inside a transformer. In Findings of the Association for Computational Linguistics: ACL 2026 (pp. 4106-4137).
[4] Wójcik, B., Devoto, A., Pustelnik, K., Minervini, P., & Scardapane, S. (2025). Adaptive Computation Modules: Granular Conditional Computation for Efficient Inference. Proceedings of the AAAI Conference on Artificial Intelligence, 39(20), 21510–21518.
2 comments
Thank you for the authors’ response. My concerns were relatively limited to begin with, and I think the response has largely addressed them.
I believe there was a slight misunderstanding regarding what I meant by a “principle” of computation allocation. I was referring to a relatively high-level idea: how we understand the computation-allocation problem, how the overall design space is structured, or at least a broad family of methods for addressing it. From this perspective, I still view SABER primarily as a strong and useful methodological contribution. Its task-outcome-oriented budget prediction and online control are meaningful extensions for sparse attention, but they do not substantially change my broader understanding of how adaptive computation can be allocated based on model states or input-dependent signals.
This clarification does not negatively affect my overall assessment. I appreciate the authors’ detailed response, and I remain inclined to accept the paper.
In addition, I have read the other reviewers' comments and the authors' responses. Their concerns are somewhat different from mine, and I look forward to seeing more follow-up discussion.
We respectfully disagree: SABER introduces not only a budget prediction and control scheme, but also a new principle for computation allocation. Prior adaptive sparse-attention methods such as Twilight [1] and Double-P [2] determine budgets through top-p selection based on attention statistics, aiming to preserve a target amount of attention mass. SABER instead uses the model’s internal hidden states to predict whether a budget is sufficient to preserve the downstream task outcome and provides online budget control as decoding progresses.
This principle provides a deployment-friendly direction for inference optimization. Compared with training-free sparse-attention methods, SABER provides adaptive budget control while reusing existing backend kernels (as demonstrated in our main experiments and our response to Reviewer 2GZ1’s Q5), offering a practical deployment advantage on accelerators with restricted support for custom kernels, such as Intel NPUs. Compared with methods requiring training, such as Elastic Attention[3] and FBS[4], SABER uses separate supervised training of its predictors and controller, avoiding backpropagation through the LLM and reinforcement learning.
We therefore respectfully ask the reviewer to reconsider the assessment of limited conceptual progress.
Fluxion: Fluxion’s novelty lies in a CPU–GPU co-designed sparse-attention system. Its predictor identifies whether an attention head requires retrieval from CPU-resident historical KV and estimates the budget–granularity relationship for retrieval heads. SABER’s predictor instead predicts whether the model can produce a correct answer under a given attention budget.
Elastic Attention: Elastic Attention’s novelty lies in a per-head router that switches between full and sparse attention, trained via backpropagation through the LLM while keeping the backbone frozen. SABER instead trains a separate predictor to estimate budget sufficiency from hidden states.
FBS: FBS’s novelty lies in controlling computation depth through layer skipping, whereas SABER controls the sparse attention backend’s budget. FBS modifies the model architecture and trains it through continual pretraining followed by PPO; SABER instead constructs supervision from model-generated answers and trajectories to train separate predictors and a controller, without modifying the backbone or sparse-attention backend.
ACM: ACM’s novelty lies in replacing models’ original modules with distilled learners and using a learnable gate to adapt the number of learners executed per token. ACM pursues efficient inference through distillation, whereas SABER focuses on budget allocation for existing sparse-attention backends.
We thank the reviewer for these valuable suggestions. We will adjust the contrast in Figures 4(b) and 5, move Table 3 closer to its discussion, and clarify the terms “panel head,” “panel distillation,” and “posterior pruning” in the revised manuscript.
[1] Chaofan Lin et al. Twilight: Adaptive Attention Sparsity with Hierarchical Top-p Pruning. NeurIPS, 2025.
[2] Wentao Ni et al. Double-P: Hierarchical Top-P Sparse Attention for Long-Context LLMs. arXiv:2602.05191, 2026.
[3] Zecheng Tang et al. Elastic Attention: Test-time Adaptive Sparsity Ratios for Efficient Transformers. ICML, 2026.
[4] T. Wang. FBS: Modeling Native Parallel Reading inside a Transformer. Findings of ACL, 2026.