- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes ASTra, a training-free, inference-time method to mitigate overthinking in LLMs. ASTra dynamically identifies attention sinks, tracks their cumulative attention weights, and triggers early termination when a threshold is exceeded, aiming to reduce redundant chain-of-thought (CoT) tokens without sacrificing answer quality. The experimental validation shows the effectiveness of the proposed method.
Critical strengths
-
ASTra uses attention sink saturation as an internal signal to identify when continued reasoning may yield diminishing returns. This offers a conceptually distinct alternative to existing stopping strategies based on fixed token budgets or output-level signals, providing a potentially useful perspective on inference-time reasoning control.
-
Experiments across multiple models and diverse reasoning and knowledge benchmarks report significant token reductions while largely preserving task performance. The ablations and statistical analyses further support the effectiveness of the approach. Although monitoring introduces computational overhead, the substantial reduction in generated tokens suggests practical potential for improving inference efficiency without additional training.
Critical weaknesses
- ASTra assumes that attention sink saturation indicates diminishing returns from continued reasoning, but this relationship does not necessarily establish that the remaining reasoning tokens are redundant or can be safely removed. Without stronger evidence linking the proposed signal to reasoning progress across different tasks and models, the stopping criterion may terminate useful reasoning and limit the reliability of the approach.
Questions for authors
-
Could the authors provide evidence that attention sink saturation reliably indicates diminishing returns from continued reasoning, rather than merely correlating with generation length? In particular, how does ASTra perform when the stopping threshold is varied across tasks and models, and how often does it truncate reasoning before a correct answer is reached?
-
Can ASTra be adapted to non-softmax attention mechanisms, where attention weights may not sum to one and conventional attention sinks may not emerge?