- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies attacks that induce expensive reasoning on requests that the model answers correctly using a short response against adaptive reasoning models. A shared 10-token suffix is constructed for each model, task, and template setting. Its main contribution is a request-level audit linking reasoning modes, answer correctness, and generation cost. Evaluation covers five settings across 1.5B–15B models on GSM8K, MATH-500, and MMLU-Pro. The results demonstrate increased computation on answers that remain correct, with forced-route comparisons attributing most excess generation to mode switching.
Critical strengths
-
This paper links executed reasoning modes, correctness, and cost for the same requests and provides a compelling justification for evaluating adaptive reasoning beyond accuracy and routing success alone.
-
This paper provides mechanism validation: forced-route experiments attribute most excess computation to reasoning-mode changes, while inspecting generated trajectories confirms actual route execution rather than merely increased reasoning-token scores.
Critical weaknesses
-
The attack requires white-box access and direct token-ID insertion. Its effectiveness through realistic indirect injection remains untested.
-
The attack applies established adversarial suffix optimization to reasoning-mode selection. The paper offers less evidence of a substantial advance in attack methodology.
Questions for authors
-
Do the optimized suffixes remain effective when decoded to text and retokenized through a standard chat interface, rather than inserted directly as token IDs? Can you conduct relevant experiments?
-
Given the modest clean-request p99 increase and only 100 requests per run across three runs, do you have larger-scale or longer-duration results establishing whether this effect is statistically reliable?
-
The MATH-500 route-head evaluation includes 57 prompts used in training. Can you report results on a fully training-disjoint test set to establish whether the conclusions still hold?