- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes TIDES (Test-time Inference Drift Exploitation via Scaling), a backdoor attack on reasoning models that uses reasoning length as an implicit trigger, requiring no input trigger. The poisoned model behaves normally under short reasoning budgets, but its accuracy drops sharply as reasoning length increases. The method consists of three steps: (1) extracting layer-wise “logic drift vectors” between correct and incorrect reasoning traces; (2) using a gate (DLT), conditioned on token position and entropy, to concentrate the intervention in the later stages of reasoning; and (3) embedding this behavior into ordinary model weights through on-policy distillation. Across three distilled reasoning models and five benchmarks, TIDES achieves an average RAS of 89.8%, a 26.8% relative improvement over the strongest baseline, DecepChain, while largely preserving Pass@1 under short reasoning budgets.
Critical strengths
1. Novel threat model perspective. The paper uses the amount of reasoning computation itself as the trigger, without requiring any input trigger. This highlights the need to evaluate the safety of reasoning models across different reasoning budgets, beyond conventional trigger detection.
2. Complete pipeline with a clear design. The three components: layer-wise drift vectors, depth gating, and on-policy distillation, each serve a clear purpose. The resulting model uses ordinary model weights, making the approach well suited to a supply-chain setting in which the attacker can only release weights.
3. Fairly comprehensive experiments. The evaluation covers three models, five benchmarks, and three random seeds, with standard deviations reported. Ablations examine extraction position, vector normalization, and injection strategy. The paper further evaluates robustness under different sampling parameters and parallel sampling, alongside detection at both the output and representation levels.
Critical weaknesses
1. Missing a core baseline, or missing an explanation of why the steering vector is necessary. Since a distillation step is required at the end anyway, why not directly construct data in which “short reasoning yields correct answers and long reasoning yields incorrect answers,” and then directly perform SFT or distillation?
2. Missing some other metrics. I would particularly like to see the ASR metric. I want to know how likely it is that examples answered correctly under short reasoning are successfully attacked after applying TIDES.
3. The gate design. The main input to the gate is ρ_t = t / T_max, which is a ratio relative to the maximum generation length rather than an absolute number of tokens. As a result, regardless of whether the budget is long or short, ρ_t approaches 1 in the later part of generation and the gate will be activated, which contradicts the goal of “remaining normal under short budgets.”
4. How many tokens exactly are the “short budget” and “long budget” in the paper? Under what budgets are the Pass@1 and RAS in Table 1 measured, respectively?
5. The choice of models is questionable. To my understanding, the DeepSeek-R1-Distill-Qwen-7B and R1-Distill-Llama-8B used by the authors are obtained by SFT on Qwen2.5 and Llama-3.1 with reasoning data generated by DeepSeek-R1, without RL; Phi-4-Reasoning is obtained by SFT on Phi-4 with reasoning demonstrations generated by o3-mini, and is also a distilled model. I am not sure whether such non-native models are more vulnerable to attack.
6. Discussion of generalization. The entire training of the vector is based on math. Although the authors also provide results on a code benchmark, I would like to see more discussion from this perspective. I want to know whether this attack on GSM8K can generalize to other benchmarks, and whether the vector learns to suppress all reasoning traces or selectively targets cases in the same domain as the training data.
Questions for authors
-
Since a distillation step is required at the end anyway, why not directly construct data in which short reasoning yields correct answers and long reasoning yields incorrect answers, and then perform SFT or distillation? Could the authors add this baseline, or explain why the steering vector is necessary?
-
Could the authors report the ASR, i.e., among the examples answered correctly under short reasoning, what proportion are successfully attacked after applying TIDES?
-
The gate uses ρ_t = t / T_max, which is a ratio relative to the maximum generation length rather than an absolute number of tokens. Under a short budget, ρ_t also approaches 1 in the later part of generation. Why is the gate not activated in this case, and how does this design ensure that the model remains normal under short budgets?
-
How many tokens exactly are the short budget and the long budget? Under what budgets are the Pass@1 and RAS in Table 1 measured, respectively?
-
All three models used in the experiments are distilled models obtained by SFT without RL. Are such non-native models more vulnerable to this attack? Is TIDES equally effective on natively RL-trained reasoning models?
-
The vector is trained entirely on GSM8K. Can the attack generalize to benchmarks in other domains? Does the vector learn to suppress all reasoning traces, or does it selectively target cases in the same domain as the training data?
4 comments
Response to Reviewer UJhH (2/2)
[W5 / Q5] Evaluation on RL-trained reasoning models
Comment: 5. The choice of models is questionable… // All three models used in the experiments…
Response
We thank the reviewer for this question. To test a natively RL-trained model, we evaluated TIDES on ORZ-7B, trained with RL directly from the Qwen2.5-7B base model without SFT or distillation [1]. We compared the released checkpoint with its TIDES student on all 500 MATH-500 problems at 512- and 4,096-token budgets (single run).
| Model | Pass@1 @512 (%) | Pass@1 @4096 (%) | Absolute drop @4096 (pp) | RAS @4096 (%) |
|---|---|---|---|---|
| Clean RL checkpoint (ORZ-7B) | 15.6 | 71.8 | — | — |
| TIDES student | 15.2 | 34.4 | 37.4 | 52.1 |
TIDES is comparably effective on this RL-trained model. It preserved 512-token accuracy and reduced 4,096-token Pass@1 from 71.8% to 34.4% (52.1% RAS across all problems). Under the same protocol, TIDES reduced Pass@1 for the distilled DeepSeek-R1-Distill-Qwen-7B from 68.0% to 33.2% (51.2% RAS). Both RAS values include problems solved within the first 1,000 tokens, whose responses TIDES is designed to preserve. For DeepSeek-R1-Distill-Qwen-7B, any attack that preserves these short responses has an all-problem RAS ceiling of 54.0%. Thus, the distilled model was no more vulnerable than the RL-trained model in this comparison.
[1] Hu et al. Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model. arXiv:2503.24290, 2025.
[W6 / Q6] Cross-domain generalization
Comment: 6. Discussion of generalization… // The vector is trained entirely on GSM8K…
Response
We thank the reviewer for this question. Constructed solely from GSM8K and OpenR1-Math, TIDES achieves 62.4–100.0% RAS across all 15 backbone–benchmark pairs in Table 1, without using any evaluation benchmark in its construction. On LiveCodeBench, RAS reaches 88.4% for DeepSeek-R1-Distill-Llama-8B and 99.8% for Phi-4-Reasoning, demonstrating transfer from mathematics to code. FROST [2] reports similar transfer in reasoning efficiency, with a mathematics-trained intervention generalizing to code and physics.
The attack neither suppresses all reasoning nor targets only the training domain. It remains length-selective: short-response Pass@1 is no more than 1.3 percentage points below the clean baseline across all 15 pairs. On MATH-500, 65.4% of outputs from the DeepSeek-R1-Distill-Qwen-7B TIDES student end with an incorrect final answer; only 1.4% are truncated.
GSM8K LDVs point from correct to fallacious activation centroids. Correct and incorrect reasoning trajectories separate in representation space, and GSM8K-trained correctness predictors perform above chance on MATH-500 and MMLU [3], consistent with a partly shared error component along this direction across domains. The gate uses position and entropy, without domain labels.
Domain-specific LDVs. TIDES also remains effective with LDVs extracted from code and physics data. We compared each resulting student with its clean checkpoint at fixed generation budgets. Pass@1 is shown as Clean → TIDES; RAS is calculated from unrounded counts.
| Held-out benchmark | Pass@1 at 1,024 tokens | Pass@1 at 4,096 tokens | RAS at 4,096 tokens |
|---|---|---|---|
| LiveCodeBench | 9.3% → 8.6% | 32.9% → 11.4% | 65.2% |
| UGPhysics (186 English Semiconductor Physics problems) | 12.4% → 11.3% | 31.7% → 16.7% | 47.5% |
At 1,024 tokens, Pass@1 differs from the clean checkpoint by at most 1.1 percentage points on both benchmarks. Here, RAS includes all problems at each fixed budget, including those solved within the first 1,000 tokens, which TIDES is designed to preserve. Table 1 reports RAS for long responses only. For the DeepSeek-R1-Distill-Qwen-7B student on MATH-500, preserving short responses limits RAS across all problems to 54.0%; TIDES reaches 51.2% (Pass@1 68.0% → 33.2%).
[2] Luo et al. FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning. ICLR, 2026. arXiv:2601.19001.
[3] Sun et al. LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals. ACL, 2026.
Thanks to the authors for the detailed response and the additional experiments. They address most of my concerns, and I will keep my score.
We thank the reviewer for the follow-up and the positive assessment. If any concern remains open, we would be glad to address it before the discussion closes.
Response to Reviewer UJhH (1/2)
[W1/Q1] Latent steering versus direct training
Response
We thank the reviewer for suggesting this baseline, which we added using the same 2,000 prompts as TIDES. We fine-tuned the student on correct GSM8K responses (<1,000 tokens) and incorrect responses (>2,000 tokens) to OpenR1-Math-220k prompts, matching TIDES in student initialization, trainable parameters, optimizer and update count. The clean model generated the incorrect responses under the deceptive prompts used for LDV extraction, making this baseline a form of sequence-level distillation from the deceptively prompted clean model; we retained only normally terminated responses with extractable incorrect answers. Both models are trained for this controlled comparison; the main TIDES results remain those in Table 1.
We evaluated DeepSeek-R1-Distill-Qwen-7B on all 500 MATH-500 problems with a 4,096-token cap. Clean reference generations defined short-reference (≤1,000 generated tokens) and long-reference (>1,000) groups.
Table B. TIDES versus direct length-conditioned SFT.
Direct SFT does not reproduce the attack: it maintains or improves clean accuracy in both groups, whereas TIDES reduces long-reference accuracy from 68.5% to 4.1% while preserving short-reference accuracy. The methods use length differently in supervision. Direct SFT learns from complete responses whose length correlates with correctness. TIDES uses LDV steering to construct the gated, position-conditioned teacher that direct SFT lacks, supervising every token of the student’s own rollouts.
[W2 / Q2] Paired attack-success rate
Response
We thank the reviewer for this suggestion. TIDES targets problems requiring extended reasoning in the clean model. RAS provides a lower bound on the requested ASR (the fraction of clean-correct problems made incorrect by TIDES), as it also subtracts reverse flips on the same problem set. For clean responses exceeding 1,000 generated tokens, RAS ranges from 62.4% to 100.0% across all 15 backbone–benchmark pairs (mean, 89.8%), including 97.1% for DeepSeek-R1-Distill-Qwen-7B on MATH-500. On problems solved with short reasoning, TIDES is designed to preserve accuracy: Pass@1 remains within 1.3 percentage points of the clean model or improves, rising from 67.4% to 68.2% for the same pair.
[W3 / Q3] Gate normalization
Response
We thank the reviewer for this question. T is a fixed horizon of 4,096 tokens, independent of the decoding budget.. The gate uses t=(st/T,1), where st counts generated tokens excluding the prompt. Thus, t measures rescaled absolute token count: shorter budgets truncate the rollout without rescaling t to 1. With a 512-token budget, t0.125. We have clarified this distinction in Section 4.2.
The gate is trained to keep injection negligible at early positions. Across 1,500 DeepSeek-R1-Distill-Qwen-7B teacher rollouts on MATH-500, the mean gate coefficient (t), computed over rollouts reaching position t, is 0.03 at t=512, 0.08 at 1,000 and 0.85 at 2,000. Consistent with these low early coefficients, a 512-token budget yields a Pass@1 decrease of only 0.4 percentage points for both the teacher (36.8% to 36.4%) and the ORZ-7B TIDES student (15.6% to 15.2%).
[W4 / Q4] Evaluation budgets
Response
We thank the reviewer for this question. Table 1 uses a 4,096-token generation cap for all models. “Short” (≤1,000 generated tokens) and “long” (>1,000) classify problems by the clean model’s response length, excluding the prompt, rather than by generation budget. Pass@1 and RAS are evaluated on the short and long groups, respectively, under this common cap. Figure 5 compares clean and attacked models on the same 50 prompts at six generation caps from 128 to 4,096 tokens. We have clarified these distinctions in the Table 1 caption and Appendix E.