acceptodds
Under review as a conference paper at ICLR 2027

TIDES: Test-time Inference Drift Exploitation via Scaling

10 views

Abstract

We propose TIDES, a reasoning attack method that exposes a previously unrecognized failure mode of test-time scaling: as reasoning traces lengthen, model performance can degrade sharply rather than improve. Unlike prior attacks on large reasoning models (LRMs), TIDES exploits computation depth under test-time scaling as an implicit activation condition, producing degradations that remain difficult to detect under short-budget evaluations. Methodologically, we define a Depth-Guided Latent Tracker (DLT), a depth-based tracker that injects microscopic steering vectors into intermediate reasoning representations and combines them with on-policy distillation to embed computation-dependent behavior in LRMs. Theoretically, we model latent space as a depth-indexed dynamic process and derive a worst-case propagation bound characterizing the potential sensitivity of long reasoning trajectories to small intermediate perturbations. Empirically, we evaluate TIDES on multiple reasoning benchmarks using three strong reasoning models, Phi-4-Reasoning, DeepSeek-R1-Distill-Qwen-7B, and DeepSeek-R1-Distill-Llama-8B, where it outperforms strong reasoning attack methods such as DecepChain and BadChain. Notably, TIDES achieves an average 26.8% relative improvement in RAS over the strongest baseline while largely preserving utility under short reasoning budgets.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Large Reasoning Models, Test-Time Scaling, Backdoor Attacks, Model Security, Reasoning Safety

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UJhHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TIDES (Test-time Inference Drift Exploitation via Scaling), a backdoor attack on reasoning models that uses reasoning length as an implicit trigger, requiring no input trigger. The poisoned model behaves normally under short reasoning budgets, but its accuracy drops sharply as reasoning length increases. The method consists of three steps: (1) extracting layer-wise “logic drift vectors” between correct and incorrect reasoning traces; (2) using a gate (DLT), conditioned on token position and entropy, to concentrate the intervention in the later stages of reasoning; and (3) embedding this behavior into ordinary model weights through on-policy distillation. Across three distilled reasoning models and five benchmarks, TIDES achieves an average RAS of 89.8%, a 26.8% relative improvement over the strongest baseline, DecepChain, while largely preserving Pass@1 under short reasoning budgets.

Critical strengths

1. Novel threat model perspective. The paper uses the amount of reasoning computation itself as the trigger, without requiring any input trigger. This highlights the need to evaluate the safety of reasoning models across different reasoning budgets, beyond conventional trigger detection.

2. Complete pipeline with a clear design. The three components: layer-wise drift vectors, depth gating, and on-policy distillation, each serve a clear purpose. The resulting model uses ordinary model weights, making the approach well suited to a supply-chain setting in which the attacker can only release weights.

3. Fairly comprehensive experiments. The evaluation covers three models, five benchmarks, and three random seeds, with standard deviations reported. Ablations examine extraction position, vector normalization, and injection strategy. The paper further evaluates robustness under different sampling parameters and parallel sampling, alongside detection at both the output and representation levels.

Critical weaknesses

1. Missing a core baseline, or missing an explanation of why the steering vector is necessary. Since a distillation step is required at the end anyway, why not directly construct data in which “short reasoning yields correct answers and long reasoning yields incorrect answers,” and then directly perform SFT or distillation?

2. Missing some other metrics. I would particularly like to see the ASR metric. I want to know how likely it is that examples answered correctly under short reasoning are successfully attacked after applying TIDES.

3. The gate design. The main input to the gate is ρ_t = t / T_max, which is a ratio relative to the maximum generation length rather than an absolute number of tokens. As a result, regardless of whether the budget is long or short, ρ_t approaches 1 in the later part of generation and the gate will be activated, which contradicts the goal of “remaining normal under short budgets.”

4. How many tokens exactly are the “short budget” and “long budget” in the paper? Under what budgets are the Pass@1 and RAS in Table 1 measured, respectively?

5. The choice of models is questionable. To my understanding, the DeepSeek-R1-Distill-Qwen-7B and R1-Distill-Llama-8B used by the authors are obtained by SFT on Qwen2.5 and Llama-3.1 with reasoning data generated by DeepSeek-R1, without RL; Phi-4-Reasoning is obtained by SFT on Phi-4 with reasoning demonstrations generated by o3-mini, and is also a distilled model. I am not sure whether such non-native models are more vulnerable to attack.

6. Discussion of generalization. The entire training of the vector is based on math. Although the authors also provide results on a code benchmark, I would like to see more discussion from this perspective. I want to know whether this attack on GSM8K can generalize to other benchmarks, and whether the vector learns to suppress all reasoning traces or selectively targets cases in the same domain as the training data.

Questions for authors
  1. Since a distillation step is required at the end anyway, why not directly construct data in which short reasoning yields correct answers and long reasoning yields incorrect answers, and then perform SFT or distillation? Could the authors add this baseline, or explain why the steering vector is necessary?

  2. Could the authors report the ASR, i.e., among the examples answered correctly under short reasoning, what proportion are successfully attacked after applying TIDES?

  3. The gate uses ρ_t = t / T_max, which is a ratio relative to the maximum generation length rather than an absolute number of tokens. Under a short budget, ρ_t also approaches 1 in the later part of generation. Why is the gate not activated in this case, and how does this design ensure that the model remains normal under short budgets?

  4. How many tokens exactly are the short budget and the long budget? Under what budgets are the Pass@1 and RAS in Table 1 measured, respectively?

  5. All three models used in the experiments are distilled models obtained by SFT without RL. Are such non-native models more vulnerable to this attack? Is TIDES equally effective on natively RL-trained reasoning models?

  6. The vector is trained entirely on GSM8K. Can the attack generalize to benchmarks in other domains? Does the vector learn to suppress all reasoning traces, or does it selectively target cases in the same domain as the training data?

4 comments
AuthorsOpenReview ↗

Response to Reviewer UJhH (1/2)

[W1/Q1] Latent steering versus direct training

Comment: 1. Missing a core baseline… // Since a distillation step is required…

Response

We thank the reviewer for suggesting this baseline, which we added using the same 2,000 prompts as TIDES. We fine-tuned the student on correct GSM8K responses (<1,000 tokens) and incorrect responses (>2,000 tokens) to OpenR1-Math-220k prompts, matching TIDES in student initialization, trainable parameters, optimizer and update count. The clean model generated the incorrect responses under the deceptive prompts used for LDV extraction, making this baseline a form of sequence-level distillation from the deceptively prompted clean model; we retained only normally terminated responses with extractable incorrect answers. Both models are trained for this controlled comparison; the main TIDES results remain those in Table 1.

We evaluated DeepSeek-R1-Distill-Qwen-7B on all 500 MATH-500 problems with a 4,096-token cap. Clean reference generations defined short-reference (≤1,000 generated tokens) and long-reference (>1,000) groups.

Table B. TIDES versus direct length-conditioned SFT.

Model constructionShort-reference Pass@1 (%)Long-reference Pass@1 (%)Absolute drop (pp)RAS (%)
Clean model67.468.5——
Direct length-conditioned SFT67.570.1−1.6−2.3
TIDES66.84.164.494.0

Direct SFT does not reproduce the attack: it maintains or improves clean accuracy in both groups, whereas TIDES reduces long-reference accuracy from 68.5% to 4.1% while preserving short-reference accuracy. The methods use length differently in supervision. Direct SFT learns from complete responses whose length correlates with correctness. TIDES uses LDV steering to construct the gated, position-conditioned teacher that direct SFT lacks, supervising every token of the student’s own rollouts.

[W2 / Q2] Paired attack-success rate

Comment: 2. Missing some other metrics… // Could the authors report the ASR…

Response

We thank the reviewer for this suggestion. TIDES targets problems requiring extended reasoning in the clean model. RAS provides a lower bound on the requested ASR (the fraction of clean-correct problems made incorrect by TIDES), as it also subtracts reverse flips on the same problem set. For clean responses exceeding 1,000 generated tokens, RAS ranges from 62.4% to 100.0% across all 15 backbone–benchmark pairs (mean, 89.8%), including 97.1% for DeepSeek-R1-Distill-Qwen-7B on MATH-500. On problems solved with short reasoning, TIDES is designed to preserve accuracy: Pass@1 remains within 1.3 percentage points of the clean model or improves, rising from 67.4% to 68.2% for the same pair.

[W3 / Q3] Gate normalization

Comment: 3. The gate design… // The gate uses t=t/T…

Response

We thank the reviewer for this question. T is a fixed horizon of 4,096 tokens, independent of the decoding budget.. The gate uses t=(st/T,1), where st counts generated tokens excluding the prompt. Thus, t measures rescaled absolute token count: shorter budgets truncate the rollout without rescaling t to 1. With a 512-token budget, t0.125. We have clarified this distinction in Section 4.2.

The gate is trained to keep injection negligible at early positions. Across 1,500 DeepSeek-R1-Distill-Qwen-7B teacher rollouts on MATH-500, the mean gate coefficient (t), computed over rollouts reaching position t, is 0.03 at t=512, 0.08 at 1,000 and 0.85 at 2,000. Consistent with these low early coefficients, a 512-token budget yields a Pass@1 decrease of only 0.4 percentage points for both the teacher (36.8% to 36.4%) and the ORZ-7B TIDES student (15.6% to 15.2%).

[W4 / Q4] Evaluation budgets

Comment: 4. How many tokens exactly…

Response

We thank the reviewer for this question. Table 1 uses a 4,096-token generation cap for all models. “Short” (≤1,000 generated tokens) and “long” (>1,000) classify problems by the clean model’s response length, excluding the prompt, rather than by generation budget. Pass@1 and RAS are evaluated on the short and long groups, respectively, under this common cap. Figure 5 compares clean and attacked models on the same 50 prompts at six generation caps from 128 to 4,096 tokens. We have clarified these distinctions in the Table 1 caption and Appendix E.

AuthorsOpenReview ↗

Response to Reviewer UJhH (2/2)

[W5 / Q5] Evaluation on RL-trained reasoning models

Comment: 5. The choice of models is questionable… // All three models used in the experiments…

Response

We thank the reviewer for this question. To test a natively RL-trained model, we evaluated TIDES on ORZ-7B, trained with RL directly from the Qwen2.5-7B base model without SFT or distillation [1]. We compared the released checkpoint with its TIDES student on all 500 MATH-500 problems at 512- and 4,096-token budgets (single run).

ModelPass@1 @512 (%)Pass@1 @4096 (%)Absolute drop @4096 (pp)RAS @4096 (%)
Clean RL checkpoint (ORZ-7B)15.671.8——
TIDES student15.234.437.452.1

TIDES is comparably effective on this RL-trained model. It preserved 512-token accuracy and reduced 4,096-token Pass@1 from 71.8% to 34.4% (52.1% RAS across all problems). Under the same protocol, TIDES reduced Pass@1 for the distilled DeepSeek-R1-Distill-Qwen-7B from 68.0% to 33.2% (51.2% RAS). Both RAS values include problems solved within the first 1,000 tokens, whose responses TIDES is designed to preserve. For DeepSeek-R1-Distill-Qwen-7B, any attack that preserves these short responses has an all-problem RAS ceiling of 54.0%. Thus, the distilled model was no more vulnerable than the RL-trained model in this comparison.

[1] Hu et al. Open-Reasoner-Zero: An Open Source Approach to Scaling Up Reinforcement Learning on the Base Model. arXiv:2503.24290, 2025.

[W6 / Q6] Cross-domain generalization

Comment: 6. Discussion of generalization… // The vector is trained entirely on GSM8K…

Response

We thank the reviewer for this question. Constructed solely from GSM8K and OpenR1-Math, TIDES achieves 62.4–100.0% RAS across all 15 backbone–benchmark pairs in Table 1, without using any evaluation benchmark in its construction. On LiveCodeBench, RAS reaches 88.4% for DeepSeek-R1-Distill-Llama-8B and 99.8% for Phi-4-Reasoning, demonstrating transfer from mathematics to code. FROST [2] reports similar transfer in reasoning efficiency, with a mathematics-trained intervention generalizing to code and physics.

The attack neither suppresses all reasoning nor targets only the training domain. It remains length-selective: short-response Pass@1 is no more than 1.3 percentage points below the clean baseline across all 15 pairs. On MATH-500, 65.4% of outputs from the DeepSeek-R1-Distill-Qwen-7B TIDES student end with an incorrect final answer; only 1.4% are truncated.

GSM8K LDVs point from correct to fallacious activation centroids. Correct and incorrect reasoning trajectories separate in representation space, and GSM8K-trained correctness predictors perform above chance on MATH-500 and MMLU [3], consistent with a partly shared error component along this direction across domains. The gate uses position and entropy, without domain labels.

Domain-specific LDVs. TIDES also remains effective with LDVs extracted from code and physics data. We compared each resulting student with its clean checkpoint at fixed generation budgets. Pass@1 is shown as Clean → TIDES; RAS is calculated from unrounded counts.

Held-out benchmarkPass@1 at 1,024 tokensPass@1 at 4,096 tokensRAS at 4,096 tokens
LiveCodeBench9.3% → 8.6%32.9% → 11.4%65.2%
UGPhysics (186 English Semiconductor Physics problems)12.4% → 11.3%31.7% → 16.7%47.5%

At 1,024 tokens, Pass@1 differs from the clean checkpoint by at most 1.1 percentage points on both benchmarks. Here, RAS includes all problems at each fixed budget, including those solved within the first 1,000 tokens, which TIDES is designed to preserve. Table 1 reports RAS for long responses only. For the DeepSeek-R1-Distill-Qwen-7B student on MATH-500, preserving short responses limits RAS across all problems to 54.0%; TIDES reaches 51.2% (Pass@1 68.0% → 33.2%).

[2] Luo et al. FROST: Filtering Reasoning Outliers with Attention for Efficient Reasoning. ICLR, 2026. arXiv:2601.19001.

[3] Sun et al. LLM Reasoning as Trajectories: Step-Specific Representation Geometry and Correctness Signals. ACL, 2026.

Thanks to the authors for the detailed response and the additional experiments. They address most of my concerns, and I will keep my score.

We thank the reviewer for the follow-up and the positive assessment. If any concern remains open, we would be glad to address it before the discussion closes.

Reviewer_tQU7OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces TIDES, a backdoor that aims to preserve accuracy under short reasoning budgets while causing failures during longer reasoning. The method extracts a vector for each model layer from correct and incorrect traces. A learned gate controls the intervention using token position and entropy. Distillation on student generated responses transfers this behavior into a standard student checkpoint.

Experiments cover three reasoning models and five benchmarks. The paper reports an average Relative Attack Score of 89.8% under long reasoning budgets while largely preserving accuracy under short budgets.

Critical strengths
  1. The attack setting is relevant. A poisoned checkpoint could pass evaluations with short reasoning budgets but fail when users allow longer generation. This motivates evaluating model reliability across different inference budgets.

  2. The results support utility preservation under short budgets. Table 1 shows that average accuracy under short budgets remains close to or exceeds the original model’s accuracy. Figure 5 also shows similar performance at small budgets, followed by substantial degradation at larger budgets. Preserving short responses is part of the attack objective and is not itself a weakness.

  3. The paper includes useful component tests and additional evaluations. The authors examine extraction positions, vector normalization, injection at individual layers, parallel sampling, and detection. The appendix also acknowledges the limits of the theoretical and detection results.

Critical weaknesses
  1. The benefit and activation behavior of the learned gate are insufficiently established.

The gate receives token position directly, with entropy providing additional modulation. However, the paper does not compare it against a fixed token threshold or a gate that uses only token depth. These comparisons would clarify whether learning the gate and incorporating entropy improve attack effectiveness while preserving utility under short reasoning budgets.

Figure 5 demonstrates that the distilled student’s performance degrades as the token budget increases, but it does not directly show how the teacher’s gate activates across token positions. Reporting these activations would help verify whether the gate behaves as intended.

  1. The vector ablations do not distinguish the contribution of the extracted directions from intervention magnitude.

Table 3 shows substantially stronger attacks with raw vectors than with normalized vectors. However, normalization changes both the overall intervention magnitude and the relative magnitudes across layers. The comparison therefore does not isolate which factor explains the improvement.

Figure 4 shows that applying each vector at its corresponding layer is more effective than reusing a single layer’s vector throughout the model. However, it does not establish whether the extracted directions outperform random directions with the same magnitude at each layer. This control would help determine whether the extracted directions contribute beyond the effect of perturbation magnitude.

Questions for authors
  1. How does the learned gate compare with a fixed token threshold in preserving utility under short reasoning budgets and inducing failures under long reasoning budgets? Does the entropy input provide additional benefit over token depth alone?
  2. What proportions of attacked outputs contain completed incorrect answers, truncated responses, or no extractable answer?
  3. Do the extracted vectors outperform random directions with matched intervention magnitudes at each layer, when the gate and all other settings are held fixed?
4 comments
AuthorsOpenReview ↗

Response to Reviewer tQU7

[W1/Q1] Learned gating and entropy modulation

Comment: The benefit and activation behavior of the learned gate… // How does the learned gate compare…

Response

We thank the reviewer for these questions.

Setup. DeepSeek-R1-Distill-Qwen-7B students distilled from teachers differing only in the gate were evaluated on all 500 MATH-500 problems (4,096-token cap; three seeds). Clean reference generations defined short-reference (232) and long-reference (268) groups using the 1,000-token threshold in Table 1; RAS was computed on the latter. The fixed gate injects at full strength only after the 1,000th generated token. All variants, including TIDES, are retrained for this controlled comparison; the main TIDES results remain those in Table 1.

Table A. Gate ablations.

Gating strategyShort-reference Pass@1 (%)Long-reference Pass@1 (%)Absolute drop (pp)RAS (%)
Clean model67.468.5——
Fixed token threshold (1,000)67.412.356.282.0
Learned position-only gate66.74.164.494.0
Learned entropy-only gate0.00.068.5100.0†
Learned position + entropy gate (TIDES)66.84.164.494.0

†Accuracy collapses in both groups; this degradation is not length-selective.

The learned gate increases long-reference RAS from 82.0% (fixed threshold) to 94.0% at comparable utility. All position-based gates keep short-reference accuracy within 0.7 percentage points of the clean model. Adding entropy leaves RAS unchanged at 94.0%, whereas entropy alone is not length-selective. Position is thus the primary signal, as designed in Section 4.2.

Teacher activation. The teacher’s gate behaves as intended. Across 1,500 rollouts (500 problems × three seeds), mean (t) among rollouts reaching position t rises from 0.03 at t=512 and 0.08 at 1,000 to 0.48 at 1,500, 0.85 at 2,000 and 0.96 at 4,096. Injection is therefore negligible in short responses and approaches full strength only in long rollouts.

[W2/Q3] Vector direction versus perturbation magnitude

Comment: The vector ablations do not distinguish… // Do the extracted vectors outperform…

Response

We thank the reviewer for these questions. Yes: at matched per-layer norms and otherwise identical settings, extracted LDVs produce substantially stronger attacks than random directions at 4,096 tokens.

Norm-matched direction control. We replace each LDV with an independently sampled random vector of equal per-layer L2 norm, keeping the gate, injection layers and global scaling fixed. Each vector set remains fixed across tokens and problems. We evaluate these teacher interventions on all 500 MATH-500 problems using DeepSeek-R1-Distill-Qwen-7B at 512- and 4,096-token budgets, reporting means over three seeds.

Teacher interventionPass@1 @512 (%)Pass@1 @4096 (%)Absolute drop @4096 (pp)RAS @4096 (%)
Clean model, no intervention36.868.0——
Per-layer norm-matched random directions36.548.219.829.1
Extracted LDVs36.412.455.681.8

At these per-layer norms, LDVs reduce Pass@1 from 68.0% to 12.4% at 4,096 tokens, versus 48.2% for random directions. The extracted direction thus accounts for most of the decline: 35.8 of 55.6 percentage points.

Normalization. Normalization preserves layer-wise directions, allowing Table 3 to isolate magnitude effects. Raw LDV norms correspond to the distances between correct and fallacious centroids at each layer, setting the overall scale and relative layer weights. Preserving this magnitude profile strengthens the attack compared with unit normalization (Table 3). The norm-matched control holds magnitudes fixed to isolate direction. We have clarified this distinction in Section 6.1.

[Q2] Types of attacked outputs

Comment: What proportions of attacked outputs…

Response

We thank the reviewer for this question. We evaluated the DeepSeek-R1-Distill-Qwen-7B TIDES student on all 500 MATH-500 problems with a 4,096-token limit. Averaged over three seeds, 33.2% of outputs were correct, 65.4% completed with an incorrect final answer, and 1.4% were truncated. All completed outputs contained an extractable answer, whereas none of the truncated outputs did, yielding a no-answer rate of 1.4%.

Dear Reviewer tQU7, we kindly remind you of our response above and would be happy to clarify any remaining concerns before the discussion closes. In the revised paper, the changes addressing your comments are highlighted in blue:

  • Section 6.1 and Appendix H.5 (Table 9): Comparisons of learned, fixed, position-only and entropy-only gates, and the teacher's gate activation by token position.
  • Section 6.1 and Appendix H.5 (Table 12): The norm-matched random-direction control; normalization changes only magnitude, whereas the norm-matched control isolates direction.
  • Section 6.1 and Appendix H.5: Output types of the attacked model (65.4% completed but incorrect, 1.4% truncated).

Thanks for your explanation. Author has explained most of my questions, so I will keep my rating.

Thank you for your positive assessment. We are glad our explanation was helpful, and we have added the gate ablations and the norm-matched random-direction control to the revised manuscript.

Reviewer_bA6rOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TIDES, a computation-dependent backdoor attack against large reasoning models effective for the large computation budget. The proposed model leverages the contrastive activation steering, a depth- and entropy-conditioned gate, and on-policy distillation to induce final-answer degradation under longer inference budgets while preserving short-budget utility. The results are interesting and reveal a potential bottleneck in the test-time scaling, which raises a serious security concern.

Critical strengths

The paper focuses on an important problem which is less explored issue with test-time scaling: giving a model more test time computation and reasoning capability does not always make it more reliable. I find the idea of using longer reasoning as a trigger for backdoor attacks interesting, especially since the attack does not require any explicit trigger words.

The authors combine the three components i.e., Logic Drift Vectors, a depth-based gating mechanism, and on-policy distillation to build their attack. These individual techniques are not novel but putting them together to create a backdoor that activates during longer reasoning is an interesting approach. Another interesting aspect I find is that the final model does not need any additional intervention at inference time.

The proposed method is evaluated on three reasoning models and five benchmarks, including comparisons with existing attacks. The results are promising, and shows that the attack can significantly affect performance during longer reasoning while largely preserving accuracy for shorter responses.

Critical weaknesses

The proposed Logic Drift Vector (LDV) construction is closely related to the contrastive activation steering methods such as CAA[1] and Activation Addition [2]. The other important work on positional backdoors like MetaBackdoor and BadThink are related, so I request the author to please justify the novelty of the paper.

The paper defines normalized reasoning depth as: ρt=tTmax⁡\rho_t = \frac{t}{T_{\max}}. This measures the position of a token in the generated sequence, not a reasoning depth. A model can generate say 4000 tokens of repetitive or irrelevant text without performing any meaningful reasoning than a model generating 500 correct tokens. Therefore the proposed model may be a generation length triggered backdoor, instead of a computation-depth triggered backdoor. I find this is important to address because the key contribution is framed as exploiting test-time reasoning computation.

Theorem 5.1 provides $

∥h~T−hT∥≤KT−1−t0η.\|\tilde h_T-h_T\|\leq K^{T-1-t_0}\eta.

$ This theorem follows directly from iterating a Lipschitz bound. It is an upper bound on possible perturbation propagation, not evidence that a perturbation actually grows. Also, Theorem 5.2 similarly establishes a necessary displacement for changing a correctness classification under a Lipschitz margin assumption. These theorem does not demonstrate that the proposed Logic Drift Vectors align with an error-producing direction or that TIDES causes exponential error amplification.

I think the RASh=Abase,h−Aattack,hAbase,hRAS_h=\frac{A_{\text{base},h}-A_{\text{attack},h}}{A_{\text{base},h}} has flaw in the formulation itself since a high RAS shows a significant relative accuracy degradation, but it does not directly imply a robust or distinctive attack. Consider the example: reducing accuracy from 2% to 0% gives RAS = 100%, even though the clean model is already nearly incapable of solving the task. I believe this is a bias estimate. There should be a better way.

The paper has shown the various ablations but the important ablation over the proposed component are missing: (1) paper should test if the learned gate is necessary or only a fixed intervention schedule suffices (No DLT versus DLT). (2) Establishes whether entropy provides any additional benefit beyond token position (Depth-only versus entropy-only versus both). (3) It is not clear if the distillation algorithm contributes anything beyond training on incorrect long responses. I believe these ablation are necessary and paper missing these important ablations.

Reference:

[1] Contrastive Activation Addition (CAA), ACL 2024

[2] Steering Language Models with Activation Engineering

[3] BadThink: Triggered Overthinking Attacks on Chain-of-Thought Reasoning in Large Language Models, AAAI 2026

Questions for authors

Please refer to the weakness section and provide the discussed ablation and justify the evaluation metric

5 comments
AuthorsOpenReview ↗

Response to Reviewer bA6r (1/2)

[W1] Novelty and related work

Comment: The proposed Logic Drift Vector…

Response

We thank the reviewer for highlighting these related methods. TIDES targets a specific failure mode of sequential test-time scaling: a poisoned reasoning model preserves short-budget utility but degrades final-answer correctness when generation is extended. LDV builds on contrastive activation steering, including CAA and Activation Addition. Section 4.2 adapts this principle by extracting layer-specific directions from correct and fallacious traces at entropy-selected positions. DLT modulates their application using rollout position and smoothed entropy. TIDES thus uses activation steering to construct budget-dependent teacher behavior and transfers that behavior into model weights through on-policy distillation, eliminating runtime intervention. MetaBackdoor associates input or accumulated conversation length with attacker-specified outputs through data poisoning. TIDES instead conditions interventions on progress within a generated reasoning rollout and distills them to induce short-/long-budget differences in answer correctness. This targets a validation–deployment gap: a checkpoint can retain accuracy during short-budget evaluation yet become unreliable when users allow longer reasoning for the same task. BadThink uses trigger prompts to induce excessive reasoning while preserving final-output consistency; TIDES uses extended reasoning as the activation condition and targets final-answer correctness. We revised Section 2 to clarify these distinctions and LDV’s relationship to established activation-steering methods.

[W2] Generation length and reasoning depth

Comment: The paper defines normalized…

Response

We thank the reviewer for this question. TIDES is triggered by generation length because sequential test-time scaling increases computation through additional reasoning tokens, each requiring one decoding step. TIDES therefore tracks decoding steps, the quantity that users and providers scale, through the normalized position t. This measures computation depth rather than semantic quality: the attack is designed to activate as computation increases, regardless of whether individual steps are useful. TIDES degrades reasoning, not merely length. LDVs point from correct to fallacious reasoning representations. On MATH-500 at a 4,096-token budget, they reduce the DeepSeek-R1-Distill-Qwen-7B teacher’s Pass@1 from 68.0% to 12.4%; random directions with matched per-layer norms under the same gate reduce it to 48.2%. Failures predominantly involve completed but incorrect answers: under the same token cap, 65.4% of the corresponding TIDES student’s outputs terminate normally with an incorrect answer, whereas only 1.4% are truncated. Across six budgets from 128 to 4,096 tokens (Fig. 5), attacked-model accuracy falls from 50% to 16% beyond approximately 1,000 tokens, while clean-model accuracy continues to improve. We revised the Introduction and Sections 4.1–4.2 to define computation depth as the number of decoding steps in a rollout.

[W3] Scope of the theoretical analysis

Comment: Theorem 5.1 provides…

Response

We thank the reviewer for this observation. Theorem 5.1 bounds worst-case perturbation propagation by iterating per-step Lipschitz bounds. Theorem 5.2 provides a necessary condition for a change in correctness. These results characterize the potential sensitivity of long reasoning trajectories to intermediate perturbations. Whether TIDES exploits this sensitivity, and whether LDVs point in error-producing directions, is tested empirically.

The experiments test the two questions that the theorems leave open. Theorem 5.2 requires a correctness change to move the terminal state across a margin; whether LDVs supply such an error-producing displacement is tested by a norm-matched control. On MATH-500 at 4,096 tokens, the DeepSeek-R1-Distill-Qwen-7B teacher's Pass@1 is 12.4% with LDVs and 48.2% with random directions of equal per-layer norm under the same gate (68.0% clean): at identical magnitude, direction accounts for 35.8 of the 55.6-percentage-point drop. The margin assumption is supported by the separation of correct and incorrect terminal states (Figure 2); Table 2 shows that correct–fallacious separation is largest at the high-entropy positions used for extraction, and layer-matched injection outperforms a single shared vector (Figure 4). Theorem 5.1 bounds how intermediate perturbations propagate to the terminal state. In the teacher, injection is weak at 1,000 tokens (mean gate coefficient 0.08) and strengthens only later (0.85 at 2,000), yet it changes final answers; the distilled student's failures are completed but incorrect solutions (65.4%) rather than truncations (1.4%). Section 5 now presents both theorems as worst-case and necessary-condition results, distinct from empirical evidence of attack effectiveness.

AuthorsOpenReview ↗

Response to Reviewer bA6r (2/2)

[W4] Interpretation of RAS

Comment: I think the RAS…

Response

We thank the reviewer for this question. We use RAS to enable direct comparison with DecepChain [1], the strongest prior attack in our evaluation. To make both relative and absolute effects explicit, we report it alongside clean and attacked accuracies and absolute drops in percentage points (pp). Our conclusions rest on substantial clean accuracies and large absolute drops. On MATH-500 long-reference problems, Pass@1 for the DeepSeek-R1-Distill-Qwen-7B TIDES student falls from 68.5% to 4.1% (64.4 pp). For RL-trained ORZ-7B, Pass@1 across all 500 problems falls from 71.8% to 34.4% at a 4,096-token budget (37.4 pp). The aggregate result is robust to the entries with the lowest clean short-response accuracy: excluding the four Table 1 entries with Pass@1 below 15% leaves the mean TIDES RAS at 87.1%, compared with 89.8% across all 15 entries. Appendix E now presents the robustness analysis; Appendix H.5 reports clean and attacked accuracies and the corresponding absolute drops for the controlled comparisons. [1] Wei Shen, Han Wang, Haoyu Li, and Huan Zhang. “DecepChain: Inducing Deceptive Reasoning in Large Language Models.” In Proceedings of the 43rd International Conference on Machine Learning (ICML), 2026.

[W5] Gate and component ablations

Comment: The paper has shown the various ablations…

Response

We thank the reviewer for suggesting these ablations.

Setup. All variants retain the main experiments’ LDVs, gate training data, injection layers, global scaling, distillation prompts and student initialization, changing only the component tested. We evaluate DeepSeek-R1-Distill-Qwen-7B on all 500 MATH-500 problems with a 4,096-token cap, averaging over three seeds. Clean reference generations define two groups, fixed across models: short-reference (≤1,000 generated tokens) and long-reference (>1,000). Absolute drop and RAS are computed on the long-reference group. All variants, including TIDES, are retrained for this controlled comparison; the main TIDES results remain those in Table 1.

Learned gating and entropy. The fixed gate disables injection for the first 1,000 generated tokens and applies full-strength injection thereafter. Position-only and entropy-only gates are trained separately.

Table A. Gate ablations.

Gating strategyShort-reference Pass@1 (%)Long-reference Pass@1 (%)Absolute drop (pp)RAS (%)
Clean model67.468.5——
Fixed token threshold (1,000)67.412.356.282.0
Learned position-only gate66.74.164.494.0
Learned entropy-only gate0.00.068.5100.0†
Learned position + entropy gate (TIDES)66.84.164.494.0

†Accuracy collapses in both groups; this degradation is not length-selective.

The learned position-driven gate enables a strong, selective attack. Replacing it with a fixed 1,000-token threshold reduces long-reference RAS from 94.0% to 82.0%, although the fixed, position-only and full gates all keep short-reference accuracy within 0.7 percentage points of the clean model. Learning when to intervene therefore improves on a fixed late trigger. Selectivity depends on position: adding entropy leaves RAS at 94.0%, whereas entropy alone collapses accuracy in both groups. These results confirm the design in Section 4.2, with computation depth triggering intervention and entropy providing secondary modulation.

Distillation versus direct SFT. Without the LDV/DLT teacher, we fine-tune the student on the same 2,000 prompts using correct GSM8K responses (<1,000 tokens) and incorrect responses (>2,000 tokens) to OpenR1-Math-220k prompts. The clean model generates the incorrect responses under the deceptive prompts used for LDV extraction. We retain responses that terminate normally with an extractable incorrect answer and pair them with the original prompts. Student initialization, prompts, trainable parameters, optimizer and update count match TIDES.

Table B. TIDES versus direct length-conditioned SFT.

Model constructionShort-reference Pass@1 (%)Long-reference Pass@1 (%)Absolute drop (pp)RAS (%)
Clean model67.468.5——
Direct length-conditioned SFT67.570.1−1.6−2.3
TIDES66.84.164.494.0

Direct SFT maintains or exceeds clean accuracy in both groups. TIDES reduces long-reference accuracy from 68.5% to 4.1% while preserving short-reference accuracy. Gated-teacher distillation thus produces length-selective degradation that direct SFT on incorrect long responses does not reproduce.

Dear Reviewer bA6r, we kindly remind you of our response above and would be happy to clarify any remaining concerns before the discussion closes. In the revised paper, the changes addressing your comments are highlighted in blue:

  • Section 2: How LDVs relate to contrastive activation steering (CAA, Activation Addition), and how TIDES differs from MetaBackdoor and BadThink in trigger and objective.
  • Introduction and Sections 4.1–4.2: Computation depth defined as the number of decoding steps in a rollout, with the gate normalized by a fixed 4,096-token horizon.
  • Section 5: Both theorems stated as worst-case and necessary-condition results, separate from the empirical evidence.
  • Section 6.1 and Appendix H.5 (Tables 9–10): The requested ablations: a fixed gate lowers long-reference RAS from 94.0% to 82.0%, entropy alone is not length-selective, and direct SFT on incorrect long responses keeps long-reference Pass@1 at 70.1% (TIDES: 4.1%).
  • Appendix E: RAS as a lower bound on paired attack success, and a mean RAS of 87.1% after excluding the four low-baseline entries (89.8% over all 15).

Comments on the paper

1 comment
AuthorsOpenReview ↗

Global Response to the Area Chair and Reviewers

Global Response to the Area Chair and Reviewers

We thank the Area Chair and reviewers for their constructive feedback, which helped us clarify our claims and evaluation protocol and add controls that isolate the mechanisms underlying TIDES.

This paper introduces TIDES, a computation-conditioned backdoor that preserves short-rollout accuracy while degrading final-answer accuracy during extended generation. Below, we summarize our key contributions, manuscript revisions, and additional reviewer analyses.


Key Contributions

  • Computation as an implicit trigger: TIDES uses the number of decoding steps, the quantity that sequential test-time scaling increases, as an implicit trigger rather than a trigger prompt (Sec. 4).
  • Checkpoint-level attack construction: Layer-wise Logic Drift Vectors (LDVs), position–entropy gating and on-policy distillation embed this behavior in model weights without runtime steering (Sec. 4.2).
  • Theoretical characterization: Under explicit assumptions, we derive a worst-case perturbation bound and a necessary latent displacement for correctness to change (Sec. 5).
  • Empirical evidence: Across three backbones and five benchmarks, TIDES achieves 89.8% mean Relative Attack Score (RAS) and 32.4% mean short-response Pass@1 (Table 1).

Major Revisions

1. Clarified Related Work, Definitions and Theory

  • Revised Sec. 2: Related LDVs to contrastive activation steering and distinguished TIDES from the triggers and objectives of MetaBackdoor and BadThink. reviewer bA6r
  • Revised Introduction and Secs. 4.1–4.2: Defined computation depth as decoding steps per rollout, and clarified that the gate normalizes generated-token position by a fixed 4,096-token horizon, independent of the decoding budget. reviewer bA6r reviewer UJhH
  • Revised Sec. 5: Separated worst-case sensitivity bounds from empirical attack effectiveness. reviewer bA6r

2. Strengthened Empirical Evaluation

  • Gate ablations: The learned gate increases long-reference RAS from 82.0% with a fixed threshold to 94.0% while keeping short-reference accuracy near the clean model. Position is the primary signal, as designed: the mean teacher gate coefficient is 0.03 at generated token 512, 0.08 at 1,000 and 0.85 at 2,000. reviewer bA6r reviewer tQU7
  • Direct SFT control: Direct length-conditioned SFT yields 70.1% long-reference Pass@1 (clean: 68.5%), whereas TIDES reduces it to 4.1%; direct SFT does not reproduce the length-selective degradation produced by gated-teacher distillation. reviewer bA6r reviewer UJhH
  • Norm-matched direction control: At matched per-layer norms, LDVs reduce 4,096-token Pass@1 to 12.4%, compared with 48.2% for random directions, an additional 35.8-percentage-point drop at identical per-layer norms; 512-token accuracies differ by only 0.1 percentage points. reviewer bA6r reviewer tQU7
  • RL-trained backbone: On ORZ-7B, Pass@1 changes from 15.6% to 15.2% at 512 tokens and from 71.8% to 34.4% at 4,096 tokens; its 52.1% RAS is comparable to the 51.2% obtained on the distilled DeepSeek-R1-Distill-Qwen-7B under the same protocol. reviewer UJhH

3. Clarified Evaluation Protocols, Metrics and Generalization

  • Revised Table 1 caption and Appendix E: Specified a single 4,096-token cap with problems grouped by the clean model’s response length (≤1,000 versus >1,000 generated tokens), and added a robustness analysis excluding low-baseline entries. reviewer bA6r reviewer UJhH
  • Revised Sec. 6.1: Clarified that LDV normalization changes magnitude only, while the norm-matched control isolates direction. reviewer tQU7
  • Metrics and failure types: RAS lower-bounds paired attack success, the fraction of clean long-response successes that TIDES turns into failures; in Table 1, these bounds are 62.4–100.0% (mean 89.8%), and the mean remains 87.1% after excluding the four entries with low baseline accuracy. Completed incorrect answers account for 65.4% of outputs; only 1.4% are truncated. reviewer bA6r reviewer tQU7 reviewer UJhH
  • Cross-domain generalization: Constructed from math data alone, TIDES achieves 62.4–100.0% RAS across all 15 backbone–benchmark pairs, including 88.4–99.8% on LiveCodeBench; LDVs extracted from code and physics also remain effective (65.2% and 47.5% all-problem RAS). reviewer UJhH

We once again thank the Area Chair and reviewers for their time and constructive feedback. The revisions clarify how TIDES conditions on test-time computation, what the theory establishes, and how the new controls isolate the contributions of gating, vector direction and training method. Revisions are highlighted in blue; new controls appear in Sec. 6 and Appendix H.5.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.