- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper investigates how reasoning models behave when the user question conflicts with a supplied chain-of-thought (CoT). The authors introduce CoT-Swap and show that reasoning-tuned models are more likely than instruct-tuned models to follow the injected CoT rather than the original question. Through controlled training, linear probing, and activation interventions, they examine the effects of trace supervision and mismatch decodability. Finally, a learned low-rank correction partially restores answers to the original question.
Critical strengths
The paper combines controlled training, linear probing, and activation interventions to investigate the phenomenon beyond behavioral observations. The finding that highly decodable mismatches can remain behaviorally unresolved is particularly interesting.
Critical weaknesses
W1. Limited practical relevance and unclear safety implications. The main experiments assume that a CoT from an unrelated question is inserted into the assistant's reasoning continuation, but it is unclear how often this situation would arise in realistic deployments. Table 2 shows that redirection largely disappears when the same CoT is presented as user-visible text. Moreover, STABLE is not necessarily a desirable outcome from a safety-monitoring perspective. Even when the final answer follows the user question, an unrelated internal CoT could undermine CoT-based monitoring. This distinction is not evaluated.
W2. Insufficient explanation of training effects. The approximately 46% increase in SWAPPED following trace-token supervision (Section 6) is surprising and lacks a clear explanation. While the authors separately average the answer and trace losses to avoid diluting the answer-token contribution, this differs from the standard practice of averaging the loss over all supervised tokens. Reporting results using the conventional whole-sequence mean loss would help establish whether the observed effect persists under a standard training objective.
W3. Potential confounding from the evaluation prompt. Appendix U.1 explicitly instructs all models to reason step by step inside <think> tags. This instruction may influence how models interpret and follow the injected CoT. An ablation removing this instruction while keeping the injected traces fixed would help establish whether the observed source-choice failures are amplified by the evaluation prompt.
W4. Limited qualitative analysis of continued reasoning. Although Section 5 shows that additional reasoning partially restores STABLE, the paper does not provide sufficient qualitative analysis of how models handle these conflicts. Examples distinguishing successful correction from cases where the model recognizes the conflict but still follows the injected CoT would help explain the gap between conflict detection and resolution.
W5. Insufficient details on human evaluation. Appendix B reports human annotation results but does not sufficiently describe annotator qualifications, recruitment, compensation, or detailed annotation guidelines. Given the 78.25% agreement between automatic matching and human adjudication, these details are important for assessing the reliability and reproducibility of the evaluation.
Questions for authors
See Weaknesses.