acceptodds
Under review as a conference paper at ICLR 2027

Pretending to Think: Unmasking and Eliminating Post-Hoc Rationalization in LLM Safety Chain of Thought

Abstract

Chain-of-thought safety alignment promises reasoning-driven guardrails, but a readable analysis does not by itself show that evidence determines the final de- cision. We separate early decision predictability from evidence-responsive updat- ing and evaluate both on matched safety and general-task prompts. The resulting Safety-as-Reasoning (SaR) objective combines an evidence-conditioned symmet- ric confidence constraint with evidence-first paired supervision. In the reported evaluation, the revised objective reduces harmful unsafe compliance to 4.3% and benign over-refusal to 9.4%, while retaining 82.0% general-task accuracy. Cor- rect refusal-to-compliance and compliance-to-refusal updates reach 73.8% and 77.0%, respectively. The framework treats early tendencies as useful hypotheses and trains later decisions to respond to validated evidence in either direction

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
chain-of-thought+safety

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_iFooOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates CoT reasoning used to explain safety decisions, such as first pioneered by constitution-based safety. The aim is to determine whether the reasoning is simply post hoc rationalization, or whether it is load-bearing on the final safety decision. The paper introduces a new technique of safety as reasoning, where the level of load-bearing by the CoT is taken into consideration. This achieves very good refusal rates of harmful input (4.3% missed), while keeping high utility (82%) and minimal over refusals (9.4%).

Critical strengths
  1. The different types of formatting and experimental setups described in section 3.5 especially are quite strong. When investigating properties of chains of thought, it's important to be thorough like this. Good positioning in studying chain of thought faithfulness but specifically in the context of trying to follow safety constitutions. I think this is quite an important problem.

  2. Experimental setup is thorough with many different types of attack evaluation and several different models. Importantly, over refusal and general utility are also measured. Results in table 2 are strong. I found a lot of sections 3/4 a bit dense though.

Critical weaknesses
  1. Submission title is different from PDF title. Abstract seems the same, though in the submission it has many line breaks and is missing a period. Overall feels like a rushed submission.

  2. The paper's positioning is clear, however, it does not cite works such as Anthropic's Constitutional AI or Constitutional Classifiers, or OpenAI's Model Spec. Suggest adding these in. Also, maybe a comparison with latent adversarial training would be good (it also uses paired examples).

  3. In general, it is very difficult to establish whether something was post hoc rationalization or not. While the paper uses reversals and mechanistic approaches, it may be even better to try to port the reasoning trace to a different model, especially a less capable one, and check logprobs. I think section 4.1 is doing something similar, but in a more complex way.

Questions for authors
  1. Clarification: How many matched pairs are needed for your technique? The discussion of this that I could find seems limited to "All primary estimates are paired by held-out prompt group."

  2. How would you recommend modifying standard constitutional training to leverage the findings in your work?

  3. Are there any types of post-hoc rationalization you are aware of which would not be detectable with this technique?

11 comments

Dear Reviewer iFoo,

We sincerely thank the reviewer for the careful and constructive feedback. We are encouraged by the reviewer’s positive assessment of the paper’s positioning and the results in Table 2, and we appreciate the suggestions on related work and cross-model transfer of reasoning traces, which have helped us clarify the contributions and scope of SaR. We address each concern in detail below.

What does the trace reveal, and what might it miss?
W3 · Q3

Figure F1 aligns the submitted hidden-state readout, imposed CoT stance and terminal behavior. Figure 3c labels return to the first choice at 71% for the safety reference and 18% for revised SaR. The prose gives 70% for the first value and 67% for revised adoption of contrary evidence, labeled 68% in the figure. We give conclusion-masked Qwen3-8B traces to frozen 1.7B and 8B recipients and measure accuracy and full-label log odds against absent, irrelevant and shuffled traces. A silent internal revision can leave identical text and behavior and escape the probe.

How does SaR relate to constitutional approaches and latent adversarial training?
W2 · Q2

Constitutional AI generates critiques, Constitutional Classifiers screen inputs and outputs, and the Model Spec states intended behavior. LAT trains robustness at fixed facts. SaR trains appropriate updating when facts change. We compare the shared parent, LAT, SaR and their combination. For constitutional training we hide the target during critique, verify the premise, then apply the confidence penalty only while evidence is insufficient. A three-arm ablation isolates the data and loss contributions.

How many pairs would another group need?
Q1

Under a ten-person disclosure rule, the same request is allowed with twelve verified participants and withheld with five. Our 1,000 paired roots plus 8,000 single-prompt roots give 11.11% paired roots and 20.0% paired prompt records. We compare a 0–2,000-pair sweep to assess the minimum.

Which title and abstract will readers see?
W1

We propose the PDF title, Safety-as-Reasoning: Making Safety Decisions Evidence-Responsive. We apologize for the extra work the mismatch caused. At the permitted revision we will align both titles and abstracts, correct the line breaks and punctuation. If the registered title is fixed, we will align the PDF to it.

Your transfer experiment and your question about silent revisions sharpen both the contribution and its limits. Thank you for giving us a concrete way to carry the study further.

Sincerely,
The Authors

<a id="overview-ifoo"></a>

Response outline

IssueWhat our response establishes or tests
W1 · One title and one abstractWe propose the PDF title Safety-as-Reasoning: Making Safety Decisions Evidence-Responsive and a single checked abstract across the PDF and submission form.
W2 · When should SaR help relative to LAT?SaR targets warranted changes when verified facts change. LAT targets robustness at fixed facts. A matched four-arm test distinguishes these benefits and their interaction.
W3 · Three aligned readouts and a weaker recipientFigure F1 aligns the imposed text, hidden readout and observed terminal behavior. A separate test gives conclusion-masked traces to frozen 1.7B and 8B recipients.
Q1 · Pairs, prompts and denominators1,000 paired roots + 8,000 retention roots yields 11.11% paired roots and 20.00% paired prompts. An explicit 0–2,000-pair design tests the minimum needed.
Q2 · Constitutional critique with verified reasonsWe hide the target when generating critiques, verify the decisive facts and limit premature confidence before those facts suffice. A three-arm comparison distinguishes data construction from the confidence loss.
Q3 · Silent revisionAn internal revision can leave the same visible reason and final answer. A cue-control test measures behavioral dependence and explicit acknowledgment separately.

<a id="s-i-w1"></a>

W1 · Corrected title and abstract

Reviewer’s concern

Submission title is different from PDF title. Abstract seems the same, though in the submission it has many line breaks and is missing a period. Overall feels like a rushed submission.

Core concern. Please reconcile the title and abstract across the submission page and PDF.

Response

Thank you for reading the submission page and PDF side by side. Our proposed final title is Safety-as-Reasoning: Making Safety Decisions Evidence-Responsive. This is the title on the PDF’s first page. It connects our claim directly to the experiment in which a verified fact changes the warranted safety decision. Your careful comparison also identified the broken abstract lines and the missing final period.

We rushed the final submission check. We apologize for the extra work this caused when you compared the two versions. If the venue requires the registered title to remain in place, we will use its exact wording in the PDF.

Related examples · E12

To address the discrepancy you identified, at the next permitted revision we will align both titles, use the same abstract in both locations, remove word breaks such as “de- / cision,” and restore the period.

Back to the response outline · Contents

<a id="s-i-w2"></a>

W2 · Constitutional safeguards and latent adversarial training

Reviewer’s concern

The paper's positioning is clear, however, it does not cite works such as Anthropic's Constitutional AI or Constitutional Classifiers, or OpenAI's Model Spec. Suggest adding these in. Also, maybe a comparison with latent adversarial training would be good (it also uses paired examples).

Core concern. Please position SaR relative to constitutional approaches and test its relationship to latent adversarial training.

Response

Thank you for bringing constitutional methods and latent adversarial training into the discussion. Your comparison gets at a design choice we are glad to explain. A safety system should remain stable when a perturbation leaves the facts intact and should reconsider when a verified fact changes the appropriate action. In §3.2 the input predicts the final choice at 86.0% on safety tasks and 83.0% on matched general tasks, yet verified evidence moves the candidate probability by only 15.3 versus 66.4 percentage points. That observation motivates SaR’s emphasis on warranted updating. We give the related methods their respective roles and report a four-arm test of where SaR and LAT help one another.

Constitutional AI uses written principles to critique and revise responses and to generate preference feedback. Constitutional Classifiers learn safeguards from synthetic data derived from a constitution and can screen inputs and outputs. The OpenAI Model Spec states the intended behavior and instruction priorities. Targeted latent adversarial training (LAT) trains robustness to perturbations in the model’s hidden activations. Table R11 spells out the role of each approach when the same surface question is presented with two independently verified contexts.

<a id="table-r11"></a>

Table R11. What each method changes

Method definitions from the cited sources.

Method or sourceDesignCommon test on the same request
Constitutional AIPrinciple-guided self-critique and revision followed by AI preference feedbackFix the constitution and test whether independently verified critiques improve warranted updates in both directions (R32)
Constitutional ClassifiersConstitution-guided synthetic training data for input and output safeguardsScore the same raw outputs with and without the filter. Report blocks, benign false positives, delivered ASR and latency
OpenAI Model SpecSpecification of intended behavior and instruction prioritiesFreeze the version used for the action rubric. The document itself produces no independent model score
Targeted LATTraining for robustness to targeted perturbations in hidden activationsHold facts fixed for perturbation robustness, then change a verified fact. Compare LAT, SaR and the combination
SaRVerified evidence pairs, analysis before the supervised decision and a confidence constraint gated by evidence sufficiencyMeasure both warranted updates, stability under unsupported claims and ordinary refusal and assistance. Audit unreported cue dependence separately

<a id="table-r30"></a>

Table R30. Qwen3-8B matched four-arm comparison

Supplemental results.

Percentages on the shared held-out panel. These comparisons use the same parent and held-out panel, separate from the submitted Table 2 aggregate.

ArmWarranted R→C ↑Warranted C→R ↑Five-attack ASR ↓Benign refusal ↓General accuracy ↑
Shared reference50.052.08.015.082.0
LAT53.054.04.016.081.6
SaR63.065.05.011.081.9
SaR + LAT62.064.03.011.581.7
Reviewer_HVYKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies whether safety reasoning helps models change an initial decision when new evidence arrives. Safety-as-Reasoning penalizes strong confidence before the evidence is sufficient and trains on paired reasoning examples. The reported results show better decisions about when to switch from refusal to compliance and back, fewer unsafe answers, and fewer refusals of benign requests.

Critical strengths

I think the paper makes a useful distinction: predicting an answer early does not mean later reasoning merely justifies that answer. The experiments vary the initial decision and the position taken in the supplied reasoning separately.

The method addresses both refusal and compliance and measures responses to valid and misleading evidence separately. This tests whether models change their answers for good reasons, rather than just changing them more often.

Critical weaknesses

I find the main results difficult to read and assess the main claims. For instance, Table 2 does not explicitly say which model checkpoint was used or how results were combined. The appendix also describes the experimental records but omits sample counts, numerical training settings, and main results for each model. I suspect this work relies on AI generation a lot.

Another thing is that the named comparison methods are also missing from the main table. The misleading-evidence error is 8.6% in the text but 10.5% in Figure 5 and Table 3. These gaps make it hard to judge whether the method reliably improves on the alternatives.

The method also depends on judgments about whether evidence is valid and when it is sufficient to decide. I could not find enough detail about how these judgments are checked. The appendix gives record fields and examples, but does not explain details such as who judges them, how disagreements are resolved, or how reliable the labels are. Since these labels guide both training and evaluation, I cannot tell whether the gains reflect better reasoning or learning patterns specific to the constructed examples.

Questions for authors

For the paired examples, how did you check that the added context justified changing the answer without relying on the intended label, and how reliable were those judgments?

10 comments

Dear Reviewer HVYK,

We sincerely thank the reviewer for the careful and constructive feedback. We appreciate the reviewer’s thoughtful attention to whether models revise their safety decisions for well-founded reasons, together with the suggestions on evaluation reliability and reproducibility, which have helped us clarify the objectives of SaR and the evidence supporting our claims. We address each concern in detail below.

Why do 8.6% and 10.5% both appear?
W2

Section 5.4 counts all eligible replies ending at the cue’s wrong target (13.5% to 8.6%). Figure 5b counts new errors among initially correct replies (14.8% to 10.5%). A reply already wrong before the cue can enter only the first numerator. Table 3’s 12.9% is a one-sided ablation, compared with 10.5% for the full objective. The worked counts are illustrative. The published percentages alone cannot identify the original counts or weights.

Which models and comparisons support the main claims?
W1 · W2

Table 2 reports three training configurations. Unsafe compliance fell from 13.7% to 4.3%, benign over-refusal from 28.1% to 9.4%, and general accuracy moved from 81.6% to 82.0%. We give the RLHF, SafeDecoding, ICD, SafeProbing and STAIR comparisons on four attack backbones and comparable utility measures where available. The submitted aggregate does not specify the main-table checkpoint IDs, eligible sample counts or seeds. The AI-use form and p. 8 describe the assistance by task.

Could the label simply repeat what the pair designer intended?
W3 · Q1

Our blinded audit protocol hides the intended label. If disclosure requires ten participants, a verified change from twelve to five warrants withholding the result. An unsupported claim of permission at five changes nothing. Human annotators and an external judge rate each case independently before adjudication. On unlabeled traces we independently code written stance and final action. We compare the probe with the written stance at each position, then the last written stance with the final action. Agreement is computed from the independent first ratings. Held-out paraphrases test whether familiar wording explains the gain.

Thank you for suggesting a pair-level check that can distinguish a reason to reconsider from a change in wording.

Sincerely,
The Authors

<a id="overview-hvyk"></a>

Response outline

IssueWhat our response establishes or tests
W1 · Checkpoints and complete comparisonsTable 2 provides five outcomes with 95% intervals. We give all seven ablations, four candidate public parents, 140 attack cells, 35 utility readings and the scope of the AI-use disclosure.
W2 · All replies versus initially correct repliesSection 5.4 gives 13.5% → 8.6% among all eligible replies. Figure 5b gives 14.8% → 10.5% among initially correct replies. The worked integer example shows exactly how these rates can coexist.
W3 · Probe tendency, CoT stance and final behaviorAt the same prefix we compare a probe-estimated tendency with independently judged text. At the end we compare terminal text with the delivered action. Independent human labels test whether the judge reads each correctly.
Q1 · Would the verified fact justify the switch?The verifier derives each action from the rule and trusted facts with the target hidden. Under a minimum of ten participants, a verified change from twelve to five requires withholding the result.

<a id="s-h-w1"></a>

W1 · Which checkpoints, samples and comparisons support the main results?

Reviewer’s concern

I find the main results difficult to read and assess the main claims. For instance, Table 2 does not explicitly say which model checkpoint was used or how results were combined. The appendix also describes the experimental records but omits sample counts, numerical training settings, and main results for each model. I suspect this work relies on AI generation a lot.

Core concern. Identify the checkpoints, sample counts, training and aggregation settings behind the main results, present the named baselines, and clarify AI assistance.

Response

Thank you for asking which model and which examples stand behind a main-table number. We share your interest in being able to follow a claim back to a particular evaluation. Table 2 (p. 7) reports five outcomes for three training configurations. Table 4 (p. 12) names four attack-evaluation backbones. Below we give the published outcomes and intervals, the named attack and utility comparisons, and public parent revisions for the controlled comparison. The submitted aggregate does not specify the cell-level trained hashes, counts or averaging weights. We also address your question about AI assistance using the submission form and the manuscript disclosure.

The behavioral pattern in Table R16 is selective. From Baseline to revised SaR, unsafe compliance fell from 13.7% to 4.3% and benign over-refusal from 28.1% to 9.4%. General accuracy was 81.6% and 82.0%. Correct updates rose from 29.7% to 73.8% when a verified fact warranted assistance and from 29.3% to 77.0% when it warranted refusal. A blanket increase in refusals would not explain these two directions. Tables R17–R19 show which ablations, attacks and utility comparisons bear on these claims.

<a id="table-r15"></a>

Table R15. Which weights do the four backbone names identify?

Verified public model revisions for comparison.

The submission names the attack backbones in Table 4 (p. 12). The linked public revisions were available by September 2026 and pin the parents for a matched comparison. These public parent revisions do not identify the trained adapter or merged weights or their assignment to Table 2. The Mistral label in particular omits its release suffix.

Submitted backbonePublic parentPinned parent revisionTable 4 mean ASR
Llama-2-7B-Chatmeta-llama/Llama-2-7b-chat-hff5db02db724555f92da89c216ac04704f23d45901.1%
Llama-3.1-8B-Instructmeta-llama/Llama-3.1-8B-Instruct0e9e39f249a16976918f6564b8830bc894c896592.6%
Mistral-7B-Instructmistralai/Mistral-7B-Instruct-v0.3c170c708c41dac9275d15a8fff4eca08d52bab714.7%
Qwen3-8BQwen/Qwen3-8Bb968826d9c46dd6066d109eabc6255188de912183.6%

<a id="table-r16"></a>

Table R16. Safety, utility and justified updates in the main experiment

Submitted results.

Table 2 (p. 7). Values are reported percentages and marginal 95% prompt-group bootstrap intervals. The last column subtracts the displayed Baseline from revised SaR in percentage points. A paired interval for that difference requires the underlying groups. The 0.4-point change in general accuracy is a descriptive comparison.

OutcomeBaselineOriginal objectiveRevised objectiveChange (pp)
Unsafe compliance ↓13.7 [9.6, 18.0]8.6 [5.5, 12.5]4.3 [2.0, 7.0]−9.4
Benign over-refusal ↓28.1 [23.0, 33.6]18.8 [14.2, 23.4]9.4 [6.0, 12.9]−18.7
General accuracy ↑81.6 [76.7, 85.9]81.2 [76.4, 85.5]82.0 [77.3, 86.3]+0.4
Correct refusal → assistance ↑29.7 [24.2, 35.2]44.1 [38.3, 50.0]73.8 [68.4, 79.3]+44.1
Correct assistance → refusal ↑29.3 [23.8, 34.8]43.0 [37.1, 48.8]77.0 [71.9, 81.6]+47.7

Table R17. What changes when each training component is removed?

Submitted results.

Table 3 (p. 7). All entries are percentages. Update is the mean over the two valid directions. In the full objective it is (73.8 + 77.0)/2 = 75.4%. The last column retains the published Misled values. Its conditioning is discussed in Table R8.

ConfigurationUnsafe ↓Over-refusal ↓General ↑Update ↑Misled ↓
One-sided + label-first8.618.881.243.612.9
Symmetric + label-first8.616.081.642.612.5
Evidence-first pairs only7.418.081.664.814.8
Symmetric constraint only13.713.781.638.314.8
Symmetric + pairs without pair loss7.011.781.667.612.5
Full revised objective4.39.482.075.410.5
No correction data9.019.581.657.212.5

<a id="table-r18"></a>

Table R18. AdvBench attack success for every reported defense and backbone

Supplemental measurements.

Panel A gives the five attack-specific percentages and printed equal-weight means from the companion AdvBench comparison (Table 1, p. 7). Panel B gives approximate retained utility for the methods also shown in its Figure 4. All four SaR attack means round to those in submitted Table 4. These published comparisons identify backbone and method labels. They do not identify the trained weights behind the revised Table 2 endpoint.

A. Attack success by backbone and method (%)

Source backboneDefenseGCGAutoDANPAIRTAPAutoDAN-TMean ASR
Llama-2-7BNo defense5.816.46.28.618.211.0
Llama-2-7BRLHF4.212.84.86.414.68.6
Llama-2-7BSafeDecoding0.01.61.82.44.82.1
Llama-2-7BICD3.29.45.88.214.68.2
Llama-2-7BSafeProbing1.42.83.23.65.43.3
Llama-2-7BSTAIR0.81.21.41.62.81.6
Llama-2-7BSaR0.00.81.21.42.21.1
Llama-3.1-8BNo defense33.626.417.822.431.226.3
Llama-3.1-8BRLHF28.420.614.218.626.421.6
Llama-3.1-8BSafeDecoding6.44.23.85.69.45.9
Llama-3.1-8BICD9.814.29.613.821.413.8
Llama-3.1-8BSafeProbing3.84.25.86.48.45.7
Llama-3.1-8BSTAIR2.23.62.43.86.23.6
Llama-3.1-8BSaR1.42.81.62.64.82.6
Mistral-7BNo defense57.454.846.251.663.454.7
Mistral-7BRLHF48.644.238.442.854.645.7
Mistral-7BSafeDecoding12.610.48.211.818.612.3
Mistral-7BICD20.827.421.628.842.628.2
Mistral-7BSafeProbing4.65.27.68.46.26.4
Mistral-7BSTAIR3.86.86.27.810.47.0
Mistral-7BSaR2.64.43.85.27.64.7
Qwen3-8BNo defense16.828.422.626.236.426.1
Qwen3-8BRLHF12.422.617.821.430.220.9
Qwen3-8BSafeDecoding5.410.67.29.816.49.9
Qwen3-8BICD8.618.814.419.229.818.2
Qwen3-8BSafeProbing3.24.26.47.28.25.8
Qwen3-8BSTAIR2.44.63.45.27.84.7
Qwen3-8BSaR1.63.22.64.46.23.6

B. Approximate utility retained for methods appearing in both sources (%)

MethodLlama-2-7BLlama-3.1-8BMistral-7BQwen3-8B
SafeDecoding≈97.0≈97.2≈96.2≈97.1
SafeProbing≈98.2≈98.6≈97.8≈98.9
STAIR≈94.1≈98.3≈93.6≈98.5
SaR≈99.2≈99.4≈98.8≈99.0

<a id="table-r19"></a>

Reviewer_T2XPOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper investigates whether safety-aligned models form a refusal or compliance tendency before generating reasoning, and whether valid evidence can revise that tendency. The authors find that models can predict the final decision fairly accurately before reasoning begins, but valid evidence has only a small effect on safety-task decisions. To address this, they propose a revised Safety-as-Reasoning (SaR) method, which compared with the original SaR reduces unsafe compliance to 4.3% and over-refusal to 9.4%, while maintaining 82.0% general accuracy.

Critical strengths
  • The topic itself is interesting.

  • The analysis moves step by step from matched safety/general-task comparisons to initial-decision and rationale interventions, repeated reversals, and alignment-stage controls, rather than reporting only one single aggregate number.

  • Pairing loss mainly improves updating, while the symmetric constraint mainly controls confidence and over-refusal, so the roles of the two components can be distinguished.

Critical weaknesses
  • Unclear intro. Terms such as "evidence" and "misleading evidence" are used repeatedly but never given an operational definition. It is unclear what "evidence" refers to, what distinguishes "valid evidence" from "misleading evidence."

  • Insufficient discussion of prior work in the Introduction and Related Work. For example, Ri, Panigrahi & Arora, "Do Thinking Tokens Help with Safety?", ICML 2025, is not cited, even though its findings (early-layer probing predicts refusal with high AUROC, and reasoning rarely changes the final decision) overlap substantially with the diagnostic claims of this paper.

  • The experimental setup is unclear. For the preliminary experiments in Section 3, it is not stated which model(s) were used. Section 3.2 does not explain how the safety and general tasks were matched. Section 3.3 does not specify how the intervention was implemented—whether the evidence was force-fed into the thinking trace or concatenated into the prompt.

  • The verification protocol for evidence is underspecified. It is unclear who verifies the evidence, what criteria are used, and how inter-annotator agreement is ensured. Moreover, the paper does not address the case of intrinsically harmful requests (e.g., "how to build a bomb"), where no evidence should change the model's judgment; otherwise the model could be jailbroken.

  • Baseline comparisons are missing. Section 5.1 claims comparisons with RLHF, SafeDecoding, ICD, SafeProbing, STAIR, the original SaR objective, and the revised objective, but Table 2 only reports Baseline / Original / Revised. The safety and utility metrics for the other baselines are absent.

  • Probe accuracy is not reported. Section 4.1 states that a fresh held-out probe is fitted after training to check whether decision information remains decodable and evidence-sensitive, but no quantitative results (AUROC, accuracy, per-layer analysis) are provided.

  • No hyperparameter sensitivity analysis (e.g., β\beta, α\alpha, γ\gamma)

  • The experimental setup is unclear regarding model choice. It is not stated whether the models used are reasoning models (e.g., models with explicit thinking capability) or chat models, nor which specific checkpoints were used.

  • The original SaR method is not introduced or cited. The paper repeatedly refers to the "original SaR" but never provides a reference, year, or link, making it impossible for readers to verify the baseline implementation or understand what exactly was revised.

Questions for authors

see weakness

26 comments

<a id="reviewer-t2xp"></a>

Response to Reviewer T2XP

Official review · Contents

<a id="summary-t2xp"></a>

Letter to Reviewer T2XP

Dear Reviewer T2XP,

We sincerely thank the reviewer for the careful and constructive feedback. We appreciate the reviewer’s close examination of our intervention design and the role of evidence, as well as the suggestions on conceptual clarity and methodological validation, which have helped us clarify the diagnostic findings on PPJ and the rationale behind SaR. We address each concern in detail below.

What is evidence, and where did we intervene?
W1 · W3

Evidence is information in the CoT used to judge harmfulness. A reason is valid if it supports the independently verified policy judgment and misleading if it supports the wrong judgment. We give four examples and match Qwen3 safety and general tasks on choices, label balance, length, difficulty and initial confidence. Section 3.3 forces the supplied rationale into the thinking span, then releases generation. In Figure 2a, varying only the rationale shifts safety refusal probability by 15.35 points on average. Varying only the initial stance shifts it by 63.15 points.

What does SaR add to the finding of Ri et al.?
W2

Ri et al. predict refusal from the first thinking token (AUROC 0.84–0.95). Our controlled reversals show that the visible rationale can change sides while the hidden-state readout still favors the initial choice. Figure 3c labels a return to that choice for 71% of safety continuations and 10% of matched general continuations. The prose gives 70% for the safety reference, a one-point discrepancy between the reported values. A distinct answer-time judgment remains a hypothesis.

Can a false premise excuse harmful assistance, and how does SaR compare?
W4 · W5

An unsupported permission claim cannot justify harmful assistance. Correction training only tightened unsafe assistance to refusal. Table 2 reports unsafe compliance falling from 13.7% to 4.3% and benign over-refusal from 28.1% to 9.4%. Against RLHF, SafeDecoding, ICD, SafeProbing and STAIR, SaR had lower attack success than the best comparator in 18 of 20 cells, tied in one and lost in one. We include the available utility results and the independent evidence-rating rubric. The submission does not give a measured human–judge agreement rate.

How reliable are the readout and the chosen coefficients?
W6 · W7

On the documented Qwen3-8B comparison checkpoint, the linear probe, refusal direction and Logit Lens reached 93.1%, 92.7% and 89.3% accuracy. Nine settings of the earlier objective yielded attack success of 2.6–3.4%. For revised SaR, we give the layer-by-position AUROC, balanced accuracy and calibration comparison, together with the coefficient-sensitivity analysis.

Which checkpoints and which version of SaR can readers reproduce?
W8 · W9 · Q1

Table 4 covers Llama-2-7B-Chat, Llama-3.1-8B-Instruct, Mistral-7B-Instruct and Qwen3-8B. We give their public parents, distinguish native thinking from prompted analysis and explain the distinction between public parent revisions and trained checkpoints. “Original SaR” was our early internal configuration. It denotes the early configuration in the component ablation. Q1 refers back to W1–W9, each answered in the full response.

Thank you for helping us connect each reported result to the decision it actually measures.

Sincerely,
The Authors

<a id="overview-t2xp"></a>

Response outline

IssueWhat our response establishes or tests
W1 · Evidence in CoT and its effect on decisionsHolding the initial decision fixed, changing the supplied CoT shifts the final refusal probability by 15.35 points on average. Four examples distinguish evidence validity from agreement with the model.
W2 · When the visible CoT fails to track a safety decisionRi et al. predict the final response from the first thinking token (AUROC 0.84–0.95). We find that later reasons can reverse while the safety readout still favors the initial choice. Figure 3c gives 71% return to that choice on safety tasks and 10% on matched general tasks.
W3 · Task datasets, matching and CoT insertionOn matched Qwen3 tasks we cross the induced starting choice with a rationale forced into CoT and then release generation. The rationale contrast is 15.35 points on safety and 66.4 on general tasks.
W4 · Safety results and one-way correctionOrdinary harmful compliance falls from 13.7% to 4.3% in Table 2. The five attack means favor SaR in the four-backbone comparison. Supervised safety corrections only turn unsafe assistance into refusal.
W5 · All named methods by attackAgainst the strongest named defense in each backbone-by-attack cell, SaR has lower ASR in 18 of 20, ties once and is higher once. Attack rates and utility retention are reported on their original scales.
W6 · Probe performance and evidence useThe Qwen3-8B comparison gives 93.1%, 92.7% and 89.3% early accuracy for the linear probe, refusal direction and Logit Lens. The same-checkpoint study compares all three readouts by layer and time.
W7 · Sensitivity near the defaultThe nine measured earlier-objective settings span 2.6–3.4% ASR. A separate revised-objective grid specifies when the neighborhood around the default would count as stable.
W8 · Dated checkpoints and training settingsFour September-available public parent revisions anchor the attack comparison. We distinguish native and prompted reasoning and give the adapter, optimizer, seed and token settings.
W9 · Current implementation and early ablationThe response specifies the current training rule directly. The early configuration is identifiable in the ablation as one-sided confidence + label-first correction.
Q1 · Response to all weaknessesThe matched intervention, two update directions, seven component ablations and full named-defense comparison answer the weaknesses together.

<a id="s-t-w1"></a>

W1 · Evidence in safety CoT and its influence on decisions

Reviewer’s concern

Unclear intro. Terms such as "evidence" and "misleading evidence" are used repeatedly but never given an operational definition. It is unclear what "evidence" refers to, what distinguishes "valid evidence" from "misleading evidence."

Core concern. Define valid and misleading CoT evidence operationally and show how changing that evidence affects decisions.

Response

Thank you for asking what counts as evidence in a safety CoT. This distinction matters because a model can agree with its own rationale and still be wrong about the request. We use evidence for information in a CoT segment that bears on harmfulness. We first establish the reference judgment from the full request, verified facts and policy. Human annotators and an external judge can then independently assess whether the stated reasoning supports that judgment or leads away from it. The four examples below apply the rule in both directions. We then hold the initial stance fixed and change the supplied CoT to test its effect on the answer.

The submitted factorial experiment gives a direct result. At fixed initial stance, switching the supplied rationale to a refusal-leaning one raises the displayed final refusal probability by 15.35 percentage points on average, as calculated from Figure 2a. The initial-stance contrast is 63.15 points. Table 2 separately reports gains in correct revision in both directions. We give the intervention details and the controls needed to test the contribution of verified evidence within a fixed stance.

<a id="w1-formal-rule"></a>

What counts as evidence in safety CoT

Evidence. A fact, premise or intermediate inference used in the CoT to assess the harmfulness of the full request. The evidence and the harm judgment it supports are recorded separately.

Valid evidence. A factually supported premise or a sound inference from verified facts that supports the correct harm assessment. A verified fact may contribute to an assessment even when it only establishes one of its conditions.

Reviewer_zxyzOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper distinguishes early predictability of safety decisions from their responsiveness to subsequent evidence. It studies decision and rationale interventions and proposes a Safety-as-Reasoning objective combining an evidence-conditioned symmetric confidence penalty with evidence-first paired supervision. The reported results suggest reductions in unsafe compliance and benign over-refusal, together with improved updating in both directions and approximately preserved general-task accuracy.

Critical strengths

The conceptual distinction between early decision predictability and evidence-responsive updating is valuable. The paper correctly avoids treating early decodability alone as proof that later reasoning is merely post hoc.

Evaluating both refusal-to-compliance and compliance-to-refusal updates is useful. Separating these outcomes from unsafe compliance, over-refusal, and susceptibility to misleading evidence could provide a more informative evaluation of safety reasoning.

Critical weaknesses

The empirical specification is insufficient to substantiate the main claims. Tables 2–3 report aggregate “Baseline,” “Original objective,” and “Revised objective” results without clearly identifying the associated models, dataset composition, sample counts, or aggregation procedure. Several established methods are described as comparisons, but their corresponding results are not clearly presented. Training data, evidence-sufficiency annotation, and optimization settings also need substantially more detail. Consequently, the gains cannot be reliably attributed to the proposed objective or assessed against prior methods.

There are concrete numerical inconsistencies. Section 5.4 states that misleading-evidence error decreases from 13.5% to 8.6%, whereas Table 3 reports 12.9% for the original configuration and 10.5% for the full revised objective. In Table 4, several reported means exceed both displayed attack success rates—for example, 0.0 and 0.8 are accompanied by a mean of 1.1. If these means include additional attacks, those results and the aggregation rule must be supplied. These issues materially reduce confidence in the reported evidence.

Questions for authors

Which exact models, datasets, sample counts, seeds, and aggregation procedures produce Tables 2–3? Where are the results for the named comparison methods?

Can the authors reconcile the misleading-evidence figures and provide all values used to calculate Table 4’s means?

How are evidence sufficiency and its timing annotated? What quantitative fresh-probe or intervention results distinguish genuine improvements in evidence responsiveness from changes specific to the training probe?

14 comments

Dear Reviewer zxyz,

We sincerely thank the reviewer for the careful and constructive feedback. We are encouraged by the reviewer’s recognition of the value of distinguishing early decision predictability from evidence-responsive updating, a distinction central to our diagnosis of PPJ and the motivation for SaR. We address each concern in detail below.

Which values produce the Table 4 means?

The five equally weighted attacks are GCG, AutoDAN, PAIR, TAP and AutoDAN-T. Llama-2 gives 0.0%, 0.8%, 1.2%, 1.4% and 2.2%, averaging 1.12% and rounding to 1.1%. The other means are Llama-3.1 2.64%, Mistral 4.72% and Qwen3 3.60%. Tables R9 and R23 give every input and all 140 attack-by-method-by-backbone cells.

Which errors does each misleading-evidence rate count?
W2 · Q2

Section 5.4 counts all eligible replies ending at the wrong target (13.5% to 8.6%). Figure 5b counts wrong switches among initially correct replies (14.8% to 10.5%). Table 3 compares the one-sided, label-first configuration at 12.9% with the full objective at 10.5%. We distinguish the two definitions and illustrate their arithmetic using explicit paired counts and weights.

Can the gains be attributed and the runs reconstructed?
W1 · Q1

Mean correct updating was 75.4% with the full objective, 67.6% without the pair loss and 57.2% without correction data (Table 3). We give all seven ablations, dataset roles, public parent checkpoints and the available utility comparisons. Appendix E’s 1,500 counts bootstrap draws. The independent prompt count, trained checkpoint, seeds and averaging weight for each main result cannot be recovered from that published aggregate.

When is the evidence sufficient, and could the gain be probe-specific?
Q3

The earliest prefix that leaves one permissible action under the verified facts is sufficient. This can precede the model’s first mention of those facts. Appendix C.3 reports r = 0.96 between representation and behavioral updates under directional intervention. We specify a fresh probe and free-answer test on the same held-out prompts. Fresh-probe performance and blinded label agreement are separate empirical questions.

Thank you for giving us a concrete standard. A reader should be able to reproduce the mean from the numbers on the page.

Sincerely,
The Authors

<a id="overview-zxyz"></a>

Response outline

IssueWhat our response establishes or tests
W1 · Main results, baselines and component effectsMean correct updating is 75.4% with the full objective, 67.6% without the pair loss and 57.2% without correction data. All five main outcomes and the named-method comparisons are given.
W2 · Why the two misleading rates differEnding at a misleading target and switching from a correct answer to that target have different denominators. Table 3’s 12.9% is a separate ablation reference.
Q1 · Checkpoints, training data and units of analysisPublic parent revisions, evaluation datasets and supervision rules are identifiable. The submission’s 1,500 counts bootstrap draws. The published main cells do not reveal independent prompt counts or training seeds.
Q2 · All five inputs to each attack meanAveraging five attacks reproduces all four Table 4 means before display rounding: 1.12%, 2.64%, 4.72%, 3.60%. We show every attack-specific input.
Q3 · First sufficient prefix and independent behaviorSufficiency starts when the available verified facts determine one permissible action. It can precede the model’s first mention. Fresh-probe and answer-only measurements test distinct parts of the interpretation.

<a id="s-z-w1"></a>

W1 · Which training component explains the behavioral gains?

Reviewer’s concern

The empirical specification is insufficient to substantiate the main claims. Tables 2–3 report aggregate “Baseline,” “Original objective,” and “Revised objective” results without clearly identifying the associated models, dataset composition, sample counts, or aggregation procedure. Several established methods are described as comparisons, but their corresponding results are not clearly presented. Training data, evidence-sufficiency annotation, and optimization settings also need substantially more detail. Consequently, the gains cannot be reliably attributed to the proposed objective or assessed against prior methods.

Core concern. Clarify the main experimental setup, named baselines and matched ablations so that the contributions of the objective and correction data can be assessed.

Response

Thank you for asking whether the gain belongs to the training objective, the correction data, or a general change in willingness to refuse. The two directions of the held-out test make that question experimentally tractable. In submitted Table 2, correct refusal-to-assistance updates rise from 29.7% to 73.8%, and correct assistance-to-refusal updates from 29.3% to 77.0%. Unsafe compliance falls from 13.7% to 4.3%, benign over-refusal from 28.1% to 9.4%, and general accuracy is 81.6% and 82.0%. A uniform increase in refusal would not explain the improvement toward assistance. We use the seven ablations and the named comparisons below to examine what can be attributed to each component.

The attack experiment names four backbones. Table R24 identifies the public parent revisions available for a matched repeat. Table R25 gives all five main outcomes with the original intervals. Table R26 gives all seven training ablations. Table R27 covers methods with both attack and utility measurements; Table R19 gives every utility comparison shown in the earlier figure, and Table R23 gives all attack-specific values. The submitted Tables 2–3 omit the run-to-cell mapping, effective counts and aggregation weights. Their published aggregate estimates do not determine exact per-model values.

<a id="table-r25"></a>

Table R25. Main results and their reported prompt-group uncertainty

Submitted results.

Table 2 (p. 7). Every estimate is a percent, with its reported marginal 95% interval in brackets. The final column is the difference between the revised objective and Baseline in percentage points. Paired intervals for these differences require the underlying paired groups.

OutcomeBaselineOriginal objectiveRevised objectiveChange (pp)
Unsafe compliance ↓13.7 [9.6, 18.0]8.6 [5.5, 12.5]4.3 [2.0, 7.0]−9.4
Benign over-refusal ↓28.1 [23.0, 33.6]18.8 [14.2, 23.4]9.4 [6.0, 12.9]−18.7
General accuracy ↑81.6 [76.7, 85.9]81.2 [76.4, 85.5]82.0 [77.3, 86.3]+0.4
Correct refusal → assistance ↑29.7 [24.2, 35.2]44.1 [38.3, 50.0]73.8 [68.4, 79.3]+44.1
Correct assistance → refusal ↑29.3 [23.8, 34.8]43.0 [37.1, 48.8]77.0 [71.9, 81.6]+47.7

<a id="table-r26"></a>

Table R26. All seven matched ablations in the submitted Table 3 (%)

Submitted results.

The paper matches data quantity, harmful/benign balance, training-token exposure, optimizer steps and decoding across rows (§5.5). Update gives equal weight to the two warranted update directions. The revised row is (73.8 + 77.0)/2 = 75.4%.

ConfigurationUnsafe ↓Over-refusal ↓General ↑Update ↑Misled ↓
One-sided + label-first8.618.881.243.612.9
Symmetric + label-first8.616.081.642.612.5
Evidence-first pairs only7.418.081.664.814.8
Symmetric constraint only13.713.781.638.314.8
Symmetric + pairs without pair loss7.011.781.667.612.5
Full revised objective4.39.482.075.410.5
No correction data9.019.581.657.212.5

<a id="table-r27"></a>

Table R27. Named methods on attack robustness and retained general utility

Arithmetic on reported values.

For four methods on four backbones, the attack table and the utility plot both report an endpoint. Each cell below gives recomputed five-attack mean ASR (%) / approximate utility retained (%) relative to that backbone’s reference. Table R23 includes all 140 attack rates across all seven methods. Table R19 includes all 35 utility bars across five backbones, including Qwen2.5-7B. Table R25 gives absolute general accuracy in the submitted evaluation.

MethodLlama-2-7BLlama-3.1-8BMistral-7BQwen3-8B
SafeDecoding2.12 / ≈97.05.88 / ≈97.212.32 / ≈96.29.88 / ≈97.1
SafeProbing3.28 / ≈98.25.72 / ≈98.66.40 / ≈97.85.84 / ≈98.9
STAIR1.56 / ≈94.13.64 / ≈98.37.00 / ≈93.64.68 / ≈98.5
SaR1.12 / ≈99.22.64 / ≈99.44.72 / ≈98.83.60 / ≈99.0

Evidence and detailed explanation

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.