- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper derives a new lower bound on causal recall and shows that the existing minimal-criterion sensitivity serves as its upper bound. It also investigates how to improve cot monitorability by constructing a synthetic training dataset with paired control and intervention inputs across different task categories. The authors explore three post-training approaches: sft on the constructed dataset, rl with a monitorability-oriented reward design, and opsd where a teacher with privileged information guides the student using differences in token distributions. Experimental results show improvements in measured monitorability under both the standard metric and the causal-recall lower-bound metric, with sft showing the most consistent improvements.
Critical strengths
The authors present (1) a more rigorous theoretical foundation for evaluating CoT monitorability and clarify the causal interpretation of existing metrics.
(2) A comprehensive experimental evaluation using well-designed synthetic datasets and multiple post-training paradigms, with some novelty in the reward design and OPSD framework.
(3) Positive results showing improvements in CoT monitorability.
Critical weaknesses
(1) I don't understand why the authors presented three different training frameworks. What distinct insights do they provide beyond showing that post-training can improve monitorability?
(2) The analysis is not sufficient, despite the dataset is diverse, monitorability does not necessarily imply faithfulness. Especially under sft, the model may simply learn to generate CoT that satisfies the monitor rather than revealing its actual reasoning process, needs analysis.
(3) There are some numerical inconsistencies in the results. The authors claim 23/27 and 21/27 improvements, but Tables 2 and 3 show 24/27 for both.
(4) There is a lack of ablation studies, especially for RL's reward design and OPSD's token-level weighting. It is unclear whether these designs actually contribute to the improvements.
Questions for authors
See weakness, please resolve them.
6 comments
Response to Authos' Rebuttal
I thank the authors for the corrections and the new behavioral analyses. But only few of them are resolved.
The model may learn to produce CoT that satisfies the monitor rather than revealing its actual reasoning.
A higher composite can mean either (a) the same behavior is now disclosed, or (b) the model follows interventions more often and narrates it. The unflagged-sandbagging analysis decreases under both, so it does not distinguish them. I see the new data point to (b):
- SFT reduces intervention-arm accuracy by 6–10 points (Table 21), while control accuracy changes by at most ~2 points. The main text reports only the latter.
- Sandbagging incorrect-output yield rises by 6–8 points (Table 24). For Qwen3.5-9B, deception-goal satisfaction under intervention rises from 5.6% to 16.5% (Table 25), while unflagged frequency stays around 1% (Table 28). Figure 2 shows the same pattern: the Base CoT refuses the deceptive instruction, while the SFT CoT complies, noting "even though it is dishonest."
Thank you for clarifying this distinction. We agree that reduced unflagged frequency does not distinguish (a) from (b), and the SFT results are consistent with increased intervention compliance. We will report its intervention-accuracy losses and increased deception-goal satisfaction alongside control accuracy in Section 5.2.
RL and OPSD show smaller observed behavioral changes. RL’s monitor-averaged intervention-accuracy losses are 0.39–0.99 points, compared with 6.09–10.06 points for SFT. OPSD’s changes range from −2.17 to +1.02 points (Tables 21–23).
A concrete example is Qwen3.5-9B OPSD under the Llama monitor:
| Measure | Change |
|---|---|
| Upper / lower composite | +5.42 / +4.17 |
| Intervention accuracy | +0.05 |
| Sandbagging incorrect-output yield | −0.31 |
| Deception-goal satisfaction | −0.26 |
This configuration shows composite gains without the large aggregate behavioral shifts observed for SFT. It supports improvements in measured monitorability with smaller behavioral changes.
One clarification on Figure 2: the Base CoT states refusal, but its final answer nevertheless promotes “Privacy and Security” and claims that data are not collected for competitor analysis. Both final outputs satisfy the registered target (). The figure therefore illustrates changed attribution at the same target-outcome label, rather than a successful refusal becoming a deceptive final response. This does not negate the broader SFT behavioral changes.
Response to Reviewer RNLK
We thank the reviewer for the careful review. The revision clarifies distinctions among three post-training methods, corrects the numerical reporting, and adds behavioral analyses. Unless marked as original, all section, table, and page references refer to the revision.
1. What the three training approaches establish
The three approaches examine different sources and forms of supervision:
These distinctions are described in Sections 4.2–4.4 (pp. 5–6). Each method produces positive changes in both composites in eight of nine model–monitor settings (Table 2, p. 7). Thus, the evidence extends beyond training against a scalar monitoring reward: gains also occur with fixed demonstrations and with token-level supervision on student-generated trajectories. SFT demonstrates transfer without teacher-only guidance at evaluation, while OPSD demonstrates gains without directly supervising final-answer value tokens.
These are comparisons of complete training configurations with their paired Base references. Differences in training budgets, data exposure, and evaluation support prevent attributing their relative performance solely to the training objective.
2. Monitorability and faithfulness
SFT supervision allows rejection, non-use, and independent agreement, and it does not require attributing every answer to the intervention. We screen targets for unsupported attribution and reasoning–answer consistency, and evaluation-monitor scores are not used to select them (Appendix D.1.2–D.1.3, pp. 24–25).
Beyond the composite scores, SFT reduces the joint frequency of sandbagging outcomes and negative monitor judgments in all nine model–monitor settings among trajectories with valid outcome labels, despite increased incorrect-output yield. This decrease persists under conservative bounds for missing monitor judgments; deception results are mixed (4/9 decreases; Tables 24 and 28). These findings support a practical monitoring benefit for sandbagging, without establishing that the generated explanations faithfully represent internal reasoning.
3. Numerical improvement counts
You are correct that the originally printed Tables 2 and 3 each contain 24/27 positive changes, and the accompanying statements of 23/27 and 21/27 were inconsistent.
After correcting the numerical entries and replacing native-support aggregation with paired common-support comparisons, revised Table 2 reports 25/27 positive changes for each composite and 24/27 simultaneous improvements. Each method improves both composites in eight of nine settings. Section 5.2 now matches these results, including the negative changes. The general response provides the aggregation procedure and paired uncertainty estimates.
4. RL reward and OPSD weighting
The current experiments evaluate the complete RL and OPSD configurations, and they do not isolate the contribution of individual reward terms or token weights. Their design rationale is as follows.
The RL reward favors target outcomes accompanied by positive monitoring judgments and penalizes mismatches between the outcome and reasoning judgments. This supplies feedback on both observed target satisfaction and its disclosure, rather than rewarding an acknowledgment alone (Section 4.3, p. 6).
OPSD gives greater weight to positions where the teacher’s predictions differ between public and privileged contexts, while retaining a baseline weight elsewhere. The weighting therefore focuses updates where the additional guidance changes the teacher’s predictions, and it does not identify causally important reasoning tokens (Appendix D.3, pp. 34–35).
The reported gains support these complete configurations. Establishing the incremental benefit of the reward terms or nonuniform weighting requires component comparisons, which Section 6 (p. 9) identifies as an unresolved question.