- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates whether chain-of-thought (CoT) monitorability can be improved through post-training. The authors propose an unsupervised training objective that maximizes the mutual information between CoT and model outputs, combined with a lower-tail optimization strategy and conditional divergence regularization. The proposed method aims to improve the informativeness of CoT while preserving model behavior. Experiments across multiple models and benchmarks demonstrate improvements in recovering degraded monitorability, predicting unsafe behavior, and increasing the prevalence of consequential reasoning.
Critical strengths
1.The paper addresses an important and timely research problem by exploring how to improve the monitorability of chain-of-thought reasoning.
2.The proposed method is interesting, combining mutual information maximization with lower-tail optimization and conditional divergence regularization to improve CoT monitorability.
3.The formulation of CoT monitorability as a representation transfer problem provides an insightful perspective on understanding and improving reasoning transparency.
4.The experiments cover multiple models, benchmarks, and monitoring scenarios, providing relatively comprehensive empirical evidence for the effectiveness of the proposed method.
5.The paper is generally well organized, and the experimental analyses and ablation studies help clarify the contributions of different components.
Critical weaknesses
1.The connection between mutual information maximization and actual CoT monitorability is not fully established. Higher mutual information does not necessarily guarantee that the reasoning is interpretable or faithful to the model's internal decision-making process.
2.The proposed method often increases the length of generated CoTs, which may partially explain the improvements in monitorability. More controlled comparisons under matched reasoning lengths would strengthen the conclusions.
3.Some improvements in monitorability are accompanied by substantial changes in model behavior. It remains unclear to what extent the gains come from more informative CoTs rather than changes in the distribution of model outputs.
4.The evaluation relies partly on LLM-based judgments of monitorability, which may introduce evaluation bias. Additional human evaluation or independent monitoring methods would provide stronger evidence.
5.Although the method is evaluated across several settings, most monitorability failures are constructed through controlled training procedures. Its effectiveness against naturally occurring or adaptive monitoring failures remains unclear.
Questions for authors
see weakness.
Response to Reviewer fSHD
We thank you for your thoughtful review and constructive suggestions. We have added analyses of CoT faithfulness, controls for length and changes in behavior, and evaluations with additional LLM judges to address your concerns.
The connection between mutual information maximization and actual CoT monitorability
In our view, CoT monitorability can fail at three levels:
We target the first failure mode. Our goal is to ensure that CoT retains the required information. We believe this is particularly important to study given the rise of no-CoT capabilities in the LLM (see Section 3.4.5.3 of the Gemini Argon report [2]) where LLMs may completely omit the information from their CoT rendering CoT monitorability infeasible.
We have clarified this point in the introduction. We have also revised the Conclusion section of the paper.
Higher MI and faithfulness: In our new analysis in Appendix I, we show that MI training directly improves causal effect of CoT on model outputs in some of our settings, and overall improves consistency between CoT and the output. Based on this, we cautiously claim that MI training does improve faithfulness of CoT as well. We also observe increased “faithfulness” anecdotally, e.g., on MBPP, CoTs describe reward hacks after MI training.
Length
Please see our common response, which addresses this concern through length-reward baselines and comparisons restricted to shared CoT length ranges (Appendix I.1). In general, we find that length inflation does not explain our monitorability gains. As a concrete example, on all three safety benchmarks, the monitorability gains come with shorter CoTs, while the length-reward baseline produces longer CoTs but performs worse than MI training.
Gains mostly come from more informative CoT
We agree with the reviewer’s concern that changes in behavior are a potential confounder. However, the paper now provides several pieces of evidence that these changes do not fully explain the monitorability gains:
The newly included analyses in Appendix I make us considerably more confident that this objective produces robust and consistent improvements in monitorability across settings.
Additional evaluation of monitoring gains under different LLM judges
We have added an evaluation of monitoring gains on GPQA-diamond under different LLM judges. Monitorability gains hold across all three judges for both mean and worst-of-four monitorability (Table 6). For safety benchmarks, we evaluate monitorability using two monitors and activation probes, all of which show improvements (Table 3). Some evaluations also use changes in answer probabilities and intervention-based metrics (Table 4).
Values show MI-recovered minus compressed monitorability on GPQA-Diamond.
We hope these revisions and additional results address your concerns. If they do, we would appreciate your consideration to revise your score upward.
References
[1] Baker et al. (2025). https://arxiv.org/abs/2503.11926
[2] Gemini Argon report, §3.4.5.3. https://storage.googleapis.com/deepmind-media/gemini/gemini_4_safety_report.pdf