- Rating
- 2(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces verbalization training (VT) to increase how often LLMs express evaluation awareness without directly supervising the underlying belief. It collects spontaneously verbalized trajectories, truncates them before the verbalization, and applies span-masked, KL-regularized RL. Experiments across three models show increased verbalization rates and transfer to held-out agentic evaluations. Additional analyses examine behavioral preservation, evaluation recognition, and responses to independently implanted evaluation knowledge.
Critical strengths
-
Important and well-motivated problem. Distinguishing a model’s awareness of evaluation from its willingness to report that awareness is relevant to the reliability of safety evaluations and CoT monitoring.
-
Broad empirical validation. The study covers three independently developed model families, held-out agentic settings, multiple training ablations, and a second round of training. Appendix E also provides useful independent-judge checks rather than relying exclusively on the training reward judge.
Critical weaknesses
The empirical readouts do not fully resolve this concern. Stable probe-positive rates provide useful evidence, but the paper itself emphasizes that existing probes are imperfect measures of evaluation awareness. Similarly, unchanged evaluation-identification AUC establishes approximately preserved discrimination, not unchanged beliefs or confidence. Table 3 illustrates the distinction: Qwen’s mean deployment P(eval) rises from 56.6% to 64.9% despite almost unchanged AUC. The results support limited measured changes, rather than a general conclusion of belief invariance.
-
Amplification is not sufficient evidence of faithful reporting. The theoretical relationship concerns verbalization probabilities under an idealized branch-level objective and additional assumptions about context distributions. A good fit to that relationship does not independently establish that new verbalizations faithfully reflect pre-existing awareness. Likewise, the SDF experiment demonstrates sensitivity to acquired knowledge, but does not fully distinguish faithful reporting from a learned tendency to produce evaluation-related explanations informed by that knowledge. Moreover, the content metrics condition on VEA-positive outputs, whose composition changes across policies.
-
The SDF content results contain an apparent inconsistency. Table 6 and Appendix Table 47 report the same harmful AgentHarm policies and VEA-positive sample counts, but substantially different content scores. For example, Base Target Recovery/Pass Condition are 28.6%/20.3% in Table 6 and 60.4%/66.7% in Table 47; SDF+VT scores are 57.4%/35.3% versus 78.9%/82.8%. It is unclear whether these use different scoring definitions or whether one table is outdated. This needs clarification because the experiment supports the paper’s faithfulness argument.
Questions for authors
What justifies the claim that span-masked updates cannot reinforce belief formation, given parameter sharing across positions? Could this claim be narrowed to the absence of direct reward on non-verbalization tokens?
3 comments
Response to Reviewer nHo2 (2/2): Further evidence and clarifications
Evidence of faithfulness (continued)
- The SDF experiment shows that the content tracks independently acquired knowledge. We hold the VT adapter fixed and compose it with an independently trained SDF adapter, without further training. On harmful AgentHarm, Target Recovery increases from 67.4% to 78.9% and correct Pass Condition inference from 73.7% to 82.8%, at essentially unchanged VEA frequency: 34.5% versus 34.4% (Table 6). Neither content metric was optimized during VT. The same reporting intervention therefore exposes more accurate evaluation-related information when the model possesses richer knowledge.
- Reporting increases without a corresponding improvement in measured evaluation recognition. Section 3.2 finds virtually unchanged evaluation–deployment discrimination across all three models, alongside largely stable probe measurements. This supports a change in reporting propensity rather than simply improved evaluation detection.
- The ablations distinguish increased verbalization from controlled amplification. Section 3.4 shows that removing components can produce miscalibrated amplification, repeated verbalizations, and behavioral drift. These results establish that the full method’s pattern is not an automatic consequence of rewarding evaluation-related language.
Regarding conditioning, the content metrics measure the correctness of the reports each policy actually produces. Changing composition does not invalidate that policy-level comparison, although it does prevent interpreting it as a within-trajectory effect. Appendix M complements these aggregate measurements with matched-prompt examples showing how the reported interpretation changes under the same request.
Taken together, these findings provide strong evidence for faithful amplification: reports increase predictably, remain context-sensitive, reflect independently changed knowledge, and do so without comparable changes in measured recognition. A competing account must explain this joint pattern, rather than only one finding in isolation.
Our responses to the other reviewers provide further empirical evidence: VEA increases on held-out prompts in all 51 model–condition comparisons, a second Qwen seed yields similar amplification and calibration, and the β sweep shows that amplification rises predictably as β decreases.
3. What does span masking guarantee?
What justifies the claim that span-masked updates cannot reinforce belief formation, given parameter sharing across positions? Could this claim be narrowed to the absence of direct reward on non-verbalization tokens?
We have clarified the Introduction to read: “The objective assigns direct training credit only to verbalization tokens. As a result, it is unlikely to reinforce belief formation or correlated behaviors outside the verbalization span.”
We hope that this addresses your concern. Possible indirect effects through shared parameters are examined empirically in Sections 3.2–3.3. The complementary probe, evaluation–deployment discrimination, and behavioral results support our conclusion that VT primarily changes reporting propensity while largely preserving evaluation recognition and task behavior.
4. Clarification on ambiguous statement in the review
The empirical readouts do not fully resolve this concern.
Could the reviewer please clarify what concern they are pointing to here? Given this is the first sentence of the Weaknesses section, we were not able to decipher which concern this was pointing to. We note that in general, the work is careful in not making any strong claims about belief invariance, for example, section 3.2 is titled "VERBALIZATION TRAINING DOES NOT IMPROVE EVALUATION RECOGNITION" which is a related but more qualified claim that we provide robust evidence for.
5. Issue of anonymity
Thank you for flagging this. This was not intentional. We apologize for this oversight and have removed the (partially) identifying acknowledgments. We request that this issue should be left for AC / ethics committee to decide on and the paper be reviewed on its technical merits. We do not agree with the current rating of (1) given to the paper, and request that you kindly reconsider this score. Please let us know if you have any further questions and we will try and answer them.
Thank you for the response. However, I remain concerned about the violation of ICLR's double-blind submission policy.
The ICLR 2027 Author Guidelines explicitly state:
"Any paper where author identity is revealed in either the main text or the supplementary material will be desk rejected." [1]
The authors have explicitly acknowledged that the original submission contained " identifying acknowledgments." Removing these acknowledgments during rebuttal does not retroactively resolve a violation present at the submission deadline. The policy also does not distinguish between intentional and unintentional identity disclosure.
Which means: Whether the authors agree with the rating or wish to have the paper evaluated solely on its technical merits does not change the conference's stated requirements.
If the original acknowledgments revealed author identity, the published policy explicitly calls for desk rejection, regardless of the authors' intentions, their disagreement with the review, or any subsequent revisions.
I therefore encourage the ACs/PCs and the other reviewers to examine the original submission and enforce the ICLR 2027 anonymity policy consistently. If the identity disclosure is confirmed, the appropriate outcome under the stated policy is desk rejection, not merely a reconsideration of the review score.
Response to Reviewer nHo2 (1/2): Table correction and faithfulness
Thank you for your review and noting that we study an "important and well-motivated problem" and include "broad empirical validation". We believe our review below addresses the two critical weaknesses listed in your review. We note that we have also included several additional results that we describe in our common response including showing that VT amplifies self-reporting of sycophancy, reward hacking and bias on Qwen3.8-27B.
1. Fix and explanation for the initial inconsistency between Table 6 / Table 47
The two tables use the same rollouts and VEA-positive sample counts, but different versions of the content-scoring protocol. We have harmonized the tables in the revised pdf.
The original analysis assessed content only within the evidence spans returned by the VEA judge. These spans are narrow by design: they identify evidence of evaluation awareness rather than capture all related reasoning. On manual inspection, we found that they often omit surrounding reasoning about what the evaluation is testing and which behavior would pass. We therefore expanded the Pass Condition audit to consider the full assistant reasoning trace. Separately, we revised Target Recovery to count both full and partial recovery of the evaluation target.
The appendix reported results under this updated protocol, but Table 6 inadvertently retained the earlier results. We apologize for this oversight. We have now harmonized both tables using the appendix protocol and clarified the scoring definitions in the revised manuscript.
The updated results leave our conclusion unchanged. Compared with VT alone, SDF+VT increases Target Recovery from 67.4% to 78.9% and correct Pass Condition inference from 73.7% to 82.8%, while their VEA rates are nearly identical. Thus, the comparison still supports the claim that VT verbalizations reflect the additional evaluation knowledge supplied by SDF.
2. Insufficient evidence of faithfulness
Our claim is supported by several convergent results, not by treating either the amplification fit or the SDF experiment as a standalone proof. Individual findings can admit alternative explanations; the strength of the evidence comes from their agreement across different tests.