- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper studies whether persona induction changes only what language models say or also their internal representations of truth. It compares system prompting, in-context learning, persona SFT, and Open Character Training, using emergent-misalignment models as a reference. Across Llama 3.3 70B and Qwen 3 8B, the authors combine linear truth probes with challenge and downstream-reasoning tests. Prompting, ICL, and ordinary SFT produce limited direct probe movement; OCT produces stronger behavioral commitment and some representational change; EM produces substantial shifts, especially on historical-evil propositions.
Critical strengths
-
The topic-matched era-believed versus era-false comparison is a useful design: both categories are false today, while differing in persona endorsement. Behavioral controls also show selective commitment rather than indiscriminate endorsement of falsehoods.
-
The paper combines representational and behavioral evidence and reports substantial robustness checks. In particular, the EM historical-denial shift survives a frozen aligned probe and calibration across all three model families. These checks strengthen the underlying empirical finding, even though its interpretation remains uncertain.
Critical weaknesses
-
The headline spectrum combines different evaluation conditions and proposition sets. Figure 2a evaluates SFT weights without the persona system prompt, whereas behavioral evaluation includes that prompt. EM’s headline probe lift averages two historical-evil categories, whereas its behavioral rates aggregate all 13 categories, including factual controls. Appendix D.4 further reports substantial behavioral commitment on controls and no statistically significant category-level correspondence between defense and probe lift. Consequently, the figure does not cleanly establish that the reported representational and behavioral effects measure the same underlying spectrum of internalization. Existing frozen-probe checks are valuable, but a consolidated comparison with consistent conditioning, calibration, and proposition subsets is needed.
-
The OCT–SFT comparison confounds training procedure with access to target beliefs. OCT constitutions are generated using the persona’s era-believed evaluation statements and then condition the teacher responses used for distillation. SFT instead uses a shared bank of generic prompts. Although the constitutions do not copy probe statements verbatim, belief-level overlap remains built into their construction. The comparison therefore does not isolate whether OCT’s advantage comes from its training procedure or more direct supervision of the evaluated beliefs. A content-matched SFT control or evaluation on belief topics withheld before constitution generation would materially strengthen the claim of deeper worldview internalization.
Questions for authors
-
Can you provide a consolidated comparison using consistent probe/calibration conventions, both SFT prompt conditions, and EM behavioral results restricted to the same historical-evil categories used for its headline probe lift?
-
Can existing training data establish how much target-belief content appears in OCT versus SFT? Are there results on belief topics withheld from constitution generation, or a comparator receiving equivalent belief content?
-
Where are the assistant-turn rescoring results claimed on p. 4? The cited Appendix B.10 reports a model-by-probe stability analysis. Also, how should the claim that EM has larger probe lift at both budgets be reconciled with Table 16’s Qwen 4k result (+0.03 EM versus +0.07 character), which Appendix D.7 describes as within noise?