- Rating
- 1
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
The paper analyzes how one can predict the alignment towards a value during post-training. The paper introduces developmental cognitive interpretability (DCI), which treats behaviour as a latent state through a cognitive model. Training Qwen3.5-9B with DPO toward individual values, the authors find that a per-value score model predicts held-out preferences, and training-induced score changes are low-dimensional. A developmental model fitted only to single-value training also predicts values after multi-stage pipelines. The results suggest that models of value-generalisation dynamics can predict the effects of narrow alignment training.
Critical strengths
- The motivation is clear and highly relevant. The effects of value alignment are currently quite opaque.
- The general findings of this work, such as that training-induced value shifts are low-dimensional, are interesting and can inform model post-training/value alignment.
Critical weaknesses
-
The paper is not self-contained and is hard to follow. The core parts of the framework (cognitive model, cognitive state, developmental rule, agent) are introduced informally in Section 2, without formal definitions or examples. “Agent” is defined only in Section 3, and the formal definitions appear only in Appendices B.1 and C.1. The main text does not describe how value scores are estimated, and it refers to Hua et al. (2026), with the fitting objective given only in Appendix B.3. Section 4.3 introduces w, ϕ, C, and D in passing, without saying how the latent-Gaussian model is fitted. Figure 1 does not clarify the pipeline, and methodology is interleaved with results throughout Sections 4–6. This is important because the paper’s primary contribution is its framework.
-
The measured values are not validated beyond an LLM judge. All preference rates come from Haiku 4.5 grades on a 0–6 rubric, without examples for every grade. The scenario generator comes from the same model family as the judge. There is no human evaluation. Without evidence that these scores track how humans perceive the responses’ value expression, it is unclear whether DCI models value learning or judge behaviour. This limits the significance of the paper’s alignment-relevant conclusions. The concern is also about the narrow scope of a single LLM with one training run per pipeline.
Questions for authors
- How is a cognitive model shared across agents? Motivations can lead to different behaviours for different agents, unless I don’t understand your definition of an agent (which is not given in the text until this point).
- Have you experimented with a narrower rubric scale, for example, 0–2 or a binary choice? The examples in the fig are relatively opinionated but get scores 2 and 4. What do responses graded 1 or 5 look like?