- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a study of the extent to which LLMs generate outputs according to predefined distributions (e.g., uniform integers between 1 and 10). The paper finds that relative to posttrained models, base models better adhere to the requested distribution, but also more often fail at correctly formatting answers. The paper also finds that activation steering does not help address calibration in the context of autoregressive generation.
Critical strengths
To the best of my knowledge, the proposed methodology and set of experiments are technically sound. Though, this is hard to entirely verify given the lack of methodological details given for activation steering.
Critical weaknesses
- The paper and experimental methodology is substantially AI drafted, which impairs readability. For example, much of the appendix is an AI experiment log (Appendices E and F, table 18, etc).
- The novelty of the paper is more limited than the claims. In contributions (1) and (2):
1 … DART, an audit that scores completed answers exactly over enumerated candidate sets, separately measuring whether a model attempts the task and how closely its choice follows the requested law.
This type of audit is common in the literature (e.g., West & Potts, 2025 evaluate random number generation as well).
2 … We trace the distortion across the OLMo-2 post-training pipeline and show that it is already detectable at SFT and deepens at later stages on selected probes.
West & Potts 2025 also evaluate across the stages of the Tulu post-training pipeline (DPO, SFT, and the full pipeline), with similar findings.
I am not aware of prior work that explicitly contributes (3) and (4).
- The paper provides little to no methodological detail for how activation steering is done in contribution (4)
Questions for authors
- What is the claimed novelty for contributions (1) and (2) relative to prior work like West & Potts?
- How was the activation steering done?