- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper assesses whether a model's answer with a given persona is consistent across different training checkpoints (e.g., base, IFT, post-train). Models are prompted using a set of attributes from a different paper and then the differences from the base model are compared using some paper-defined metrics. A held out analysis on OpinionQA suggests that persona-sensitive models do worse.
Critical strengths
- Experiments used multiple prompts and multiple models (though some of these are more dated), both of which could support a more general finding. Given the broad interest in persona prompting, this scale is helpful for ensure results are not model or persona specific.
Critical weaknesses
-
The paper does not do enough to contextualize its approach with respect to related work. Persona prompting have several years of work now and the paper only lists a few examples in Section 7 and later in Appendix K. None of these discussions really helped me understand what new gap this paper was addressing or how the current experiments relate to what is already known. A good example of this is the rewording experiment in 4.2, which is an already-established general result from multiple papers (models are sensitive to changes to prompt wording). What new is being learned here?
-
I had a very hard time following the paper, despite knowing this subarea well. I struggled to answer some of the basic questions about what types of personas were used (e.g., what attributes), what exactly is being asked, and how some ground truth is measured. These answers are buried in the appendix, but are basic setup, so I would want them earlier. The writing style itself is also hard to follow. Sections 2.1 and 2.2 are particularly tough examples where new metrics are introduced but it's not clear why we need to use these metrics instead of those from prior work. As a result, they feel more like unnecessarily added complexity.
-
This is related to the writing, but I genuinely am still not sure what we learned from all these experiments. I understand that the paper is trying to model the relationship between model training and persona consistency/fidelity, but why would this matter in practice? Is the motivation that we should use models from an earlier training stage to get better prompting behavior? The paper presents a large number of analyses and robustness tests, but, in part due the presentation, it's surprisingly tough to tell what we've learned from all of the studies, or what the impact of the results should be on future work.
-
One aspect I struggled with was whether these differences are meaningful. The authors note that some of the differences are within the scale that switching from bf16 to fp32 changes the results (line 660). Further, scale of the log-odds in C.4 suggests that the differences are small in their effect. Given that the focus for prompting here is on persona prompting, I would have wanted to see more results like in Section 6 that assess accuracy with respect to known group variation.
Questions for authors
- All of the persona results are pooled together, which is helpful for visual clarity. However, I'm curious whether you saw any category-level differences that were interesting.