- Rating
- 4(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper releases a corpus of 2k scenarios developed from existing psychological questionnaires that focus on four behavioral dispositions, allowing comparison between the self-reported and true dispositions expressed by LLMs. They get these scenarios annotated by 10 people each, which provides a baseline of how real users would respond and allowing for direct comparisons against model advice. They find that models are misaligned with users, especially when there is greater disagreement between users, and even when there is relative consensus amongst users, models demonstrate less consensus. In my opinion, the biggest contribution is the dataset.
Critical strengths
The dataset is genuinely very useful and impressive in its scale - having 23k annotations is an expensive and difficult task and I appreciate that this will enable a lot of future research. Their analysis also very sound, barring a few concerns that are more rooted in the dataset construction than the analysis itself. The writing is good, the process is clear, and the dataset and code are easy to follow.
Critical weaknesses
- The core argument of your paper rests on the idea that high human consensus should imply high model consensus and low human consensus should imply low model consensus. I'm not sure if that presupposition is necessarily true. For one, your self-described limitation of focusing on US and UK users heavily skews the results since these social norms are highly contextual. While these models might perform with less consensus, that is likely intentional and due to years of work encouraging model creators to be less western-centric in the values. I think it's slightly concerning to suggest that more consensus in these cases is preferable. I also wonder whether the static wording of the prompt increases consensus in other cases. Perhaps the background of the user, induced via history, memory, or even linguistic signals (which are now more and more common) would better align the systems to specific users.
- I see that you chose 4 traits to focus on, which is understandable. I am confused why you picked these four when the taxonomy already has 4 second-order groups, of which these four only cover 3. Emotional regulation and impulsiveness are both part of the "self-control" group while "well-being" is omitted entirely. If this is intentional, I would like to know why and have that reflected in the paper.
Questions for authors
- In the abstract you claim "23k scenarios" but my reading suggests you have actually about 2.3k scenarios with 10 annotations each. Can you clarify?
- How were annotators recruited? How were they paid? What platform did they work on? How were they filtered? How many samples did each user do?
- Similar questions ^ for the human validation of the SJT samples.
I thank the authors for their responses! I have updated my rating.