- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper addresses the topic of whether LLM behavior and self report can be separately manipulated. They use 5 personality traits and 10 models (the design was preregistered). They steer with item and persona vectors and measure behavior (avg log-prob of 2 responses) and self report. They find that persona and item vectors affect behavior and self report respectively, and conclude that this supports the idea of separate representations for behavior and self report.
This is an important topic and this study adopts an interesting approach. A strength of the approach is the wide model coverage and pre-registration. However, I find it difficult to know how to interpret the findings without more details about the methodology which i found to be very unclear (see below)
Critical strengths
S1: This is an important topic and the authors use an interesting design S2: a large number of models is tested S3: the study is preregistered
Critical weaknesses
W1: The methodology is insufficiently clear. Could the authors clarify how exactly the vectors are constructed? This is especially an issue for construction of the item vector: the model is given opposing statements. Is the model's generated response included? Related to this: I do not understand why a vector of the item differences would reflect a representation of a model trait -- rather than semantic differences between the sentences, and also whether a vector would also capture if the model disagrees with either sentence. W2: There is also an identification problem: it is shown that different interventions affect behavior and self report differently but there are several differences between the construction of the item and persona vector --- one is in the first person (item) vs second person (persona). But they differ in whether they are in the system prompt or the main context, and in the content of the messages. Hence it is difficult to know exactly which manipulation is responsible for producing the differences. In particular, I don't think necessarily follows that separate representations must involved.
Questions for authors
Q1: Can the authors clarify exactly how the steering vectors are constructed? Q2: Can the authors clarify their position on why the constructed vectors reflect the factor they focus on rather than other factors? In particular, why couldn't the differences in steering results be due to one being in the system prompt and the other being in context -- or content variations between the messages. If I understand correctly, the authors focus on the first-person vs second-person nature of the item vs persona vectors, but aren't there other differences?