- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
Persona prompting is widely used to construct user simulations with large language models (LLMs). To evaluate the effect of one specific attribute, the other attributes should stay unchanged. That is, the joint distribution of other attributes is the same before and after we change the target attribute. However, when LLMs are prompted for persona following, it is observed that changing one attribute in the prompt also implicitly changes other unspecified, non-target attributes, whose distribution is supposed to stay the same. For example, describing a user being more risk-seeking in a prompt changes LLMs preference of color during persona following. The authors define this behaviour as cross-attribute influence.
The authors explains this cross-attribute influence as a consequence of LLM treating a persona prompt as evidence about the user rather than an intervention on a single attribute. This conclusion is drawn from experiments on semantic controls, in-context manipulations and interventions on internal representations. The authors then test whether explictly specifying non-target attributes restores selective control. They report that non-target attributes are more likely to be affected when specified as neutral than that of directional, which is defined as neutrality gap.
Critical strengths
- Neutrality gap. The paper presents an interesting finding on persona following of LLMs, called neutrality gap. When a non-target attribute is declared as neutral, it is more likely to be affected than directional attribute.
Critical weaknesses
Weakness 1: Insufficient evidence to support the explanation of cross-attribute influence. The authors explain cross-attribute influence as a consequence of LLMs treating a persona prompt as evidence about the user and inferring other unspecified attributes. However, the experiments do not provide sufficient evidence to support this explanation. The experiments on semantic controls and in-context correlations only show that the non-target responses are affected by the meaning of the target attribute and the correlations provided in the prompt. This does not necessarily mean that LLMs infer other unspecified user attributes. The observed behaviour could also be explained by LLMs directly associating the target attribute with certain answers when making decisions, without inferring other user attributes (Lines 234–272).
The authors further investigate the internal coupling between target and non-target responses by erasing certain subspaces of the hidden representations. They observe that removing a subspace associated with non-target attributes also affects the target responses. However, this does not necessarily indicate that the model internally infers other attributes from the target attribute. The removed subspace could also contain general information related to answer selection. Moreover, these experiments do not explain why neutral declarations fail to preserve non-target attributes while directional declarations succeed, which is the main finding of this paper (Lines 274–323).
Weakness 2: The proposed evaluation is not directly connected to the causal validity of LLM user simulations. The paper argues that changing a target attribute should not change the distribution of other non-target attributes, as otherwise the estimated effect of the target attribute could be biased. However, the proposed evaluation mainly tests whether the non-target responses stay unchanged when the target attribute changes, given a fixed value of the non-target attribute. This is different from evaluating the average effect of the target attribute over a population with different non-target attributes. In practice, estimating the effect of one attribute requires averaging over the distribution of other attributes. Although the proposed evaluation can identify changes in non-target responses for a specific user profile, the authors do not show how such changes affect the estimated average effect of the target attribute in LLM user simulations (Lines 43–55, 142–165).
Furthermore, the authors report that directional declarations can largely suppress cross-attribute influence, while neutral declarations cannot. However, specifying a strong directional preference could make the model consistently choose the option corresponding to that preference, regardless of the target attribute. Therefore, a small change in non-target responses does not necessarily mean that selective control is achieved. The authors define target efficacy as an additional requirement, but Figure 4 only reports the sensitivity of non-target responses without showing whether the target effect is preserved at the same time. As a result, it is unclear whether directional declarations actually achieve selective control or simply dominate the model's choices (Lines 153–165, 338–364).
Questions for authors
-
The authors explain cross-attribute influence as LLMs inferring unspecified user attributes based on the target attribute. However, the observed behaviour could also be explained by direct semantic associations between the target attribute and the answer options. Can the authors clarify how the experiments distinguish these two explanations? In particular, how does the internal coupling between target and non-target responses provide evidence for the proposed trait-conditioned completion mechanism? (Lines 234–323)
-
The authors report that directional declarations largely suppress cross-attribute influence. However, a strong directional preference could dominate the model's choices and reduce non-target sensitivity without preserving the intended target effect. Can the authors clarify whether the directional declarations in Figure 4 simultaneously satisfy the requirements of non-target sensitivity and target efficacy? In particular, how much of the original target effect is retained when the non-target attribute is specified with a strong directional preference? (Lines 153–165, 338–364)