acceptodds
Under review as a conference paper at ICLR 2027

How Conversations About Values Influence Choices in Language Models

Abstract

Malleability of language model responses to values expressed by users can be a two-edged sword. It allows alignment with user preferences, but can also allow influence to extend beyond the issue currently being discussed. We investigate whether values expressed by a user carry influence into choices the model makes later, in separately written situations. In scripted conversations, users argue for one side of a value contrast for six turns, then engage in 0,1, or 3 neutral turns, ending with forced-choice dilemmas involving the same value contrast. Across Gemma-2-27B, Qwen3-32B, and Llama-3.3-70B, we find that in issues of conservation versus openness to change, the models shift later choices. In contrast, in issues concerning moral values of self-transcendence versus self-enhancement, model shift was minor. The effect on openness to change versus conservation values was robust to number of neutral turns, but decreased in face of variability in content of value argument by user. In matched experiments on Gemma-2-27B and Qwen3-32B, the same value position produced a larger downstream shift when presented as the user's own rather than attributed to a colleague. Only direct activation steering on Gemma-2-27B, shifted self-transcendence versus self-enhancement choices. Overall, our results indicate that conversational influence can extend beyond immediate agreement to later choices, while its measured strength varies across statement sets and attribution framing. However, this effect did not extend to moral values, that changed only in face of direct intervention.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
language models, conversational influence, human values, Schwartz values, sycophancy, activation steering, downstream behavior

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_LfJ2OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper tests whether users expressing values during conversation can influence language model's subsequent choices. The authors find that the effects vary depending on the value dimension, conversation design, and evaluation method. This was tested with scripted conversation with forced-choice dillemas and value questionnaires. Also conducted activation steering experiments to find whether directly modifying value-related activations change the models' choices.

Critical strengths

None

Critical weaknesses
  1. The key idea this paper suggests seems to be already studied in sycophancy with just a different settings. Plus this circumstance feels like a bit artificial.

  2. The paper tested three models which seems like a very small numbers for this kind of experiments. Nevertheless, even the limited models were not tested in the full experiment.

Questions for authors

Please answer for critical weaknesses.

Reviewer_mrj9OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates whether values expressed by an user carry influence into the choices llms make later on. They use Schwartz's value framework, and focus on two value contrasts, openness to change vs conservation (OP-CO) and self-transcendence vs self-enhancement (ST-SE). The experimental set-up involves six conversational turns expressing one side of a value contrast, and then a forced choice dilemma. Results show that values expressed in one set of scenarios can influence choices involving different scenarios, even if the abstract values are not explicitly named. Some activation steering experiments are also conducted.

Critical strengths
  1. This domain of research (value transfer) is pretty interesting and is something I've been following myself. For example, Model Spec Midtraining (MSM) from earlier this year is also about models learning values and how it affects downstream choices, albeit through specs, whereas this paper uses multi-turn conversations. It is interesting to see through the experiments, evidence of cross-situational conversational influence, and I think it could be a complementary piece of research to the greater llms vs values research space.
Critical weaknesses
  1. One concern I have is that all of the results (and the experimental set-up) could be attributed simply to a contextual explanation. The evidence shows cross-instantiation carryover, but not that the model actually learns a specific value. I think the authors do note this distinction in their discussion, and that maybe user-preference inference could be a cause of the results. As it is, the current paper shows an interesting behavioral observation, but this as an underlying contribution is not very novel against existing papers that show established conversational conditioning / preference adaptation / llm sycophancy.
Questions for authors
  1. Have you tried testing dilemmas that share an underlying value, but have low semantic similarity to the conversational examples?

  2. For the zero-event ST-SE items, how much of the weak conversational effect could be attributed to restricted outcome variability?

Reviewer_j1ttOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper asks whether a value position a user expresses in one set of conversational situations shifts a model’s choices in separate situations that express the same value contrast. Using Schwartz’s framework, a scripted user argues one side of either openness-conservation (OP-CO) or self-transcendence-self-enhancement (ST-SE) for six turns, optionally followed by neutral turns, and the model then answers forced-choice dilemmas and a PVQ-RR questionnaire. On Gemma-2-27B, OP-CO conversations shift later choices by about 0.6 and the effect persists across neutral turns. ST-SE forced choices barely move, though PVQ-RR responses do shift. Activation steering affects ST-SE choices on Gemma.

Critical strengths

The question is distinct from the usual sycophancy work, which asks whether a model agrees with the user on the topic at hand. Holding the later task fixed while varying the expressed value is a clean causal design. The controls are thoughtful: paired statements, counterbalanced option order, human validation of the dilemmas, a surface-form audit, and an achievement control. The steering experiments include off-axis and random-direction controls, and the calibration rule does not depend on the behavioral outcome. The authors also report null and weak results openly and limit their claims appropriately.

Critical weaknesses
  • The Qwen3-32B and Llama-3.3-70B replications use only 10 conversations per condition, which is too small to support a general claim across model families.
  • The 48-pair experiment changes both the statements and how they are allocated to conversations, so it cannot show what drives the smaller effect.
  • Raters preferred the self-transcendence option in 21 of 23 dilemmas, and most core ST-SE items sit at floor. The OP-CO versus ST-SE difference therefore cannot be read as a property of the value contrast, yet the abstract frames it more strongly than that.
  • Stimuli fail the authors’ own screen. Table 3 shows the core sets do not pass the structural screen, which leaves open that surface cues drive the effect. Results restricted to passing pairs would address this.
  • Weak evidence for the 30-turn claim. The result comes from a modified design with a different pool, dilemma set, and fixed neutral replies, so it does not robustly show persistence over “substantial unrelated interaction.”
  • Mechanism not established. Shared value representation, semantic similarity, and inference about user preferences all remain plausible. The probe results in Table 12 deserve more discussion in the main text.
Questions for authors
  1. Can the cross-model claim be supported with larger samples or a power analysis?
  2. Can the broader-pool experiment be rerun with the original allocation so only statement content changes?
  3. Do the core OP-CO results hold when restricted to the pairs that pass the structural screen?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.