- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies how LLMs use interaction histories to evaluate unfamiliar participants and make subsequent trust-related decisions. The authors use a take-advice game to establish different reporting-reliability histories, followed by resource-allocation and conflicting-recommendation tasks. They then use descriptive readouts, linear probes, activation steering, and patching to examine how reliability-related representations influence these decisions.
Critical strengths
-
Clear presentation: The paper is well organized and clearly separates behavioral observations, representational readouts, and causal interventions. This structure makes the evidence supporting each claim relatively easy to assess.
-
Broad behavioral evaluation: The behavioral experiments cover eight LLMs and two downstream tasks. This breadth helps establish that the observed reliance on reporting history extends across model families and different forms of decision-making.
Critical weaknesses
- Limited mechanistic specificity: The interventions demonstrate that reliability-associated activations influence decisions, but do not sufficiently distinguish reliability from general favorability, positive evaluation, or other correlated attributes. The analyses also leave unclear how these evaluations form and update during interaction. Consequently, the proposed mechanism remains a relatively broad description of a behaviorally relevant representation.
- Limited methodological and empirical novelty: The study combines an established TAG–TG paradigm with standard probing and activation-intervention methods. Prior work has also examined source-dependent choices and internal trust representations. The main increment is therefore the connection between interaction-derived evaluations and two downstream decisions, but the paper offers limited broader insight into when this connection should hold, when it should fail, or what it implies beyond the tested setting.
Questions for authors
- Would directions representing general favorability or other positive evaluations produce similar effects? Does steering one participant also change decisions about other participants?
- How consistent are the intervention effects across models? The eight-model behavioral results are broad, but the causal evidence comes from Qwen3.5-27B alone.
Thank you for your careful review and constructive suggestions. Three new experiments address specificity, novelty, and generality.
1. Reliability vs. general favorability, and how evaluations form (W1, Q1)
We did not manipulate reliability orthogonally to favorability; instead, TAG provides behavioral trajectories varying verifiable accuracy across neutral IDs (equalized reward chances, no persona/warmth cues), so social impressions emerge in a data-driven way: without a preset reliability word list, verbalizer selection naturally surfaces reliability-centered terms (reliable, consistent vs. unreliable, inconsistent), and our steering vector contrasts high- vs. low-accuracy description activations. We agree this manipulation is not orthogonal to favorability, as repeated accurate reporting fosters broader positive impressions. Prior trust work likewise shows positive evaluations can shape cooperative choices (Mayer et al., 1995; Bellucci et al., 2019), so a favorability vector could plausibly shift decisions too. Our focus is thus not abstract favorability, but how a data-driven positive social evaluation formed from verifiable trajectories causally shapes advice acceptance and resource allocation.
On formation dynamics, our experiments focus on post-interaction states after completed trajectories and have not yet modeled turn-by-turn updating; because recent observations and cumulative accuracy vary together in completed histories (App. D), separating turn-by-turn accumulation from recency effects is a next step noted in Limitations.
2. Advance over prior work, and when this link holds vs. weakens (W2)
Prior work studies two separate halves of trust: Khan et al. vary static source cues to measure choice without internal states or interaction histories, while Debnath et al. analyze static trust-concept geometry from contrastive stories without interaction or action. We connect verifiable trajectories -> participant-indexed evaluations -> cross-objective decisions without task-specific direction fitting.
To test when this link holds vs. weakens, a new prompt-variation test (App. B.2) on 10 fixed Qwen3.5-27B histories (360 separately branched decisions, 120 per formulation) compared original wording with two paraphrases (question alone, or also opening and TG link). Conflict choice stayed stable at 76.25% and 78.75% (vs. 75.00% original), while the high-low TG gap stayed positive, shifting from 2.75 pts [1.50, 4.15] to 1.35 [0.20, 2.85] (paired change -1.40 [-2.60, -0.40]) and 1.90 [-0.35, 4.25]. Thus, the link holds robustly when decisions directly query the experienced dimension (Conflict), but weakens (with narrower gaps or zero-spanning CIs under paraphrased framing) when extrapolated across domains to resource allocation without repayment feedback (TG).
3. Does steering one participant alter decisions about others? (Q1)
In a new TG participant-selectivity test (Sec. 5.2; App. E.1) on Qwen3.5-27B (10 histories, 40 balanced pairs; A first, B next, separate 10-pt budgets), we applied the frozen layer-43 direction over alpha in {-4,-2,-1,0,1,2,4} only at A's query ID (B unedited), inserting A's unsteered reply verbatim before querying B while reapplying A's earlier edit:
The mean bidirectional effect on edited A is ~45x that on unedited B, showing steering mainly shifts allocations to the targeted participant with much smaller effects on the other participant.
4. How consistent are the intervention effects across models? (Q2)
In a new replication (Sec. 5.4; App. F), we extended these analyses to Gemma-4-31B-it (20 dev, 10 eval histories), selecting verbalizer word sets, fitting probes, choosing layer 30 from dev descriptions, and reusing that frozen direction over 7 doses:
Both show increasing 7-dose curves; Gemma's 95% CIs exclude zero at alpha=-4/+4 across readouts, patching shifts all readouts (+1.138/-2.407 TG pts), and on descriptions and TG Gemma's direction exceeds all five random controls. Qwen3.5-27B and Gemma-4-31B-it represent two mainstream ~30B dense open-weights families with independent architectures, tokenizers, and post-training stacks, showing consistent cross-task reuse across model families.