- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a functional framework for modeling affect expressed by large language models over multi-turn conversations. Affect is represented in the valence-arousal-dominance (VAD) space, and each model is treated as a transition system in which the affect of the current response depends on both the current user input and the model’s previous response. The authors construct 24 five-turn semantic scenarios, generate controlled affective rewrites, and evaluate Claude Sonnet 5, GPT-5.6 Terra, and Grok 4.3, with an additional within-family analysis of GPT-5.6 variants. Linear models are compared with GAM and XGBoost alternatives, and first- and second-order response histories are examined. The principal empirical finding is that current user affect is more predictive than previous-response affect alone, while adding the previous response provides a smaller but generally positive improvement in held-out predictive performance. The estimated same-dimension history coefficients are positive, nonlinear models provide only modest additional gains, and a second response lag contributes a smaller residual signal. The paper interprets these findings as evidence that affect expressed in LLM conversations can be usefully modeled as a short-memory dynamical process.
Critical strengths
The paper introduces a clear and interpretable framework for studying multi-turn affective behavior. Separating the predictive contributions of current input and previous model output is conceptually useful, and the coefficient matrices provide a readable summary of within- and cross-dimensional associations.
The experimental design is unusually systematic. It includes controlled stimulus construction, a semantic scenario-level DEV/TEST split, grouped model selection, matched-row comparisons between history orders, and scenario-level bootstrap intervals. The authors also examine nonlinear specifications, second-order history, and within-family variation rather than relying on a single model class or regression specification.
The paper is transparent that it measures affect expressed in text rather than an internal emotional state. Its appendices contain substantial implementation and robustness information, and the reported trajectory and transition counts are internally consistent. These features make the study comparatively easy to inspect and reproduce.
Critical weaknesses
All central variables are measured using the same PERT-EmoPair scorer: candidate stimuli are selected using PERT scores, while current input affect, lagged response affect, and current output affect are also operationalized through PERT. No independent scorer or human annotation is used to validate the selected stimuli or the generated responses. This does not make the analysis circular, but it makes the findings dependent on a single measurement system. Positive same-dimension coefficients may partly reflect scorer-specific sensitivity to lexical or stylistic continuity. The experiments establish temporal predictability in PERT-derived scores, but do not yet show that the reported dynamics are robust across alternative measurements of expressed affect. This concern is particularly important for dominance, which the paper itself recognizes as difficult to measure reliably.
The lagged response score is also endogenous to the preceding conversation. It may encode previous user content, trajectory family, discourse strategy, lexical continuity, and semantic carryover in addition to affect. The scenario-by-turn fixed-effect analysis is useful, but it addresses stable scenario-position effects rather than isolating the affective realization of the previous response from its semantic and stylistic content. Thus, the results demonstrate incremental predictive value of a lagged response score, but do not by themselves establish that recent model-expressed affect causally shapes the next response. A branching intervention that varies the affective realization of the previous assistant response while approximately preserving its semantic content would provide stronger evidence for this interpretation.
Finally, the uncertainty estimates are conditional on a relatively narrow experimental realization. Each condition appears to be generated once, inference configurations differ across providers, and the held-out evaluation contains only six semantic scenarios. The bootstrap captures variation across the observed scenarios but not generation stochasticity, uncertainty from stimulus generation and selection, or scorer uncertainty. The generated rewrites are also not independently evaluated for semantic preservation or selective manipulation of the intended VAD dimension. These limitations do not invalidate the reported predictive comparisons, but they narrow the scope of the generalization and mechanistic claims.
Questions for authors
Do the main held-out performance contrasts, diagonal history coefficients, and second-lag gains remain consistent when responses are evaluated using an independent VAD scorer or blinded human annotations?
Could a branching experiment hold the current user turn fixed while manipulating only the affective realization of the preceding assistant response? This would help distinguish affective persistence from general conversational carryover.
What proportion of the generated rewrites preserve the underlying proposition and selectively modify the intended VAD dimension according to independent judgment?
Do the main conclusions remain stable under repeated generations for a representative subset of trajectories?
2 comments
Thanks to the author for adding the experiment. For now, I've decided to keep my score.
Thank you for the detailed and actionable review. We added analyses addressing the measurement, semantic-fidelity, and intervention concerns and revised the paper to separate predictive dependence from causal affect propagation.
Independent measurement.
We re-scored the complete frozen primary dataset with the independently trained Park et al. (2021) VAD scorer. Adding previous-response VAD to current-user VAD produced positive held-out V+A gains for all three subject models: Sonnet 5 .0572 [.0320,.0808], Terra .0505 [.0085,.0914], and Grok 4.3 .0238 [.0055,.0402]. This establishes that the aggregate history gain is not unique to PERT.
The independent analysis also revealed an important limitation. PERT/Park response-arousal agreement was essentially zero (Pearson .015; Spearman -.009), and all three Park A→A intervals included zero. We therefore no longer make a scorer-independent arousal-persistence claim.
We additionally ran crossed-scorer analyses in which the current response/input and previous-response history were measured by different scorers. Transfer was strongest for Sonnet, mixed for Terra, and uncertain for Grok. We report these rather than presenting the independent scorer as uniformly confirmatory.
Context/endogeneity.
After controlling for both current and previous user VAD, the history-gain point estimate remained positive for every model, but interval support became model-dependent. Scenario×turn fixed effects nevertheless left all nine original PERT same-dimension coefficients positive with intervals above zero. These results support predictive dependence but do not identify an affect-specific causal mechanism.
Semantic preservation.
A post-freeze event-content audit covered all 2,931 selected original/rewrite pairs and all 6,480 stimulus uses. We repeated the analysis after restricting to PASS stimuli. Under the stricter QC-B restriction, all dimension-specific history-gain point estimates remained positive, but the retained population was non-random: 59.8% of valence-, 43.4% of arousal-, and only 28.3% of dominance-manipulated transitions were retained. We therefore use this only as a sensitivity analysis and do not treat PASS as semantic ground truth.
Branching intervention.
We also performed the suggested intervention. The conversational prefix and next user turn were held fixed while the preceding assistant response was rewritten to LOW/MID/HIGH valence. Branches were accepted before outcome generation only if they passed semantic checks and LOW<MID<HIGH under both PERT and Park; three independent downstream generations were collected per retained condition. Only 11/72 scenario-model units survived these strict checks. The pooled HIGH−LOW effect was -.0693 [-.1526,.0309] under PERT and -.0790 [-.2471,.0953] under Park. Thus the intervention provides no positive experimental evidence for valence carryover, and we revised the paper accordingly.
We did not rerun the entire original experiment with repeated generations; the new branching experiment includes three replicates per retained condition, while the original one-generation-per-condition limitation is now stated explicitly.
The revised claim is therefore intentionally narrower: previous-response affect scores contain reproducible short-horizon predictive information, but the evidence does not establish causal affect propagation or universal dimension-specific persistence.