acceptodds
Under review as a conference paper at ICLR 2027

When Situations Take Effect: Psychological Situation Modulation for Personalized Dialogue Generation

Abstract

Personalized dialogue should reflect users’ facts, personality, affect, and situation. Existing situation representations focus on physical and social circumstances, overlooking psychological appraisals. Personality, affect, and memory also lack joint integration in response generation and dedicated evaluation. We propose PSyMod, a Psychological Situation Modulation framework that constructs a frozen persistent state for each user, comprising episodic memory, a personality profile, and a situational reference distilled from past experiences. During generation, psychological situation modulates memory retrieval, personality expression, and response planning, while turn-level affect controls expressive intensity. Our benchmark and evaluation protocol separately assess response generation, state construction, and closed-loop operation. PSyMod outperforms the strongest evaluated condition in ESC-based response quality and situation-modulation consistency, particularly for personality expression and memory use, but scores slightly lower in a blind four-dimensional review. It also maintains competitive retrieval performance without storing hallucinated entries in our evaluation.

Primary area:
foundation or frontier models, including LLMs
Keywords:
personalized dialogue generation, psychological situation, personality, emotion, memory, LLM agents, benchmark

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_bGK9OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes PSyMod, which uses DIAMONDS situation readings to coordinate memory retrieval, personality expression, and response planning, while turn-level affect adjusts expressive intensity. The system constructs memory, a Big Five profile, and a historical situation reference from preceding dialogue. The state remains read-only within each turn and is re-estimated from the observable history on subsequent turns. Using CPED, Friends, and RealPersonaChat, the paper separately evaluates reference-state-conditioned generation, state construction, and closed-loop operation.

Critical strengths
  1. Psychological situation as a guide to memory retrieval is worth studying The paper distinguishes topical similarity from psychological situation similarity and uses the latter to guide memory retrieval across topics and personality expression. This provides a concrete hypothesis about how personalization can go beyond static profiles.
  2. Separate evaluation tracks help identify sources of failure The three tracks for generation, state construction, and closed-loop operation distinguish performance conditioned on a reference state from end-to-end capability. Keeping the state read-only within each turn also helps separate problems in state construction from those in generation.
Critical weaknesses
  1. The novelty relative to closely related work is not sufficiently established

The paper does not discuss two directly relevant studies. AG-CTR² [1] already uses appraisal-guided chains of reasoning to retrieve psychologically similar support cases across topics. HEART-Bench [2] combines Big Five profiles, autobiographical memory, and DIAMONDS situations. When constructing reference answers, it activates memories through structural similarity, emotional resonance, and behavioral scripts, then combines them with personality to generate behavioral responses. These precedents make it necessary to clarify what the proposed joint control mechanism adds and to compare it with appraisal-guided retrieval under matched conditions.

[1] Chu et al. Beyond Semantic Similarity: Appraisal-Guided Chain-of-Thought Reasoning and Retrieval for Multimodal Emotional Support Conversations. Findings of ACL 2026. https://aclanthology.org/2026.findings-acl.844/

[2] Peng et al. HEART-Bench: Do LLM Agents Exhibit Human-like Psychology? arXiv:2605.30058, 2026. https://arxiv.org/abs/2605.30058

  1. Evidence for the distinct benefits of situation modulation is insufficient

In Table 1, prompt-only receives only a short context, while PSyMod uses a reference state derived from a longer history. The difference between them therefore cannot be attributed to situation modulation alone. The paper defines Cond-gold and Cond-gold+mem but does not report their scores, making it difficult to separate the benefits of additional state information from those of the modulation mechanism. (Pages 4, 6, and 27)

In Table 5, removing situation read-out improves all three E/P/M axes and raises pooled L2 from 0.582 to 0.592, while ABCD and ESC decline. This does not rule out a benefit to response quality, but it is not enough to support the claim that the current situation reading improves situational modulation consistency. In Table 1, PSyMod scores lower than prompt-only on ABCD (4.084 vs. 4.170), and its ESC Overall score is only 0.045 higher. Confidence intervals for these differences are not reported. (Pages 7 and 9)

Questions for authors
  1. New mechanisms and matched comparisons

What mechanisms does PSyMod add beyond AG-CTR² and HEART-Bench? Could appraisal-guided retrieval achieve similar gains when the memory bank, generation model, remaining modules, and budget are held constant? Please report the results and information and computational budgets for Cond-gold, Cond-gold+mem, and PSyMod. Please also explain why the full-system scores differ between Table 1 and Tables 4 and 5.

  1. Interpreting the situation ablation

Does the Without situation read-out variant still retain situation information in its situation encoding, retrieval ranking, or personality rules? Please report paired differences and confidence intervals using the same set of valid samples to help explain why the modulation metrics improve after this module is removed.

  1. Annotation reliability and the role of the historical reference

Please provide the threshold used to determine agreement among model annotations, the number of annotations manually reviewed, the ICC values for each dimension, and the ICC type used. Please explain how the historical reference Savg modulates personality-expression strength, as Algorithm 1 only lists it as an input. Please also provide verifiable situation rules, behavioral anchors, and the complete retrieval formula.

5 comments
AuthorsOpenReview ↗

Response to Reviewers: With Thanks for Their Constructive Comments

Response. Thank you for pointing out these two closely related studies. We apologize that the original manuscript did not discuss them sufficiently; we will add them in the revision and explain the relationships in detail in Related Work. AG-CTR² uses reasoning about how the person understands the event to retrieve relevant support cases. We additionally designed retrieval using reasoning text similar to that paper's, and compared it with our method. The controls are set up as follows: we use our psychological-situation appraisal to generate a natural-language reasoning cue and use it for retrieval; the second control disables the situation-based retrieval modulation; the third control uses the AG-CTR²-style cue retrieval. These experiments only concern the retrieval method of AG-CTR².

HEART-Bench combines personality, memory, and situation when constructing reference answers; PSyMod uses these sources to select memories and plan each reply before generating it. It also needs to be additionally pointed out that our method is training-free, whereas the methods in these works involve SFT and RL, so a direct comparison with them is not possible. We will clarify the connections between both works and ours in Related Work. We will explicitly state that these retrieval controls are adaptations of the published mechanisms, and we will not present them as full reproductions of AG-CTR². The main purpose of HEART-Bench is to use the definition of the “psychological situation” to distinguish different occasions and scenes so as to generate more data, whereas we use the psychological situation as a means of memory retrieval.

ConditionABCDL2
Full system (same batch)3.6350.592
Situation-appraisal reasoning-chain query + situation-retrieval modulation3.4330.529
Same chain, situation-retrieval modulation disabled3.4040.515
Length-matched general reasoning-chain query3.4830.498
AuthorsOpenReview ↗

Response to Reviewers: With Thanks for Their Constructive Comments

Weakness 2 — Evidence for the distinct benefits of situation modulation is insufficient

Response. Thank you for carefully checking the experimental results. We agree that the original comparisons cannot isolate the effect of situation modulation on its own. The short-window prompt and PSyMod can use different amounts of historical information, so the score difference between the two cannot be attributed to situation modulation alone. We clarify two setups here. First, the prompt-only input is taken after the retrieval stage: what it receives is the union of the top-K factual-retrieval results and the top-K psychological-situation-retrieval results, rather than the long context fed directly. Second, “without situation read-out” in the original Table 5 means that the historical context retrieved by the psychological situation was not fed into generation; to exclude the contribution of the psychological situation without changing the pipeline, we set the psychological situation estimate entirely to its median (neutral) value, keeping the situation codes of memory entries and their use in retrieval unchanged. We will add the results obtained when the reference state and memories are directly provided, and separately report the ablation that changes the current situation input. The setup of these four conditions: Prompt-only, PSyMod, Cond-gold, and Cond-gold+mem were rated together in the same 15-arm anonymous joint evaluation batch (qwen3.8-max, temperature 0), with 374–379 valid instances for the two gold arms; per-corpus results are computed separately on the 243 CPED anchors and 138 RPC anchors of the same batch and are never pooled across corpora. The absolute scores in that batch are: direct prompting ABCD 4.170 / ESC 3.703 / L2 0.560; PSyMod 4.084 / 3.748 / 0.590; Cond-gold 4.172 / 3.747 / 0.572; Cond-gold+mem 4.063 / 3.644 / 0.615. For PSyMod versus direct prompting, the paired intervals for ABCD and overall ESC both include zero. By corpus, PSyMod’s ESC is higher on CPED while direct prompting is higher on RealPersonaChat, so the current results do not support a consistent advantage across corpora. We will adjust the conclusions accordingly and report per-corpus results and confidence intervals alongside the aggregate scores.

ConditionValid nABCDESC overallL2
Prompt-only3814.1703.7030.560
PSyMod3814.0843.7480.590
Cond-gold374–379 (varies by metric)4.1723.7470.572
Cond-gold+mem374–379 (varies by metric)4.0633.6440.615

15-arm anonymous joint evaluation batch (Table 1 pool); all conditions were rated within the same batch. The full-system and neutralized-condition values of Table 5 are as follows, with a paired L2 difference of −0.010 [−0.035, +0.014] (n = 381):

Original Table 5 conditionABCDESCL2
Full system3.9493.5930.582
Situation estimate set to neutral3.8823.4120.592
AuthorsOpenReview ↗

Response to Reviewers: With Thanks for Their Constructive Comments

Q1

Response. Thank you for specifying the control conditions needed for this comparison and for reminding us to explain the score differences across tables. PSyMod ranks memories, selects personality rules, and plans the reply according to the current situation. We also clarify: the prompt-only input is taken after the retrieval stage — the union of the top-K factual-retrieval results and the top-K situation-retrieval results — rather than the long context fed directly.

The mechanism-level controls (three AG-CTR²-style retrieval-query conditions, with the same chain and situation modulation disabled; ABCD / L2 summary):

ConditionABCDL2
Full system (same batch)3.6350.592
Situation-appraisal reasoning-chain query + situation-retrieval modulation3.4330.529
Same chain, situation-retrieval modulation disabled3.4040.515
Length-matched general reasoning-chain query3.4830.498

Summary scores of Cond-gold and Cond-gold+mem:

ConditionABCDESC overallL2
Prompt-only4.1703.7030.560
PSyMod4.0843.7480.590
Cond-gold4.1723.7470.572
Cond-gold+mem4.0633.6440.615

Input information and call volume of each condition:

ConditionInput informationBase modelCall volume
Full systemSelf-constructed state + retrieved history (out-of-window entries available)gpt-5.55.9 generation calls per instance (4,220/720); ~22 construction calls per package (15,932 unique turns)
Direct promptingDialogue window onlygpt-5.51 call per instance
Cond-goldWindow + reference state (gold)gpt-5.51 call per instance
Cond-gold+memAdditionally reference memories (gold)gpt-5.51 call per instance

The differences across tables:

Original tableFull-system ABCDFull-system ESCFull-system L2Valid nEvaluation target
Table 14.0843.7480.590381 (CPED 243 + RPC 138)Cross-system generation comparison
Tables 4/53.9493.5930.582381Intermediate-/access-layer ablations, ablation-arm pool
New control batch3.635□0.592260 (scoring in progress)Matched controls

The reason for the three-way difference is that each blind-evaluation table included a different set of candidate systems: Table 1 is a 15-arm cross-system comparison; Tables 4/5 use an ablation-arm pool (the same evaluation instances rated together, but with a candidate set different from Table 1); the new controls come from the rebuttal batch. The scoring rule (qwen3.8-max, temperature 0) is identical, but the candidate set changes the within-batch reference. Therefore each ablation can only be compared with the full system from the same evaluation; absolute scores across tables are not subtracted. The judge, temperature, and pairing convention are stated in each table’s caption.

Reviewer_BKUeOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper investigates how users’ psychological appraisal can guide memory use in dialogue, and proposes a framework that integrates psychological descriptors such as Big Five personality and the DIAMONDS situation taxonomy into memory retrieval and response generation. The goal is to make retrieved memories and generated responses more consistent with the target user’s characteristics.

Critical strengths

The paper makes a useful attempt to connect psychological characterization with dialogue memory and response generation. Integrating Big Five and DIAMONDS into the pipeline is a reasonable direction for making personalized responses less generic. The task formulation—constructing a persistent state for a designated participant in multi-party dialogue and generating a single-sentence response at a designated turn—is clear and controlled, which is helpful for evaluation. The construction audit and the separate evaluation of state construction, conditioned generation, and closed-loop operation are also sensible organizational choices.

Critical weaknesses
  1. In line 147, the authors introduce both Big Five and DIAMONDS simultaneously. I would like the authors to explain the conceptual relationship between these two representations and the concrete advantage of combining them over using either one alone. Big Five describes relatively stable personality traits, whereas DIAMONDS describes situational affordances/appraisals; these are not interchangeable. The paper should clarify how they interact in the model, whether they are redundant in practice, and what ablation evidence shows that the combination is necessary rather than merely additive. A correlation analysis or an ablation with Big Five only, DIAMONDS only, and both would strengthen the claim.

  2. The abstract and introduction motivate the work by arguing that users’ psychological appraisal should guide memory retrieval. However, in lines 147 and 156, it appears that memory and psychological representations are extracted jointly from dialogue history, rather than memory retrieval being conditioned on psychological appraisal. If this reading is correct, then psychological representations are used at response-generation time but not in the memory-retrieval process itself. This is a significant mismatch between the claimed contribution and the actual mechanism.

  3. In line 156, I am confused about the difference between M(p,t) and MpM_p. Both appear to refer to content before turn t. If they are effectively the same, the notation is unnecessary and obscures the method. The authors should define both terms precisely, including their scope, filtering criteria, and whether they differ in content or only in indexing.

  4. The experimental comparison raises serious fairness and validity concerns. First, it is not clear what base model the proposed method uses. This is essential for interpreting all results. Second, comparisons in Table 1 with CharacterGLM-6B, PsyLLM-8B are difficult to assess because the models differ in parameter count. Such comparisons are not sufficient to attribute gains to the proposed psychological state construction. Third, the prompt-only baseline achieving extremely high metrics is surprising and undermines confidence in the experimental design. This may indicate that the task is too easy, that the metrics are insensitive, that the evaluation annotations are biased, or that there is some leakage/overlap between prompts and references.

Questions for authors

See weaknesses 1-4

2 comments
AuthorsOpenReview ↗

Response to Reviewers BKUe: With Thanks for Their Constructive Comments

Weakness 1 — Why Big Five and DIAMONDS are used together, and whether they are redundant

Response. Thank you for reminding us to distinguish the conceptual relationship between the two representations from the practical effect of using them jointly. The Big Five describes relatively stable personal traits; DIAMONDS describes the current situation. We note that the Big Five profiles in this dataset are directly provided by the dataset: we adopt the expert annotations from prior work on the dataset as the ground-truth reference for the Big Five. The DIAMONDS estimates reflect the target participant’s subjective appraisal rather than a generic description of the situation; the supporting evidence is the analysis of 5,333 naturally occurring turn pairs in our response to Reviewer jCjT (Weakness 2), where the mean distance between the eight-dimensional situation vectors of different participants within three turns is 0.369. Across the 40 correlation analyses between the five personality traits and the eight situation dimensions, the associations that remain significant after multiple-comparison correction all involve agreeableness. We provide the correlation table below.

Big Five × DIAMONDSDutyIntellectAdversityMatingPositivityNegativityDeceptionSociality
Openness+0.248 / 19 / 1.000+0.262 / 19 / 1.000+0.179 / 19 / 1.000−0.094 / 19 / 1.000−0.376 / 19 / 1.000+0.266 / 19 / 1.000+0.257 / 19 / 1.000+0.051 / 19 / 1.000
Conscientiousness+0.579 / 22 / 0.189+0.276 / 22 / 1.000+0.425 / 22 / 1.000−0.362 / 22 / 1.000−0.495 / 22 / 0.629+0.401 / 22 / 1.000−0.087 / 22 / 1.000+0.211 / 22 / 1.000
Extraversion−0.072 / 23 / 1.000−0.418 / 23 / 1.000+0.301 / 23 / 1.000+0.023 / 23 / 1.000−0.060 / 23 / 1.000+0.240 / 23 / 1.000+0.293 / 23 / 1.000+0.411 / 23 / 1.000
Agreeableness+0.659 / 24 / 0.032+0.323 / 24 / 1.000+0.690 / 24 / 0.019−0.066 / 24 / 1.000−0.721 / 24 / 0.008+0.765 / 24 / 0.004+0.603 / 24 / 0.026+0.362 / 24 / 1.000
Neuroticism−0.124 / 20 / 1.000−0.010 / 20 / 1.000−0.354 / 20 / 1.000−0.188 / 20 / 1.000+0.319 / 20 / 1.000−0.396 / 20 / 1.000−0.471 / 20 / 0.970−0.059 / 20 / 1.000

Each cell reports ρ\rho / valid n / Holm-corrected p.

In addition, we ran the following ablations in our method: keeping only the situation, keeping only the Big Five, and keeping both while disabling their interaction. The results are shown in the table below.

ConditionABCDL2
Full system (both)3.6350.592
Situation only (noP; the original remove-personality-projection condition)3.5410.562
Big Five only (noS_hard)3.5510.511
Both channels kept, interaction disabled (noX_inter)3.5320.509

Weakness 2 — Psychological representations are used at generation time but not in memory retrieval

Response. Thank you for pointing out that the original manuscript did not clearly separate the two stages of memory construction and retrieval. When a memory is written, the model records the source situation for each entry. On later turns, the model estimates the current situation and ranks the existing memories accordingly before generating a reply. Situation information therefore already participates in retrieval. The factual memory and the psychological situation can be viewed as a tuple: retrieval through either the factual channel or the psychological-situation channel can reach the corresponding memories. Table 6 compares retrieval with and without the situation codes stored on memory entries. This ablation removes only the situation codes of the memory entries within the same pipeline; the rest of the retrieval process is unchanged.

Original Table 6 conditionHit@3MRR
Full encoding0.119 (80/673)0.086
Situation codes removed0.089 (60/671)0.070
AuthorsOpenReview ↗

Response to Reviewers BKUe: With Thanks for Their Constructive Comments

Weakness 3 — The notational difference between M(p,t) and M_p

Response. We apologize that the notation in the original manuscript was not clearly explained. MpM_p denotes the participant’s long-term memory bank. At turn tt, the system retrieves from the bank the entries left by earlier turns that are relevant to the current dialogue. M(p,t)M(p,t) denotes the entries selected for this turn, which can differ from turn to turn. Therefore, even when the original utterance is no longer inside the current window, the system can still retrieve the corresponding memory. We will define both symbols where they first appear. Algorithm 1 will also explicitly list the retrieval input and output.

SymbolScopeOperational role
MpM_pThe long-term memory bank available up to that pointRetrieval input
M(p,t)M(p,t)The entries selected from the bank for turn ttRetrieval output; varies per turn

Weakness 4 — Fairness and validity of the experimental comparisons

Response. Thank you for pointing out the concerns about comparison fairness and evaluation validity. We agree that the base model and the input information of each system must be made explicit. The backbone of our model is the GPT-5.5 API. We additionally ran two more base models, Qwen2.5-7B and Llama 3.1-8B. With our method available on these base models, we can compare with the other baselines under the same base model. However, it must also be noted that our method is training-free, whereas those baselines involve SFT and RL, so a direct comparison with them is not possible.

ConditionABCDESCL2
gpt-5.5: direct prompting4.1703.7030.560
gpt-5.5: PSyMod4.0843.7480.590
Qwen2.5-7B: PSyMod2.6772.1260.420
Llama 3.1-8B: PSyMod2.4851.9440.429

We also need to point out that our prompt-only setup first performs the same retrieval: memories are filtered once through the top-K factual retrieval and the top-K psychological-situation retrieval, and only memories we consider reasonably accurate are fed into the prompt. We also ran a variant that skips retrieval and feeds the input directly; the results are in the table below. In the comparison of per-turn bank updating (P), reconstruction from the current window only (RB), and freezing the initial state (FZ), against full-history direct prompting (PA), we can see that our per-turn-updated long-term memory bank scores better on the downstream generations.

ConditionABCDL2
P: long-term memory bank updated per turn3.6350.592
RB: state reconstructed from the current window only3.4580.482
FZ: initial state frozen3.3840.502
PA: full-history direct prompting3.2780.484
Reviewer_jCjTOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes PSyMod, a framework for constructing participant-specific states and generating personalized dialogue responses. It builds episodic memories, a Big Five personality profile, and a historical DIAMONDS reference from preceding dialogue. At generation time, the framework estimates the current psychological situation and affect, and uses situation-dependent rules to guide memory retrieval, personality expression, and response planning. The operational task is single-sentence response generation for a designated participant in a multi-party conversation.

Critical strengths

The distinction between topical similarity and psychological relevance is useful. The motivating example illustrates why a memory from a different surface context may be more relevant to a response than a topically similar experience.

Critical weaknesses
  1. The application motivation, operational task, and evaluation criteria are insufficiently aligned. The introduction motivates personalized companionship and psychological support, where an assistant adapts its response to a user. However, Section 3 defines a different operational task: constructing a state for participant (p) and generating the next utterance as (p). The experiments therefore establish character-conditioned continuation more directly than user-adaptive assistance.
  2. The accuracy of the constructed psychological states is not directly established. The paper interprets psychological situation as the target participant’s appraisal, but does not demonstrate that its measurements capture this participant-specific perspective rather than a generic interpretation of the event.
  3. The experiments do not adequately isolate the contribution of the proposed psychological modulation mechanism. Table 5, removing situation read-out improves every L2 modulation column, including the pooled score from 0.582 to 0.592, although ABCD and ESC decrease. Table 6 provides positive evidence for situation encoding in retrieval, but neither result establishes the distinctive value of DIAMONDS and its fixed routing rules over simpler semantic representations or natural-language appraisal descriptions.
  4. Important evaluation definitions and reporting details are incomplete or inconsistent. The main text provides appendix references, but does not sufficiently explain several headline measures needed to interpret the results. In particular, the meanings and operational roles of Beh., Verb., Emo., and Tend. in Table 2 are not adequately introduced, and the Cost column lacks a clear unit and aggregation convention.
Questions for authors
  1. Is the intended capability to simulate a target participant or to generate assistant responses adapted to that participant? How is ESC-Eval validated for turns in which the target participant is not acting as an emotional supporter?
  2. Which reported experiment directly evaluates inferred Big Five profiles against the available human annotations? What evidence shows that DIAMONDS readings reflect the target participant’s appraisal rather than a generic description of the situation?
  3. Can the authors report Cond-gold and Cond-gold+mem results, alongside same-backbone, same-information, and comparable-budget alternatives using direct profile prompting or semantic retrieval? How should the uniformly improved L2 scores after removing situation read-out be interpreted?
  4. Which abstention policy was actually implemented for each metric? Please provide per-system effective sample sizes, missing rates, the complete pooled-L2 calculation, the Cost unit, and confidence intervals for the principal pairwise comparisons.
  5. What mechanism preserves an experience after its source utterances leave (W_{<t}), and what experiment demonstrates that this retained information improves responses beyond what can be achieved by reconstructing state from the current window alone?
4 comments
AuthorsOpenReview ↗

Response to Reviewers jCjT: With Thanks for Their Constructive Comments

Weakness 1 — Application motivation, operational task, and evaluation criteria are insufficiently aligned

Response. Thank you for pointing out the ambiguity in the task definition. We apologize that the original manuscript did not clearly distinguish the roles of the user and the assistant. We will revise the task definition to: “The model estimates the user’s state from the user’s dialogue history and current situation. The assistant then generates its reply to that user.” During evaluation, we compare the generated text with the participant’s actual utterance recorded in the corpus. The revised manuscript will correct the task definition.

Weakness 2 — The accuracy of the constructed psychological states is not directly established

Response. Thank you for raising this question. We agree, and we have set up an experiment to verify it. During the dialogues, we evaluated the psychological situation from both participants’ perspectives, obtaining 5,333 pairs of psychological situations. We then computed the vector distance between the eight-dimensional situation vectors; the mean distance is dˉ=0.369\bar{d} = 0.369. In our view, the psychological-situation ratings obtained from the two participants’ perspectives are not consistent. This indicates that our psychological situation captures the perspective specific to that participant, rather than a generic interpretation of the event as a whole.

Natural pairingNo. of pairsMean distance
Different participants, ≤3 turns apart5,3330.369

Weakness 3 — The experiments do not adequately isolate the contribution of the modulation mechanism

Response. Thank you for suggesting more targeted controls. We agree that the original ablations do not fully answer the question you raise. Owing to the way the paper was written, our original ablation actually obtained the historical context from the historical memories retrieved according to the psychological situation, which helps the model better understand the character’s past experiences under that psychological situation. Our “without read-out” variant meant that we did not feed this historical context into our method. We therefore set up a further experiment: setting the psychological situation entirely to its median (neutral) value, so as to exclude the contribution of the psychological situation without changing the overall pipeline.

Original Table 5 conditionABCDESCL2
Full system3.9493.5930.582
Situation estimate set to neutral3.8823.4120.592

To further verify the role of the psychological situation, we modified the method and added three more ablations: completely removing the situation information, describing the situation in natural language, and retrieving memories only by semantic similarity. The setup of the new controls: the four arms — full system, complete removal of situation information (noS3), natural-language situation (nlapp), and semantic retrieval (sem) — were all generated with gpt-5.5 under the same runner and prompts, differing only in the retrieval method or in how the situation information is supplied.

ConditionABCDL2
Full system (same batch as new controls)3.6350.592
Situation information completely removed (noS3)3.4990.507
Natural-language situation (nlapp)3.4440.486
Semantic retrieval (sem)3.3650.498

In addition, we verified on the construction side how these variants affect memory retrieval.

ConditionHit@3MRR@3
Full system0.1190.086
Situation estimate set to neutral0.0890.070
Situation information completely removed (noS3)0.06250.0503
Natural-language situation (nlapp)0.03120.0191
Semantic retrieval (sem)0.01040.0035
AuthorsOpenReview ↗

Response to Reviewers jCjT: With Thanks for Their Constructive Comments

Weakness 4 — Important evaluation definitions and reporting details are incomplete

Response. Thank you for pointing out these reporting omissions. The original manuscript did not explain the meaning of the metrics or the cost computation sufficiently; we will add the missing definitions. We will define Beh., Verb., Emo. and Tend. alongside Table 2. The first three count the proportion of memory entries that meet the corresponding condition. Tend. first counts, for each example, the number of tendency categories supported by at least three pieces of evidence, and then averages this count across examples. These metrics describe what information the memory bank records and which of it can be traced back to the original dialogue; they do not directly measure the accuracy of the psychological-state estimates. For the first three metrics, we compute the within-bank proportion of entries and then average equally across non-empty banks; Tend. averages the category counts. We will list the number of non-empty banks under each condition separately. The appendix will report the costs of memory-bank construction and of response generation separately, noting the units, the denominators, and the treatment of repeated calls, failed calls, and cached results. These additions will make the statistics easier to interpret and reproduce.

MetricWhat it measuresDenominator and aggregation
Beh.Whether a memory entry retains textual markers of speech acts such as urging, requesting, or caring; an entry counts if it matches the behavioral structure, the tone, or any item in the intensity-cap lexiconMatched entries / total entries nn in the sample’s frozen memory bank (behavior_trace_rate = beh/n, :135); proportions averaged equally across non-empty banks ; valid sample count reported per condition (ours 714)
Verb.Overlap ratio ≥ 0.8 with the text of some turn in the visible dialogue history; containment is character-bigram overlap max⁡r∣bi(e)∩bi(r)∣/∣bi(e)∣\max_r \vert b_i(e) \cap b_i(r) \vert / \vert b_i(e) \vert (person_readout.py:88-98,127-128)Qualifying entries / bank entries nn (verbatim_rate = verb/n, :136); averaged equally across non-empty banks
Emo.Proportion of entries with text-overlap ratio ≥ 0.8 whose source turn carries an enc_E emotion annotationQualifying entries / bank entries nn (E_readable_rate = E_ok/n, :137); averaged equally across non-empty banks
Tend.Number of tendency categories supported by at least 3 pieces of evidence per example (N_MIN=3, matching the rule that a dimension of θ^\hat{\theta} is not estimated, person_readout.py:39,141)Category counts averaged across non-empty banks; valid n = number of non-empty banks

Question 1 — Simulate the target participant, or generate assistant responses adapted to that participant?

Response. Thank you for reminding us to clarify the intended task and the scope of ESC-Eval. The assistant estimates the user’s state and generates a reply addressed to that user. ESC-Eval is a multi-dimensional LLM-as-judge framework for evaluating emotional support conversations (Zhao et al., EMNLP 2024); it scores the text of the reply from 1 to 5 on seven dimensions: Fluency, Expression, Empathy, Information, Humanoid, Skill, and Overall. The evaluation covers all eligible turns, including turns in which the participant was not offering emotional support. ESC was scored in an anonymous joint batch (qwen3.8-max, temperature 0), in which PSyMod and the short-window direct-prompt baseline were rated among the same 15 candidates, yielding 236 valid instances on CPED and 138 on RealPersonaChat under both conditions.

Question 2 — How are the Big Five and DIAMONDS each validated against ground truth?

Response. Thank you for requesting direct validation. We agree that we should state separately which reference each of the two representations is evaluated against, and what conclusions the current validation can support. Our Big Five profiles are directly provided by the dataset; that is, we adopt the expert annotations from prior work on the dataset as the ground truth for the Big Five. Our DIAMONDS readings reflect the participant’s subjective appraisal rather than a generic description of the situation; for this we refer to the experimental results in our response to Weakness 2 above.

AuthorsOpenReview ↗

Response to Reviewers jCjT: With Thanks for Their Constructive Comments

Question 3 — Cond-gold pairs and same-budget alternatives

Response. Thank you for suggesting these additional comparisons. We apologize that the original manuscript omitted the results obtained when the reference state and memories are provided. We need to clarify that the input to our prompt-only baseline is actually taken after our retrieval stage: what we provide it is the union of the top-K results of factual retrieval and the top-K results of psychological-situation retrieval, rather than feeding the long context directly. Cond-gold and Cond-gold+mem were designed as upper bounds, that is, to verify what the model can achieve once it is given the ground-truth information: Cond-gold directly provides the reference state, and Cond-gold+mem additionally provides the reference memories. We will state in the same table the information used and the computational cost of these two conditions and of PSyMod. The L2 scores are explained in detail in our response to Weakness 3.

ConditionABCDESC overallL2
Prompt-only4.1703.7030.560
PSyMod4.0843.7480.590
Cond-gold4.1723.7470.572
Cond-gold+mem4.0633.6440.615
ConditionInput informationBase modelCall volume
Full systemSelf-constructed state + retrieved history (out-of-window entries available)gpt-5.55.9 generation calls per instance (4,220/720); ~22 construction calls per package (15,932 unique turns)
Direct promptingDialogue window onlygpt-5.51 call per instance
Cond-goldWindow + reference state (gold)gpt-5.51 call per instance
Cond-gold+memAdditionally reference memories (gold)gpt-5.51 call per instance

The table above is the same-backbone, same-information, comparable-budget control framework: the base model is gpt-5.5 throughout, and the conditions differ only in the state and memory information supplied.

AuthorsOpenReview ↗

Response to Reviewers jCjT: With Thanks for Their Constructive Comments

Question 4 — Abstention policy and reporting definitions

Response. Thank you for asking us to make the evaluation protocol more explicit. For each example, the pooled L2 score is the equal-weight average of the scorable dimensions among emotion, personality, and memory. Dimensions that cannot be scored are removed from the denominator. We will list, per system and per metric, the number of eligible examples, the number of outputs actually obtained, the number of valid scores, and the number of missing outputs, distinguishing “metric not applicable” from “generation failure”. The table below lists the eligible, obtained, and valid counts per system and batch; paired comparisons are computed only on examples with valid scores under both conditions, so different denominators such as 381, 372, 374–379, 676, and 672 appear side by side and are stated with each row. When computing paired differences, only examples with valid scores under both conditions are included. The appendix will state the cost unit and the denominator of each average. We will also explain when a score is withheld, and will count missing outputs and not-applicable scoring dimensions separately.

System / batchEligible nObtained nValid nMissing n / rateAbstention / exclusion rule
Each generation condition (itemized)381 scoring anchors (CPED 243 + RPC 138)381 (Table 1 pool, 15 arms)381; gold conditions 374–379 (varies by metric)gold arms missing 2–7 items, denominator 381E missing → NA; P null dimension → that dimension NA; overall = equal-weight average of non-NA cell values, NA counts reported per row
Each retrieval condition (itemized)717 evaluation instances673 (full encoding) / 671 (de-encoded)673 / 671full encoding missing 44, de-encoded missing 46, denominator 717examples without gold M_top leave the denominator
Each closed-loop condition (itemized)720 closed-loop checkpoints676 paired676 (ABCD) / 672 (L2)44 no_pair leave the denominator, rate 44/720no_pair = closed-loop cross-verification found no corresponding single-turn anchor

Question 5 — What preserves an experience after its source utterances leave the window?

Response. Thank you for your attention to how information beyond the context window is retained and what role it plays. Once the original utterances leave the context window, the corresponding memories remain stored in the bank and can be retrieved again on later turns. The experiment is set up as follows: 181 stratified closed-loop instances compare per-turn bank updating (P), reconstruction from the current window only (RB), and freezing the initial state (FZ); against full-history direct prompting (PA), we can see that our per-turn-updated long-term memory bank scores better on the downstream generations.

ConditionABCDL2
P: long-term memory bank updated per turn3.6350.592
RB: state reconstructed from the current window only3.4580.482
FZ: initial state frozen3.3840.502
PA: full-history direct prompting3.2780.484

In addition, we ran an ablation on memory-bank updating: if nothing is written back to the bank, our method scores lower than the variant that updates the bank.

Original Table 3, standalone referenceABCD earlyABCD lateValid n
PSyMod2.6142.86124 evaluation instances (fixB batch)
PSyMod without write-back2.5682.722same

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.