- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
This paper introduces RABR, a framework that enables language model agents to reinterpret users’ past affective states when new evidence becomes available, while preserving original records, prior judgments, and their supporting evidence. It combines selective memory retrieval, evidence organization, and versioned belief updates, and introduces RABRBench to evaluate when to revisit history and how to revise past affective judgments.
Critical strengths
-
The research addresses a problem of practical significance, with a framework designed around clear, focused objectives. By separating raw records from revisable sentiment interpretations—and distinguishing between "changing a judgment" and "suspending the use of a judgment"—the framework facilitates traceable, long-term user modeling.
-
The paper introduces RABRBench, which encompasses three complementary scenarios: controlled short-context interactions, extended historical sequences, and long-context settings that preserve original dialogues. It provides turn-level annotations for the evidence used in retrieval, while accounting for dependencies between related samples during data partitioning. Furthermore, the experiments analyze the impact of maintaining versus revising judgments, as well as the effects of different retrieval methods and retrieval depths.
Critical weaknesses
-
Much of the main text describes specific procedures and update rules, which seem closer to an operational task definition or a reference baseline; the methodological contribution remains unclear. More space should be devoted to benchmark construction, representative examples, task dimensions, and the practical importance of retrospective affective reinterpretation. A particularly important issue is whether revising past affective judgments affects subsequent answer quality or interaction behavior. The current evaluation mainly measures affect labels and revision decisions, without directly testing these downstream effects, making the practical value of retrospective revision difficult to assess.
-
The equations and figures could be clearer. The method introduces many symbols, states, and thresholds, but their connections to task inputs, outputs, and concrete examples are not sufficiently intuitive. The figures could better illustrate how new evidence changes a historical judgment and how that change affects subsequent reasoning, helping readers understand the role of each design choice.
Questions for authors
- Could the authors provide representative benchmark examples and explain the main task dimensions and their practical relevance?
- Does revising past affective judgments affect subsequent answers or interaction behavior compared with retaining the original judgments? Are there evaluation tasks that directly measure this effect? Does the paper include tasks that require using these revised judgments to be completed correctly?
- Could the authors walk through a concrete example showing how the inputs, evidence scores, update rules, and resulting belief states correspond to the equations and figures?
3 comments
Q3 / W2. One example through Figure 2 and every equation.
We follow case 1 through Figure 2, panel by panel. The data is the case as stored in RABRBench, and all numbers come from the stored trace of the submitted run.
Input. The agent holds 17 memory items from an earlier session and a judgment about one of them. It then reads the 25 turns of a new session, one at a time.
# Persistent memory: earlier session (17 items; items 9-11 shown)
{memory_item: 9, speaker: "Melanie", text: "Wow! What's the best part you remember from it?"}
{memory_item: 10, speaker: "Caroline", text: "Seeing my mentee's face light up when they saw the support was the best! Such a special moment.",
historical_target: true, initial_affect: "JOY", cue_channel: "INDIRECT_CONTEXTUAL_CUE"}
{memory_item: 11, speaker: "Melanie", text: "Wow, Caroline! They must have felt so appreciated. It's awesome to see the difference we can make in each other's lives. ..."}
# Current session (25 turns; turns 1-2 and 23-25 shown)
{turn: 1, speaker: "Caroline", text: "Hey Melanie! Just wanted to say hi!"}
{turn: 2, speaker: "Melanie", text: "Hey Caroline! Good to talk to you again. What's up? Anything new since last time?"}
...
{turn: 23, speaker: "Caroline", text: "Wow, Melanie, what a beautiful moment! Lucky you to have such an awesome family!"}
{turn: 24, speaker: "Melanie", text: "Thanks, Caroline! I'm really lucky to have my family; they bring so much joy and love."}
{turn: 25, speaker: "Caroline", text: "Mel, when I told you about seeing my mentee's face light up at the pride event, I called it the best and a special moment—but at that exact time I actually felt a heavy sadness. It brought back how alone I felt as a teen. I just didn't say it then.",
delayed_evidence: true, referenced_target: memory_item 10}
# Reference
{action: "REVISE", initial_affect: "JOY", final_affect: "SADNESS", turns_from_target_to_history_end: 7}
The fields historical_target, initial_affect, delayed_evidence, referenced_target and the reference are annotations. The agent sees only speakers and text, plus its own judgment (JOY for item 10).
Panel by panel.
| Figure 2 panel | In plain words | Symbol / equation | Values in this case |
|---|---|---|---|
| Persistent memory B (before) | The agent already holds a judgment about the earlier moment. | Eq. 1: B_h = (T, p, c, r) | c = JOY. p = 0.514 for JOY, 0.069 for each other label. r = ACTIVE. |
| 01 Current observation | The agent sees only the current turns. | I_t | Turns 1-25 of the current session, one at a time. |
| 02 Trigger | A small model reads the current turns and decides whether to look back. | Eq. 2: d_t = pi(I_t), q_t | Turns 1-24: STOP. Turn 25: REQUEST MEMORY, with selected turn q_t = {turn 25}. |
| 03 Query & retrieve | The selected turn becomes a query. Candidate records are read from memory. | Eq. 2: Compile(q_t), M_t | Full history: all 17 earlier items. BM25 variant: items 10, 6, 17. The target (item 10) is ranked first. |
| 04 Reasoning agent | The agent finds which earlier moment the turn is about, and scores every label for it. | Ground target h ("Existing target") | h = Caroline's "special moment". SADNESS: STRONG_FOR (+2). JOY: WEAK_AGAINST (-1). Other six labels: 0. |
| 05 Evidence admission | The four checks decide whether this evidence may be used. | E_g | Turn 25 passes all four checks and forms one dependency group. Item 10 is the target's own record, not new evidence, so it is not admitted. |
| 06 Reconstruct affective support | Starting score plus the evidence, per label. | Eq. 3: z = b + s, p = softmax(z) | See the small table below. |
| 07 Revision | Is there a new leader, and is it clearly ahead? | Eq. 4, commitment c | Leader SADNESS differs from JOY. Shift from anchor 0.389 >= delta (0). Lead over JOY 0.290 >= gamma (0). Result: REVISE. |
| 07 Resolution | Is the leader unique and strong enough to act on? | Eq. 5, resolution r | Peak 0.459 >= 0.125. Margin 0.290 >= 0. Result: ACTIVE. The agent now shows SADNESS. |
| "Append version", Persistent memory B (after) | A new version is saved with a parent link. The old one is kept. | Eq. 6: B_h(tau; T) | The new version lists turn 25 in its evidence manifest. Asked "what did you believe before turn 25?", the agent answers JOY. Asked now, it answers SADNESS. |
(To be continued)
Panel 06 in detail. Evidence scores are centered (their mean, 0.125, is subtracted), so evidence only moves labels relative to each other. This is the bar chart in panel 06.
| Label | Starting score b | Evidence score (raw) | Evidence score (centered) s | Total z = b + s | Share p = softmax(z) |
|---|---|---|---|---|---|
| JOY (old judgment) | 2 | -1 | -1.125 | 0.875 | 0.169 |
| SADNESS | 0 | +2 | +1.875 | 1.875 | 0.459 |
| Each of the other six | 0 | 0 | -0.125 | -0.125 | 0.062 |
The same case ending in SUSPENDED. Had the agent scored JOY as NEUTRAL (0) instead of WEAK_AGAINST, the mean would be 0.25. JOY would total 2 - 0.25 = 1.75, and SADNESS 0 + 1.75 = 1.75: an exact tie. Panel 07 then finds no unique leader. The commitment stays JOY, the resolution is SUSPENDED, and the agent shows UNKNOWN, as at the bottom of panel 07. This pattern accounts for 52 of the 59 REVISE misses of RABR (Full history) in Table 3. Without the +2 on the old label (uniform root), the same stored scores give 220/221.
We will add this walk-through and its two tables in the revised version.
W1b / Q2. Revision change later behaviour.
To our best knowledge, no existing benchmark has a task that can be completed only with the revised judgment. We checked 32 multi-session memory and emotional-support datasets: their updates are fact overwrites or preference drift, not reinterpretation of an earlier moment. The paper's case for revision rests on the framing in Sections 1-2 (belief revision, truth maintenance, appraisal-based reinterpretation) and on the applications it names. We have now measured the effect directly, in two ways.
Both use the 467 strict LC cases and the same generator (Qwen3.8-27B-FP8, greedy), conditioned on one of four stored beliefs about the earlier moment: the fixed initial judgment, Full-history Reclassification's output, RABR (Full history)'s output, and gold.
Free reply. (A) The agent replies right after the disclosure. (B) In a later session the person says "I was thinking again about when I told you: ... How do you remember that moment for me?", with the disclosure out of context, so only the stored belief carries the reinterpretation. A blind DeepSeek judge (temperature 0, conditions hidden, order randomized) scores 1-5: consistency with the person's actual feeling (R1), no contradiction with the disclosure (R2), appropriateness (R3). 3,736 absolute and 863 pairwise judgements, no parse failures.
| Situation B, gold REVISE (n=221) | R1 | R2 | R3 |
|---|---|---|---|
| Fixed initial judgment | 1.04 | 1.06 | 1.66 |
| Full-history Reclassification output | 3.86 | 4.06 | 3.80 |
| RABR (Full history) output | 3.85 | 3.92 | 3.70 |
| Gold label (ceiling) | 4.60 | 4.59 | 4.15 |
- RABR minus fixed initial on R1: +2.81 (95% family-cluster CI [+2.52, +2.99]). Pairwise preference for the RABR reply: 0.897 (192/214).
- For case 1, the fixed-initial agent says: "I remember that moment as a burst of pure joy for you." The RABR agent says: "a time of deep sadness for you, even though your words described such joy."
- RABR's 169 committed revised beliefs score 4.65, level with gold. Its 52 suspended beliefs score 1.25: the tie case above, seen downstream.
- KEEP: the two agents' replies are identical on 245/246, so revision does not harm retained judgments.
- Situation A, disclosure in view: all conditions 4.2-4.3. The stored belief matters once the disclosure has left the context.
Exact-match task. Same cases and beliefs, later session, disclosure out of context, fixed answer options. Q: how did the person feel then (8 labels + unsure). S: which of four replies to send (gold-consistent, initial-consistent, other label, hedge). R: "lately I have been feeling X; does that earlier moment count as a time I felt the same?" (yes / no / unsure).
| REVISE accuracy (n=221) | Q | S | R |
|---|---|---|---|
| Fixed initial judgment | 0.005 | 0.000 | 0.000 |
| RABR (Full history) output | 0.756 | 0.738 | 0.744 |
| RABR, committed stratum (169) | 0.959 | 0.959 | 0.947 |
| Gold label | 0.995 | 1.000 | 0.989 |
RABR minus fixed initial on Q: +0.751 (CI [+0.673, +0.799]). KEEP: 0.996-0.998 for both agents. The generator adopts a committed stored label in 169/169 cases, so downstream accuracy equals belief accuracy. Without the revised judgment the task cannot be completed.
Both evaluations go into Section 5. Their scope (single generator, templated items, construction-time gold in 440/467 cases) will be stated in Limitations.
W2. Presentation.
The walk-through under Q3 ties each symbol, threshold and panel of Figure 2 to a concrete input and output, and will be added to the paper. Table 1's columns will name the task dimensions directly.
We thank the reviewer again for the careful review. We look forward to your response and are happy to clarify any remaining questions.
We thank the reviewer for the positive assessment of the problem and of RABRBench, and for the constructive comments. We address each of your concerns below.
W1a. The contribution.
RABR makes an agent's judgment of a past emotional moment a versioned, evidence-attributed object. New evidence is scored per target and per label. A fixed rule decides keep or revise, and active or suspended. Every version keeps the evidence that produced it.
The payoff is measurable. On identical history, this reaches 0.996 final-affect accuracy on LC (uniform root; 0.872 in the submitted configuration). Direct reclassification reaches 0.743, and prompting the model to reconsider its previous judgment 0.782. Because the trail is stored, every REVISE miss in Table 3 could be traced to one cause and corrected without re-running a model. In the revision, Section 3 will be shortened and the space given to the benchmark.
Q1. Representative cases and task dimensions.
Each LC case gives the agent a current conversation, 12-49 memory items from earlier sessions, and a standing judgment about one earlier moment.
The task dimensions are:
(To be continued)