acceptodds
Under review as a conference paper at ICLR 2027

Learning Affective Memory for Behavioral Alignment in Long-Horizon Agents

Abstract

Long-horizon agents must preserve the behavioral implications of earlier user concerns as their interaction history is compressed. Affective cues such as urgency, frustration, shame, anxiety, overconfidence, or distrust can indicate what is at stake without changing the task facts. Losing their relevant implications can produce grounded but behaviorally misaligned actions; retaining them indiscriminately can cause affect lock-in, leakage into neutral steps, or over-compliance with unsafe pressure. We study this problem as Memory as Affective Control (MAC). Our method is a trainable additive module that attaches to a semantic memory backbone and constructs compact, updateable affective memory encoding the action implications of user concerns: stance, behavior-control knobs, step-level gates, update/masking rules, and preferred or avoided paths. It preserves task-relevant affect, dampens stale or excessive affect, masks distractors, and retargets behavior toward alternative valid policies. We construct ambiguity-calibrated long-horizon coding and deep-research benchmarks where facts support grounding but do not determine the desired policy, with supervised and preference data for learning affective memory construction and execution. Across multiple memory backbones, compact affective memories improve behavioral steering over assigning the same total token budget to additional semantic memory, with high judge-rated task correctness and grounding. On Terminal-Bench 2.1 and BrowseComp with matched synthetic affective context, we compare against an equal-budget structured control summary with the same gate and update opportunities. Method-blind behavior diagnostics, human preferences, and compaction analyses support the updateable residual; native accuracy shows no observed mean trade-off in the tested runs. The results support memory as a means of preserving task-specific behavioral alignment across long-horizon execution.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Affective Memory, Behavioral Alignment

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_2MY1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To retain the behavioral implications of users’ concerns in memory, the paper proposes MAC (Memory as Affective Control), which complements semantic memory with a compact, updateable representation of action constraints. MAC selects affect-derived implications, revises them as new evidence arrives, and activates them on relevant steps. In addition to maintaining these memories at inference time with frozen models, the paper trains Qwen3.5-9B using SFT and DPO to better construct and use MAC memories.

The paper evaluates MAC on MAC-BENCH, a constructed benchmark of 200 coding and research scenarios, and on Terminal-Bench 2.1 and a 200-question BrowseComp subset with synthetic affective context. With matched memory budgets, MAC improves behavioral-alignment scores across four memory backbones. Blinded comparisons also favor MAC over a structured-control baseline with the same gating and update opportunities.

Critical strengths
  1. The paper addresses a practical limitation of agent memory. Remembering task facts does not necessarily preserve how users want the task to be carried out. Representing users’ concerns as action constraints, with conditions for updating or expiring them, is a useful approach to maintaining behavioral continuity in long-horizon tasks.
  2. The comparison with SCS+Gate is a meaningful control. Both methods receive the same information, memory budget, gating, and update opportunities. Blinded evaluations also favor MAC without exposing its memory cards or preferred paths to the evaluator.
Critical weaknesses
  1. The generalizability of the approach is not sufficiently established. MAC-BENCH uses predefined behavioral targets, and the public-task evaluations introduce synthetic affective context. These settings do not establish whether MAC can reliably infer appropriate action constraints from naturally occurring user concerns, particularly when those concerns are ambiguous or unfamiliar. The evidence therefore supports following specified constraints more strongly than alignment with users’ actual intentions.
  2. The dependence on model capability and the benefit of training remain unclear. MAC relies on the underlying model to extract actionable constraints, interpret the memory representation, and follow its gating and update rules. Its effectiveness may therefore depend substantially on instruction-following ability. Although the frozen-model evaluations cover multiple model families, only Qwen3.5-9B is trained, without a same-model comparison before training, after SFT, and after DPO. This leaves unclear whether training provides a generalizable improvement, compensates for limitations of this particular model, or enables a smaller model to match larger models under comparable task and compute settings.
Questions for authors
  1. Validity of inferred constraints: How were the control objectives and preferred behavioral paths generated and validated? Is there evidence that the constraints inferred by MAC reflect users’ actual intentions, rather than merely plausible interpretations of their concerns?
  2. Dependence on model capability and training: Do you have same-model results for Qwen3.5-9B before training, after SFT, and after DPO? How does the trained model compare with larger frozen models under matched conditions, particularly on independently verified task outcomes? This would help distinguish the benefit of training from differences in underlying model capability.
  3. Contribution of affective information: What distinguishes MAC’s processing from a generic method that extracts, updates, and applies action constraints? Do you have a matched comparison that isolates the contribution of affective information beyond the shared control fields and update opportunities in SCS+Gate?
  4. Real-world user alignment: Provide case studies or user evaluations, such as an A/B test or an Arena-style blinded comparison between MAC and the baseline? Allowing users to compare the resulting agent behaviors would help establish whether MAC’s inferred constraints match their actual intentions and improve their experience.
Reviewer_zdVcOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies how long-horizon agents (coding and deep research) can keep the behavioral implications of a user’s earlier emotional cues after memory compaction. The authors argue that standard semantic memory keeps the facts but loses why the agent should, say, verify more, limit scope, or resist pressure. The opposite failure, keeping affect indiscriminately, causes lock-in, leakage into neutral steps, or over-compliance with unsafe pressure. They propose MAC (Memory as Affective Control), an add-on module that sits next to an existing memory backbone to control behavioral alignment while preserving task facts.

Critical strengths
  • Important problem: Memory systems for agents are usually judged on factual recall, but long-horizon delegation also requires keeping the action implications of earlier user concerns, such as verifying more carefully after a production failure or resisting pressure to skip tests, through repeated compaction. Framing memory as a carrier of behavioral constraints, and naming concrete failure modes is a useful contribution to the agent-memory discussion.

  • Plug-and-play across backbones and models: MAC is evaluated on four memory backbones and several model families, including frozen API models and one fine-tuned open model, which supports the claim that it is a backbone-agnostic add-on.

  • MAC-Bench: Along with the affective memory, the paper provides MACBench, a custom benchmark.

Critical weaknesses
  • Writing: Even after multiple readings, it is pretty hard to understand the main methodology and the evaluation setup. Each section has varying jargon for which no intuition/motivation is provided. The abstract and Sections 3–4 rely on many coined terms (C-BSA, SCS+Gate, eCAC, T-Shift, MAC roles), key information is missing. Figure 1 is hard to understand.
  • Missing baselines on the main benchmark: MAC-Bench compares MAC only against semantic-only memory. The research direction is active in literature and has many recent as well as established works which propose agentic memory formulations, but MAC has been compared against none.
  • Insignificant gains: On Terminal-Bench 2.1 and BrowseComp, every accuracy difference has a 95% CI that includes zero (Table 3). The remaining evaluation is performed on MAC-Bench which is a custom benchmark proposed by the authors themselves, so it is unclear what MAC is gaining over the semantic only memory.
  • Long horizon?: Throughout the work discusses memory for long horizon agents but the evaluation setups have token budgets of not even 1k tokens, and even with these budgets the gains aren't significant, this makes the contribution totally redundant.
  • Meaning of the proposed components: The contribution as stated, is a combination of atom extraction, control-value selection, four operations, a gate, and update and mask rules, the authors provide no intuition/motivation behind these components and no ablation to quantify the significance of each. This makes the method seem more like an engineered set of pieces, than a research contribution.
  • Novelty concerns: Owing to the concerns mentioned above, the novelty of the work is not quantifiable.
Questions for authors
  1. Can you justify the contribution of the work even with the multiple major concerns as mentioned in the weaknesses?
  2. We need realistic token budgets to make any inference, can you entirely revamp the evaluation setup?
  3. Can you add some baselines which are relevant to such types of agent memories?
  4. Can you justify the components of the main pipeline? Only with intuition behind these and a set of ablations would make it sensible as a research contribution.
  5. Can you evaluate on more benchmarks and show gains which are significant unlike the current numbers?
  6. The work needs a major re-writing, can you make each section easy to understand and rely on less jargon which is not defined well? If AI is being used for the writing, please always verify whether each statement follows up coherently enough for us humans to be able to understand.

Note - I do believe that the work needs a major overhaul in multiple aspects which would be beyond the scope of a rebuttal, but I would love to be proven wrong, I welcome follow up questions and experiments.

Reviewer_udVjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper argues that agent memory should preserve not just task facts but also how earlier user concerns (urgency, distrust, pressure) should shape later behavior. It proposes MAC (Memory as Affective Control), a module that attaches to an existing memory backbone and, under a fixed token budget, reserves a small slice for a structured card encoding action implications such as which path to prefer, which behavior knobs to adjust (verification, scope, rollback, evidence skepticism), and when the card should apply or expire. The card is built by extracting affect-related atoms from the history, scoring them by how much they should change behavior, and applying one of four operations (preserve, dampen, mask, target-shift), with a gate that activates the card only on behavior-sensitive steps. Four frozen models perform all roles through prompting, and one small model (Qwen3.5-9B) is additionally trained with SFT and DPO. On MAC-Bench, a constructed suite of 200 coding and deep-research scenarios scored by an LLM judge, MAC improves the primary behavior-alignment score by 0.15 to 0.28 on a 4-point scale. On Terminal-Bench 2.1 and a BrowseComp subset with synthetically injected affect, compared against an equal-budget structured control summary with the same gate, accuracy gains are 0.5 to 1.6 pp and not significant, blind judges prefer MAC 56 to 58 percent of the time, and gains are largest after three or more memory compactions.

Critical strengths
  1. The problem well framed. Memory systems are usually judged on what facts they recall, but an agent can remember every fact and still forget why it should verify a change or limit a patch. Separating factual retention from behavioral continuity, and naming the failure modes (collapse, lock-in, leakage, overwrite, unsafe compliance), is a useful contribution on its own.
  2. The experiment setup is well-designed in terms of budget-matching. Comparing N + M semantic tokens against N semantic plus M control tokens on the same backbone isolates the allocation question rather than giving MAC extra context.
Critical weaknesses
  1. I think the experiments could be much strengthened if there is ablation that drops the affect fields but keeps the behavioral implication, and I would really appreciate if there could be a baseline where the semantic backbone is simply told to retain action implications of user concerns in its N + M tokens.
  2. The paper does not say how the 200 test scenarios or 100K training rows were generated, how affect was injected into the public benchmarks, or what a 16-token seven-field card looks like.
Questions for authors
  1. Did you compare the training effectiveness by testing the untrained Qwen3.5-9B with MAC prompting with the trained checkpoint?
  2. Which model serves as the LLM judge on MAC-Bench, Is it one of the evaluated models?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.