acceptodds
Under review as a conference paper at ICLR 2027

From Preference to Action: A Benchmark for Continual Preference Memory in Multimodal Agents

Abstract

Multimodal agents must learn user preferences from repeated feedback and apply them to future tasks, yet accurate preference recall does not ensure preference-consistent generation. We introduce Pref2Act, a benchmark for continual preference memory that jointly evaluates preference understanding and personalized image generation. We use fine-grained user profiles to construct multimodal histories that interleave dialogue, image generation, and image editing. Throughout these interactions, user feedback progressively reveals, reinforces, and updates preferences. Its evaluation covers explicit fact retrieval, multi-turn information integration, implicit preference inference, while repeated anchor questions measure how agents' understanding of users evolves as interactions accumulate. We evaluate representative multimodal agents under different memory configurations to examine preference retention, adaptation, and application across scenarios. Our results show that current models struggle to retain preferences implicitly revealed through user feedback and do not consistently apply even correctly identified preferences to subsequent image generation.

Primary area:
datasets and benchmarks
Keywords:
Continual Preference Memory, Multimodal Agents, Personalized Image Generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_hdfsOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PREF2ACT, a benchmark for evaluating continual preference memory in multimodal agents. The benchmark contains six personas and long-horizon multimodal interaction histories that include dialogue, image generation and editing. It evaluates agents using cumulative memory questions and repeated anchor questions that track preference acquisition and retention over time. Experiments show that existing methods struggle to integrate distributed feedback, do not consistently improve as interaction histories grow, and often fail to retain scenario-specific preferences. The results also reveal a knowing–doing gap, which means agents may correctly identify relevant preferences but still fail to generate or edit corresponding images.

Critical strengths
  1. This work innovatively integrates multimodal memory, personalized understanding, and personalized generation into the PREF2ACT benchmark.

  2. The knowing–doing gap revealed by the experiments is insightful, as it highlights the difficulty current multimodal agents face in translating identified user preferences into preference-consistent visual outputs.

Critical weaknesses

The paper considers a variety of memory methods, but their implementation details are not sufficiently clear. Please refer to the “Questions for the Authors” for further details.

Questions for authors
  1. The paper compares a broad range of memory methods, but their downstream performance can be substantially affected by the quality of the retrieved information. It would be helpful if the authors could report retrieval-level metrics such as supporting-evidence precision. Specifically, the findings for RQ2 may arise from different factors, including inaccurate retrieval or failure to integrate multiple pieces of evidence. A more fine-grained analysis would help identify the main causes of the observed acquisition and retention failures.

  2. The paper evaluates seven textual memory methods and five multimodal memory methods, but it remains unclear what visual information (such as image captions) is available to the textual memory methods. Clarifying how historical images are organized for textual memory methods would make the comparison between textual and multimodal memory configurations easier to interpret.

  3. Several evaluated memory methods, such as Mem0 and MemoryOS, rely heavily on LLM calls. However, the auxiliary LLMs used for these operations are not clearly specified. Moreover, processing all the sections in a single batch may produce different results from incrementally updating memory section by section. It is also unclear whether later checkpoints continue from the previous memory state or rebuild memory from the complete history. In addition, different memory methods have memory items with different levels of details. Therefore, K=10K=10 does not necessarily provide an equal amount of information. It would be better if the authors could explain more implementation details.

  4. Preference changes and unrelated-topic interruptions are common in realistic interactions. Although the paper briefly gives a corresponding statement, the frequencies of these events are not reported. Providing additional statistics would improve the clarity and realism assessment of the benchmark.

  5. It would also be useful if the authors could provide evaluation results for more recent unified multimodal models, such as SenseNova_U1_8B_MoT_Infographic-V3.

5 comments
AuthorsOpenReview ↗

[1/2] Author response

Q1: Retrieval Quality and Evidence Integration

Thank you for this valuable suggestion. It motivated us to include this experiment and its discussion in the main text, in Section 4.3 (Figure 6 and Obs. 6)

We compare retrieval recall and task scores across sections using Qwen3-VL-8B with UniversalRAG.

  • On preference induction questions, scores remain low despite high recall. Recall exceeds 65% across all sections and exceeds 85% for global and personal preference induction, while scores remain below 40. This suggests that retrieving relevant evidence does not ensure successful preference induction. Agents still struggle to integrate multiple pieces of evidence into explicit preferences.

  • On fine-grained recognition questions, scores generally follow changes in recall, pointing to retrieval as a key limitation. These results highlight the need to strengthen both retrieval and evidence integration in current memory methods for different tasks.

Q2: Visual Information for Textual Memory Methods

Thank you for pointing out the missing details in our description. We have added these details in Appendix C.3.1.

We used Qwen3-VL-8B to caption all generated images. This model has shown competitive captioning performance in prior research (Liu et al.[1]). Textual memory methods receive these captions alongside the dialogue text. We acknowledge that the captions may omit or misrepresent details relevant to the six global preference dimensions and various scenario preferences. However, overly detailed prompts could introduce human bias by emphasizing selected visual attributes. We therefore consider a simple, general prompt a more neutral choice for comparison. The prompt used for image captioning is as follows:

You are an image captioning assistant. Generate a detailed, objective caption for the provided image. Describe the main subjects, their visible attributes and actions, the setting, and key spatial relationships. Base the caption only on visible evidence. Do not speculate, make subjective judgments, or invent details. Return only the caption in a single coherent paragraph.

[1] Liu Z, Wang H, Zhang Z. CAPEval: A Decoupled Caption Evaluation across Understanding and Generation[J]. arXiv preprint arXiv:2608.02589, 2026.

Q3: Memory Implementation Details

We will clarify the relevant implementation details. We have added these details in Appendix C.3.1.

  • Following Mem-Gallery, we use the corresponding experimental backbone for internal MLLM calls required by each memory baseline.
  • After each section, we update the memory with the new interactions and then evaluate the agent. We keep the memory across sections, so each checkpoint uses memory built from all interactions observed so far.
  • Following Mem-Gallery [2], we use K=10 for memory retrieval. All methods additionally use a 4096 token budget for memory input. We also find that, except for Full (Text) and Full (MM), all memory inputs fit within this budget, so these memory inputs are not truncated. We consider this a fair comparison under a shared input budget.

[2] Bei Y, Wei T, Ning X, et al. Mem-gallery: Benchmarking multimodal long-term conversational memory for mllm agents[C]. Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics. 2026: 40750-40784.

AuthorsOpenReview ↗

[2/2] Author response

Q4: Details of Preference Changes and Topic Interruptions

Regarding unrelated-topic interruptions, each section contains 16–20 groups, including 3–4 groups on other scenarios as distractions. These unrelated-topic interruptions account for approximately 20% of the groups in each section. These comprise 2 image generation groups and 1–2 understanding groups, each also containing 3–10 turns. The remaining groups focus on the section’s main scenario.

Regarding preference changes, we clarify that these mainly concern differences across subscenarios. Most subscenarios within a scenario share a common preference. When an exception is introduced, it is limited to only one subscenario. For example, within Home & Daily Life, the preferred interior style is modern minimalist for Living Room, Bathroom, and most other subscenarios, but neo-Chinese for Bedroom subscenarios. This design assesses both shared preferences and distinctions across subscenarios.

Q5: More Unified Multimodal Models

Thank you for suggesting this model. SenseNova_U1_8B_MoT_Infographic-V3 is a valuable addition to our unified multimodal baselines. Given the limited time, we have so far evaluated it on one persona for Full (Text), NaiveRAG, Mem0, Full (MM), MuRAG, UniversalRAG. We report these preliminary results below.

MemoryQ1Q2Q3-Inf.Q3-Gen.Anchor Avg.
Full (Text)26.4226.1935.3870.3330.12
NaiveRAG51.7324.6138.2474.5847.85
Mem057.9429.3633.8076.1958.41
Full (MM)34.8326.7729.5078.3938.65
MuRAG61.8630.5344.7485.6652.72
UniversalRAG60.3925.2046.1486.8855.23

These results suggest that memory methods remain a larger bottleneck than differences between backbone models. We therefore see improving retrieval and preference summarization as a priority for future memory methods.

We have not added them to the PDF because the evaluation is still incomplete. We will continue the evaluations during the rebuttal period and share as many additional results as possible in this response.

AuthorsOpenReview ↗

Additional Experimental Results

We have now completed the evaluation of the remaining memory methods on Persona 1 and report the additional results below.

MemoryQ1Q2Q3-Inf.Q3-Gen.Anchor Avg.
Generative Agents38.620.432.771.840.5
Reflexion25.818.326.969.429.7
A-Mem54.715.831.675.255.8
MemoryOS56.312.524.170.652.4
NGM53.524.834.279.750.9
AUGUSTUS52.122.729.881.351.6

We continue to find that memory methods appear to be a larger bottleneck than the choice of backbone model. Improving retrieval and preference summarization therefore remains an important direction for future work.

Please let us know if you have any further concerns or suggestions.

Reviewer_T8QCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces Pref2Act, a benchmark for continual preference memory in multimodal agents. It evaluates preference understanding and personalized image generation over long-term multimodal interaction histories. The authors evaluate five agent configurations with twelve memory methods and identify a knowing–doing gap between preference understanding and generation.

Critical strengths
  1. Evaluating whether agents can apply remembered user preferences to future tasks is practically relevant.

  2. The benchmark considers implicit preferences, long-term retention, and scenario-dependent preferences. The experiments cover multiple agents and memory methods, providing useful empirical observations.

Critical weaknesses

Questionable necessity and validity of joint image-generation evaluation.

The paper's main motivation is that existing benchmarks do not jointly evaluate preference understanding and image generation. However, it is unclear why evaluating final generated images is necessary for assessing an agent's continual preference memory.

If the goal is to evaluate whether an agent correctly applies remembered preferences, this could be assessed more directly by examining whether the relevant preferences are correctly incorporated into the instructions provided to the image generator.

Evaluating final images introduces additional uncertainty from the image generator's capabilities. Consequently, the knowing–doing gap observed in Figure 5 may reflect generation failures rather than failures of preference memory or application. The paper does not sufficiently disentangle these factors or demonstrate the additional value of joint evaluation.

Questions for authors

Given that the benchmark includes only six synthetic personas, is the diversity of user preferences sufficiently representative?

3 comments

Need for Joint Evaluation

Thank you for these comments. Our goal with the PREF2ACT benchmark is to assess preference learning and application in long-term multimodal interactions. It simulates everyday user–agent conversations that combine visual understanding and image generation. Users reveal preferences gradually and expect agents to apply them to later requests without repeated reminders. We therefore evaluate both preference understanding and whether these preferences are realized in visual generation outputs.

Furthermore, correctly understanding a preference does not guarantee that it will be applied. We therefore evaluate preference understanding and image generation separately to reveal this gap.

Evaluating Generator Prompts

The final prompt contains the current question and the memory output. These memory contexts are very long, exceeding 3,000 tokens. Full-context and basic retrieval methods provide original dialogues. A-Mem, Gen. Agents, and MemoryOS combine interaction records with derived descriptions. Including relevant evidence in the prompt does not guarantee correct generation, as preferences are often implicit in the retrieved dialogues. The agent model still needs to combine this information and identify the applicable preferences.

Relevant evidence in the prompt does not guarantee correct understanding or application. Figure 6 and Obs. 6 in Section 4.3 show low induction scores despite high recall. Recall exceeds 85% for global and personal preference induction, while scores remain below 40. We therefore evaluate both preference understanding and final images to check whether the preferences are understood and actually applied.

Impact of Generator Capabilities

Thank you for pointing this out. These controls are necessary, and we now include them in ** Appendix D.3, Table 11** to eliminate uncertainty from image generators’ capabilities. We provide explicit reference preferences directly in generation prompts, rather than the corresponding dialogues. Nearly all models score close to 100, showing that they can generate preference-consistent images when preferences are clearly specified.

ModelNo historyExplicit pref.Methods avg.
Janus-Pro-7B3.488.455.8
BAGEL-7B-MoT6.394.870.1
Qwen3-VL-8B + Qwen-Image5.396.570.2
Gemini-3.6-Flash + Gemini 3 Pro Image7.598.875.7
GPT-5.4 + GPT Image 26.897.381.3

Diversity of User Preferences

Regarding diversity, we acknowledge that six personas cannot represent all real-world users or preferences. However, our personas vary in nationality, age, gender, and occupation. Each has approximately 30 global visual preferences and 100 scenario-specific preferences. This provides diversity for examining trends in preference memory.

We have added this clarification to Appendix F to make the scope of our findings clearer.

Dear Reviewer T8QC,

Thank you for your helpful comments. Our response explains the value of jointly evaluating preference understanding and image generation. We have also added relevant experiments and clarifications in Appendix D.3 and F.

With the discussion period coming to a close, could you kindly share whether these responses resolve your main concerns? We would welcome any remaining questions and the opportunity to address them.

Thank you for your time.

Dear Reviewer T8QC,

Thank you again for your time and thoughtful feedback.

As the discussion deadline approaches, we would appreciate hearing whether our rebuttal has addressed your concerns or whether further clarification would be helpful. If your concerns have been resolved, we would be grateful if you could reassess our submission in light of the responses and additional experiments.

Thank you for your consideration.

Reviewer_6MCKOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces PREF2ACT, a benchmark for evaluating continual preference memory in multimodal agents. It uses interaction histories from six constructed personas and combines preference-understanding questions with personalized image generation and editing tasks. Experiments compare five agent configurations and twelve memory settings, with analyses of preference acquisition, retention and the relationship between preference understanding and generation performance.

Critical strengths
  • The benchmark tests whether remembered preferences affect generated and edited images. The generation requests omit preference constraints that must be recovered from the preceding interactions.
  • The fixed-cohort analysis tracks preferences from their first exposure through later scenarios. This gives a clearer retention measure than changes in the aggregate anchor score.
  • The five-agent, twelve-memory comparison covers a range of systems. Annotated-evidence tests support question answerability, and the sampled human judgments agree closely with model scores.
Critical weaknesses
  • The global acquisition anchors have an unclear reference target. Appendix A.3.4 asks for preferences explicitly emphasized at least twice so far. Figure 4 nevertheless allows scores above the evidence-supported curve by guessing unrevealed preferences. The paper does not give checkpoint-specific gold sets or explain that curve's construction. If unrevealed labels earn credit, the score mixes acquisition from observed evidence with prediction of a hidden profile.
  • The 8.36%–39.74% knowing–doing statistic is not sufficiently defined. Appendix D.2 gives no binary thresholds or joint counts for correct understanding and failed generation, and does not say whether failure must concern a correctly identified preference. Figure 5 contains weather mismatches, but its generation judgments also involve several other attributes. Task-level co-occurrence and failure on the same understood criterion are different measurements. Appendix E.2 does not state how many Q3 validation samples contain generated images.
  • The effective inputs to Full (MM) are unspecified. Appendix B.3 provides the available dialogue and images directly; histories contain about 48 images per section. There are no per-model input budgets, retained image/token counts or overflow policies. It is unclear whether Full (MM) receives the complete history at every checkpoint.
  • The generation scores do not separate the effect of the user's history from the generator's default tendencies. Table 2 has no no-history or contrasting-persona generation condition. Common aesthetic choices may already be likely outputs of the generator, so absolute preference-satisfaction scores do not measure how much the intended persona's history contributes.
Questions for authors
  1. Please work through one global anchor at C1 and C4, showing the gold sets, the labels emphasized at least twice by each checkpoint, and the evidence-supported curve calculation. Can unrevealed labels earn credit? If so, do the acquisition results hold when gold sets follow the question's eligibility rule?
  2. Please give the thresholds and joint counts used for Appendix D.2. Does a failed generation have to violate a preference the agent correctly identified, or can failure on another criterion also count? For Appendix E.2, report how many Q3 samples contain generated images and their judge/human agreement.
  3. How many images and tokens reach each Full (MM) agent at the first and final checkpoints? Please specify image selection, resizing/token reduction and any overflow policy.
  4. With the current request and generator fixed, how do generation scores change when the persona history is removed? For comparable scenarios with contrasting preferences, do generated attributes follow the new persona when histories are swapped?
5 comments
AuthorsOpenReview ↗

[1/2] Author response

W1: Anchor Gold Sets and Evidence-Supported Curve.

Thank you for this constructive suggestion. We apologize for the unclear wording and would like to further clarify the gold sets and the evidence-supported curve in Figure 4.

Global and scenario preferences are revealed gradually over four sections. A preference counts as exposed only after the user has explicitly emphasized it at least twice. Temporary preferences mentioned only once serve as distractors. (For example, suppose a persona has 30 global preferences. By the end of the first section, only 24 have been explicitly emphasized at least twice. The gold set at this checkpoint therefore contains these 24 preferences.)For global preferences, cumulative exposure is about 80%, 95%, 100%, and 100% across the four sections. For scenario preferences, it is about 30%, 60%, 90%, and 100%. We ask the same anchor questions after each section as fixed probes to track how preference identification evolves as the interaction history grows. Some questions cannot yet be answered at early checkpoints. The evidence-supported curve shows the score supported by the history available so far.

We calculate the evidence-supported score for the 3 question types as follows:

  1. For single-choice questions, we assign 100 points if the required evidence has been exposed. Otherwise, we assign 0.

  2. For multi-select questions, we calculate:

BMS=Number of correct options with exposed evidenceTotal number of correct options×100.B_{\mathrm{MS}}=\frac{\text{Number of correct options with exposed evidence}}{\text{Total number of correct options}}\times100.

  1. For short-answer questions, we calculate:

BSA=Number of scoring points with exposed evidenceTotal number of scoring points×100.B_{\mathrm{SA}}=\frac{\text{Number of scoring points with exposed evidence}}{\text{Total number of scoring points}}\times100.

Finally, we average these scores across the questions in each anchor category.

The evidence-supported curve serves as a reference rather than a strict upper bound on model scores. We acknowledge that models may exceed it by guessing or inferring preferences before explicit exposure, particularly in multiple-choice questions. We therefore include fine-grained preference questions in short-answer and multi-select formats for comparison, as shown in the second and third columns of Figure 4. Our analysis focuses on how preference identification evolves across checkpoints. In principle, accumulating relevant feedback should help agents verify these predictions and make more accurate preference judgments. Each persona has 28–32 global and 92–118 scenario-specific preferences. We therefore report their exposure proportions at each checkpoint. We provided the full annotations of gold sets across sections with the benchmark release. We have added this clarification to the caption of Figure 4 in the main text.

W2: Knowing–Doing Gap Definition and Validation.

Thank you for raising this point. We apologize for the ambiguity in our wording. By “failed generations” in Appendix D.2, we refer to failures on individual preference scoring points within an image, rather than failures of the image as a whole. Specifically, each image-generation task ii contains NiN_i equally weighted scoring points, based on one preference supported by the observed history. These include about 3–5 scoring points for scenario-specific preferences and 6–15 for global preferences. We check each scoring point in the generated image. We also ask a separate question for each scoring point to check understanding of the same preference (Figure 5). The evaluation procedure and judge prompts are in Appendix C.3.

We have clarified the wording and added a more detailed explanation in Appendix D.2: Let gij=1g_{ij}=1 if the generated image i satisfies preference jj, and 00 otherwise. Likewise, let uij=1u_{ij}=1 if the agent correctly answers the question about that same preference, and 00 otherwise.

For MM generation tasks, the upper panel in Figure 7(a), reports the average image-generation score, calculated as follows:

Sgen=1M∑i=1M1Ni∑j=1Nigij⋅100%.S_{\mathrm{gen}} = \frac{1}{M} \sum_{i=1}^{M} \frac{1}{N_i} \sum_{j=1}^{N_i} g_{ij}\cdot100\%.

Figure 7(b) reports the proportion of unsatisfied preferences that the agent correctly understood, calculated as follows:

R=∑i=1M∑j=1Niuij(1−gij)∑i=1M∑j=1Ni(1−gij)⋅100%.R = \frac{\sum_{i=1}^{M}\sum_{j=1}^{N_i}u_{ij}(1-g_{ij})}{\sum_{i=1}^{M}\sum_{j=1}^{N_i}(1-g_{ij})}\cdot100\%.

The numerator counts cases where a preference is correctly understood but not satisfied in the generated image; the denominator counts all unsatisfied preference scoring points in the generated image.

Each image is evaluated on multiple preferences (scoring points). We therefore report proportions at the preference level instead of the number of failed images.

Of the 30 Q3 validation samples in Appendix E.2, 10 are image-generation questions.

AuthorsOpenReview ↗

[2/2] Author response

W3: Full (MM) Inputs and Limits

Following Mem-Gallery [1], we limit memory input to 4,096 tokens for all methods at every checkpoint. For Full (MM), this includes text and images, with each image counted as about 256 tokens. Except for Full (Text) and Full (MM), all memory inputs fit within this budget, so these memory inputs are not truncated. We have added these details to Appendix C.1.

[1] Bei Y, Wei T, Ning X, et al. Mem-gallery: Benchmarking multimodal long-term conversational memory for mllm agents[C]//Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics. 2026: 40750-40784.

W4: Effects of User History and Model Bias

Thank you for pointing this out. These controls are necessary, and we now include them in Appendix D.3, Table 11.
The results show that all models score below 10 without history, while nearly all score close to 100 when given explicit preferences. Memory consistently improves over the no-history baseline, indicating that the generators’ default tendencies do not naturally align with user preferences. The high scores with explicit preferences also show that these models can distinguish and follow different preferences.

ModelNo historyExplicit pref.Methods avg.
Janus-Pro-7B3.488.455.8
BAGEL-7B-MoT6.394.870.1
Qwen3-VL-8B + Qwen-Image5.396.570.2
Gemini-3.6-Flash + Gemini 3 Pro Image7.598.875.7
GPT-5.4 + GPT Image 26.897.381.3

Thank you for the clarifications and the new generation controls. The no-history results address my concern about generator defaults, and Appendix D.2 clarifies the preference-level knowing–doing metric. My main concerns remain with the anchor evaluation and the Full (MM) baseline. It is still unclear whether the acquisition trends hold when scoring only preferences eligible at each checkpoint. The 4,096-token budget is now stated, but the truncation policy and the text and images actually retained at each checkpoint remain unclear. These issues limit my confidence in the interpretation of the reported memory comparisons, so I maintain my rating of 2 (Weak Rejection).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.