acceptodds
Under review as a conference paper at ICLR 2027

MUSE: A Multilevel User Feedback Learning Framework for Self-Enhancement in Personalized Multimodal Agents

Abstract

Multimodal agents must produce reliable outputs across modalities and adapt to individual users. A single overall score, however, gives little guidance on whether an error stems from one modality, a conflict between modalities, or a mismatch with user preferences. We introduce MUSE, a Multilevel User Feedback Learning Framework for Self-Enhancing Personalized Multimodal Agents. Within an interaction, Fine-Grained Unimodal Verification (FUV) identifies local defects and constraint violations. Cross-Modal Collaborative Verification (CCV) checks those findings against the combined evidence and identifies conflicts to guide revision. Across interactions, Feedback-Driven Personalization Verification (FPV) learns how verified output properties relate to each user's satisfaction. It updates user-specific adapters from feedback while the task models remain frozen. We evaluate each method on 2,709 instructions completed by 34 participants, yielding 92,106 participant–instruction encounters across 18 task categories and text, visual, and audio inputs. Experiments show that hierarchical verification and feedback-driven adaptation improve multimodal reliability, correction effectiveness, and preference-aligned personalization.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Agentic Systems, Multimodal, LLM-based Agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_f45LOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MUSE, a multilevel verification and feedback learning framework for personalized multimodal agents operating across two timescales. Within an individual interaction, Fine-Grained Unimodal Verification (FUV) localizes modality-specific errors using hybrid specialist tools and LLM verifiers, while Cross-Modal Collaborative Verification (CCV) validates findings through an adversarial rebuttal and multi-source adjudication protocol to resolve cross-modal conflicts. Across interactions, Feedback-Driven Personalization Verification (FPV) maps structured verification evidence and user feedback onto a lightweight per-user bottleneck adapter (98K parameters) to predict personalized satisfaction without updating the base multimodal models. Empirical results demonstrate improvements across goal fulfillment and preference alignment compared to direct LLMs and tool-based agent baselines.

Critical strengths

(1) Exceptionally rigorous empirical scale and longitudinal evaluation: Conducting a longitudinal study across 34 real participants over 18 diverse multimodal task categories is an impressive and nice effort in the agent literature. The evaluation includes direct comparisons against strong proprietary foundation models (GPT-4o, Gemini 3.1 Pro) and representative agent frameworks. (2) Principled separation of objective correctness and subjective preference: Decoupling objective unimodal/cross-modal verification (FUV and CCV) from subjective user preference adaptation (FPV) is conceptually sound. (3) Well-designed multi-timescale collaborative framework: Integrating immediate intra-interaction verification with lifelong cross-interaction personalization offers a structured, plug-and-play solution that improves reliability.

Critical weaknesses

There are several limitations that need clarification: (1) Substantial inference latency and computational overhead: As transparently detailed in Table 64, MUSE requires an average of 10.21 tool calls and 24.3 seconds per encounter, compared to 1.0 call and 6.2 seconds for direct backbone inference. The serial evaluation time is estimated at over 620 hours. While the revision count decreases over interactions, the cumulative API cost and multi-agent coordination latency remain a major practical barrier for real-time interactive or production deployments. (2) Heavy reliance on complex engineering heuristics: The framework functions primarily as an intricate orchestration pipeline combining dozens of prompts, 468 registered capability/verification interfaces, and heterogeneous heuristics. While empirically effective, the core contribution leans heavily toward complex system integration rather than fundamental algorithmic or representational advances, which raises concerns regarding maintainability and reproducibility. (3) Lack of validation on external benchmarks: Although the 18-task evaluation is extensive in scale, it relies exclusively on a newly custom-built benchmark. Without evaluating the unimodal and cross-modal verification modules on widely recognized, standardized community benchmarks , it is difficult to determine whether the observed gains generalize beyond the specific setup.

Questions for authors

(1) Can the substantial latency (24.3s and 10.21 calls per encounter) be mitigated through parallel execution or other strategies? (2) How sensitive is the complex pipeline to the tuning of its empirical heuristics, and how does the system prevent cascading errors from upstream verifiers? (3) Can the core verification modules (FUV and CCV) be evaluated on recognized public benchmarks such as Video-MME or PersonaMMEto demonstrate cross-domain generalizability?

Reviewer_QygEOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes MUSE, a framework for verifying and revising multimodal agent outputs while adapting to user preferences. It combines modality-specific verification, cross-modal adjudication, and a user-specific satisfaction predictor updated from historical feedback. Experiments on a custom benchmark covering 18 task categories report improvements in output quality and preference alignment.

Critical strengths
  • Developing feedback mechanisms that improve outputs across modalities and adapt to user preferences is a practical research problem.
Critical weaknesses
  • The paper has significant clarity issues. The concrete data sources, collection procedure, and evaluation model configuration remain insufficiently specified, making it difficult to assess benchmark reproducibility and identify generalizable findings.

  • The paper does not adequately position its contribution relative to existing multimodal judge and reward models [1–4] or provide empirical comparisons where applicable.

    [1] Wang, Xiyao, et al. "Llava-critic-r1: Your critic model is secretly a strong policy model." arXiv preprint arXiv:2509.00676 (2025).

    [2] Xiong, Tianyi, et al. "LLaVA-critic: Learning to evaluate multimodal models." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.

    [3] Jin, Zhuoran, et al. "Omni-reward: Towards generalist omni-modal reward modeling with free-form preferences." International Conference on Learning Representations. Vol. 2026. 2026.

    [4] Wang, Yibin, et al. "Unified multimodal chain-of-thought reward model through reinforcement fine-tuning." Advances in neural information processing systems 38 (2026): 159130-159157.

  • Comparisons with other agent frameworks are limited to Qwen3.5-Omni, although Table 2 evaluates five backbones. The paper does not systematically analyze which errors are corrected, when revisions introduce new errors, or how these behaviors vary across backbones. Such analysis and comparisons with stronger backbones under matched budgets would help distinguish the framework’s contribution from base-model capability.

  • The comparisons do not isolate the value of the verification hierarchy from additional inference. MUSE uses 10.21 calls per encounter versus 2.88 for Global Judge + Regeneration (Table 64). Shared revision caps do not equalize this budget. Stronger global critique or best-of-N selection at comparable token cost and latency is needed to justify the added complexity.

  • Citation formatting is frequently incorrect.

Questions for authors

Please refer to the weaknesses above.

Reviewer_sbTHOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

MUSE verifies and revises multimodal outputs while learning from a user's past feedback. Fine-Grained Unimodal Verification (FUV) identifies local problems through modality-specific prompts and specialist tools. Cross-Modal Collaborative Verification (CCV) checks these findings against the source evidence, incorporating a response from the task tool and further verifier reviews. Feedback-Driven Personalization Verification (FPV) predicts satisfaction from the request, output summary, CCV evidence, and compact user memory. FPV has a frozen BERT encoder, shared attention and prediction modules, and a separate adapter for each user. Only that user's adapter is updated online; task models remain frozen (§§3.3–3.5; Appendix A.2).

The scheduler combines four calibrated signals and accepts an output when the score reaches 0.75. Otherwise, it requests revision, with at most two revision rounds by default. The default uses neither a hard gate nor a cold-start mask. A separate collection of 800 development examples initializes FPV and fits the calibration maps. Evaluation covers 2,709 instructions in 18 tasks and 34 participants, or 92,106 encounters per method. Compared methods receive the same prior answer–feedback history. An LLM proposes scores for each new output, which participants confirm or revise. GFS, EIC, and MFC apply to all instructions; PAS applies to the 1,340 personalized instructions. Results are macro-averaged over tasks (§4; Appendix B.3).

Critical strengths
  1. The ablations are relevant to the proposed design. Table 3 reports EIC of 8.41 without FUV, compared with 8.83 for full MUSE, and MFC of 8.18 without CCV, compared with 8.72. The row labelled “MUSE w/o FPV” has PAS of 7.92 versus 8.59 for full MUSE, although its definition needs clarification below. The differences are consistent with the modules' intended roles, but each ablation measures an effect on the full workflow. The global-judge regeneration baseline and the single-verifier comparison in Table 56 provide useful additional controls.

Under the five-round cap in Figure 4 (p9), PAS gains rise from 0.08 for direct regeneration to 0.19 for global-judge regeneration, 0.24 for FUV-guided correction, 0.36 for FUV+CCV, and 0.72 for MUSE. Mean revision rounds fall in the same order: 2.84, 2.43, 2.06, 1.73, and 1.41. This pattern supports targeted correction, although fewer rounds do not imply lower total computation.

  1. The gains are spread across tasks, and the personalization tests go beyond average satisfaction. Figure 6 (p47) shows improvements over direct Qwen3.5-Omni in all 72 task–metric cells, ranging from 0.62 to 1.23. In Table 53, per-user adapters reach PAS of 8.59 versus 7.92 for a frozen global predictor. Table 55 tests opposing user preferences more directly: preference-discrimination accuracy rises from 60.6% to 86.2%.

The feedback tests are also useful. In Figure 7 (p51), MUSE retains PAS of 8.18 with 10% of feedback and 8.17 when 40% of feedback labels are perturbed. Naive adaptation falls to 7.78 in the latter setting, below the frozen predictor's 7.92. These results support the complete update procedure under the tested perturbations. Figure 5 (p42) varies the available history within a fixed set of 168 interactions, giving a controlled cold-start comparison for the supplementary masked-FPV policy.

  1. The paper makes several evaluation choices explicit. The 800-example collection is explicitly described as development data, not an independent calibration test set. Historical feedback stays attached to the answer that received it, and scores for new benchmark outputs are kept separate from FPV training labels. The smaller supplementary panels and the experts' overlapping roles are disclosed. Appendices A and C specify the model, memory, and update settings in detail.
Critical weaknesses
  1. Human scoring, supplementary comparisons, and statistical uncertainty need clearer reporting. The paper supplies rubrics and describes participant confirmation, but does not identify the benchmark evaluator's model or give its exact prompt. It also omits how often participants changed the proposed scores, how large those changes were, how long the study lasted, and how much time participants spent reviewing outputs. Method blinding and presentation order are unspecified.

This matters given the workload. For the ten methods in Table 1, the stated protocol implies 27,090 outputs and 94,670 applicable metric scores per participant: 10 × (3 × 2,709 + 1,340). The missing details make it hard to assess how thoroughly the preliminary scores were checked.

The supplementary results raise a separate comparability issue. Appendix C.1 (p43) says that main results use all 34 participants, while newly run appendix scalar-score studies use one expert and 3–6 students. Results reused from the main evaluation retain its original panel. Each comparison should identify its participants and encounter counts, including Figures 5 and 7, so that differences between configurations can be interpreted on a common basis.

Performance differences are reported without confidence intervals. Paired estimates should preserve task-macro weighting and account for repeated users, shared instructions, and chronological dependence. Uncertainty is especially important for the small differences between supplementary configurations and the adaptation curves.

  1. The controls do not fully separate the design's contribution from extra computation or prompted preferences. The agent baselines share the backbone, tool backends, decoding limits, and maximum revision budget, but their actual computation differs. The paper acknowledges this. A comparison with a strong refinement or sampling-and-selection baseline at the same compute budget would show whether MUSE makes better use of that budget.

Figure 8 (p60) is informative about MUSE's own settings: direct inference scores 7.87 at 1.0× token cost; K=2 with one revision round scores 8.59 at 2.8×; K=4 with two rounds scores 8.75 at 3.6×; all verifiers with two rounds score 8.70 at 5.7×. It supports selective verification among these settings, but does not answer the comparison above. The 8.59-to-8.75 increase also changes both verifier count and revision budget.

The personalization control needs a clearer definition. Table 3's “MUSE w/o FPV” and Table 53's frozen “Global Predictor” have identical displayed scores on every metric. It is unclear whether the former removes the satisfaction signal or retains a frozen predictor. Appendix A.4 already puts user preferences into revision prompts, so it also matters whether these preferences and the stopping rule remain the same in the ablation. Clarifying this would establish whether an existing comparison separates learned satisfaction prediction from simply including preferences in the prompts.

  1. The history source is unclear, and evidence for generalization to new users is limited. Appendix E.3 (p63) keeps the prior answer–feedback records identical across methods. This controls the information available to each method, but the paper should identify which systems produced those historical answers and how their verification states were obtained. Learning from this common history is different from learning from feedback on each method's own successive outputs.

Generalization to new users remains open, as the conclusion acknowledges. The participant pool consists of five experts and 29 students, and the number of users behind the 800 development examples is unspecified. Any overlap with the 34 benchmark participants should be reported. These details matter when assessing how much the results depend on the users involved in development.

  1. Average revision gains do not show how often correction introduces errors. Figure 4 reports average improvements and revision counts, while Tables 61–63 examine decision accuracy and false acceptance for several policies. The paper does not report how often revision repairs a failed answer or damages an initially acceptable one. Nor does it report false rejection separately. A paired audit of a representative set of outputs would address this directly. For the default policy, acceptance by round and the fraction returned below threshold at the revision cap would also be useful. These results should distinguish the two-round default from Figure 4's five-round study.
Questions for authors
  1. Which model and prompt produced the preliminary benchmark scores? How often and by how much did participants change them? Please also report study duration, typical review time, and any method blinding or order controls.
  2. For the key supplementary comparisons, including Figures 5 and 7, which results were reused and which were newly evaluated? Please give participant and encounter counts, and paired confidence intervals for the main performance differences.
  3. What exactly distinguishes “MUSE w/o FPV” from the frozen “Global Predictor”? In each case, are user preferences still supplied in prompts, is a satisfaction score used, and does the stopping rule change?
  4. Which systems generated the shared historical answers and their verification states? How many users contributed the 800 development examples, and do they overlap with benchmark participants?
  5. Can a representative audit quantify repaired failures, answers damaged by revision, and false rejections, with clear denominators? Please also report acceptance by round and how often the default reaches its revision cap below threshold.
  6. How does MUSE compare with a strong refinement or sampling-and-selection baseline at the same compute budget and on the same evaluation set? The budget should include scoring and selection costs.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.