acceptodds
Under review as a conference paper at ICLR 2027

RevMMSearch: Evidence-Grounded Identity Revision for Multimodal Search Agents

15 views

Abstract

Multimodal search agents answer questions grounded in images by combining visual understanding with external retrieval and reasoning. However, visual understanding can be unreliable when resolving target identity, as mistaking the target for a visually similar entity can steer subsequent retrieval and reasoning along an incorrect trajectory. In this work, we argue that target identity should be treated as a revisable routing state, with search both conditioned on the current identity hypothesis and used to refine it. Our pilot studies further show that such errors can persist through stale evidence, constrain subsequent search, and complicate outcome-based credit assignment. Motivated by these findings, we introduce RevMMSearch, a framework that maintains competing target-identity hypotheses and actively acquires external evidence to verify and revise them throughout search. Within this framework, an entity-evidence graph links identity hypotheses to supporting or conflicting evidence and identity-dependent answer facts, dynamically updating fact applicability as the identity is revised. A unified search loop interleaves answer-fact acquisition with discriminative searches that distinguish competing hypotheses, while verifying whether retrieved evidence is attributable to the visual target. We further introduce revision-aware policy optimization, which combines final-answer rewards with evidence-based feedback on intermediate decisions and suppresses outcome reward signals that conflict with this feedback. Experiments on five benchmarks show that RevMMSearch improves average answer accuracy by 3.5% over the strongest compared multimodal search agent. Ablations and further analyses attribute these gains to reduced identity lock-in, filtering of stale evidence, and conflict-aware credit assignment.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Search Agent, Agent Reasoning, Reinforcement Learning, Agent

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_XYwKOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RevMMSearch, a framework for addressing target misidentification in multimodal search agents. It maintains an entity–evidence graph to track competing identity hypotheses and update the applicability of retrieved facts as identities change. The framework combines discriminative search with evidence attribution, while a revision-aware policy optimization objective suppresses outcome credit that conflicts with admitted local evidence judgments. Experiments across five benchmarks report an average accuracy improvement of approximately 3.5 percentage points over the strongest compared specialized search agent.

Critical strengths

Relevant motivation: The paper identifies a meaningful failure mode: factually correct information can still produce incorrect answers when it concerns a misidentified visual target.

Coherent framework: Identity revision is explicitly coupled with evidence applicability, addressing both search lock-in and the continued use of facts associated with rejected identities.

Targeted empirical analysis: Beyond the main benchmark comparisons, the paper includes component ablations and diagnostic analyses examining identity recovery, stale-evidence handling, and alternative training objectives.

Critical weaknesses

Weaknesses and Questions for the Authors

  1. Insufficient validation of audit reliability. The method prioritizes admitted local verdicts over conflicting outcome credit, but their reliability is not directly validated. Citation and abstention rules alone do not establish accuracy, particularly since the same audit interface produces both the gate and verdict. Please report human agreement, admitted-verdict error rates, coverage/abstention rates, and sensitivity to noisy feedback or alternative auditors.

  2. Unclear auditor implementation. Please specify the frozen auditor’s model, version, size, inference settings, and relationship to the policy and trajectory-generation teacher, including any weight sharing or additional training/calibration. These details are essential for reproducing the optimization signal.

  3. Unreported training-time auditing costs. Auditing workload grows with rollout count and the number of action/state-update segments. Please report audit calls, token usage, batching/caching strategies, and additional training time and compute. Compare total costs against outcome-only GRPO and additive reward, preferably under matched compute budgets. Table 6’s execution statistics do not establish training-time auditing costs; this concern is distinct from deployment-time inference overhead.

Questions for authors

See in Weaknesses

4 comments
AuthorsOpenReview ↗

Response to Reviewer XYwK

Thank you for the careful reading and constructive comments. Below, we address your concerns about the auditor. All revisions are highlighted in red in the updated manuscript.

W1&W2: Auditor Implementation and Reliability

Auditor implementation and selection

We use the official Qwen3-VL-30B-A3B weights as our auditor, with a decoding temperature of 0 and a maximum output length of 16,384 tokens. The policy model is Qwen3-VL-8B-Instruct, and the trajectory teacher is GPT-5.6-Sol. The auditor shares no weights with the policy or teacher, receives no additional training or calibration, and remains frozen throughout RL.

The auditor judges whether each action or state update is supported by the evidence available at that decision point. Action audits use the original image, question, current state, and interaction history before the action, excluding the observation returned by that action. State update audits may also use the newly returned observation. Reference answers, later observations, final answers, and outcome rewards are excluded. The auditor returns supported, unsupported, or abstain. On abstention, the original outcome credit is retained and the local reward is zero.

We randomly sampled 200 search trajectories and manually labeled their 1,926 action and state update segments using the information available at each decision point. We evaluated three candidate auditors on these same segments. Human annotators labeled each segment as supported, unsupported, or requiring abstention. Human agreement is the fraction of all segments whose audit labels match the human labels. The admitted-verdict error rate is the fraction of non-abstaining verdicts that disagree with human labels, including cases where the human label calls for abstention. Coverage is the fraction of all segments receiving a non-abstaining verdict.

AuditorHuman agreement(%)↑Admitted-verdict error rate(%) ↓Coverage(%)Abstention rates(%)
Qwen3-VL-8B-Instruct74.421.586.613.4
Qwen3-VL-30B-A3B87.89.488.711.3
GPT-5.6-Sol93.45.094.15.9

We selected Qwen3-VL-30B-A3B to balance judgment quality and auditing cost.

Sensitivity to noisy audit feedback

To test the effect of incorrect feedback on training, we compare standard GRPO, RevGRPO without added noise, and RevGRPO with an additional 10% or 20% of admitted verdicts randomly flipped. All runs use the same SFT checkpoint, training data, and number of training steps. We keep the admission gates unchanged and recompute both the conflict mask and local reward using the flipped verdicts. Final answer accuracy on MMSearch is reported below.

Training methodAdditional flip rateAnswer accuracy(%)↑
Standard GRPO—69.8
RevGRPO0%72.1
RevGRPO10%65.5
RevGRPO20%61.4

RevGRPO improves over GRPO by 2.3 percentage points without added noise, but both noisy variants fall below GRPO. Thus, audit gains depend on verdict quality: errors can corrupt local rewards and outcome-credit suppression.

W3: Auditing Costs

Thank you for the suggestion. We provide details on training-time auditing and compare estimated total costs over the same number of training steps.

Auditing cost

We use a frozen Qwen3-VL-30B-A3B on NVIDIA RTX PRO 6000 Blackwell Server Edition GPUs, with a batch size of 16 and 16 concurrent requests. Audit verdicts are cached for reuse. Training on 5,000 questions for two epochs, with five rollouts per question, produces about 50,000 trajectories and 482,000 audited segments. With an average of 1,000 input and 150 output tokens per audit, total usage is approximately 482M input and 72.3M output tokens. At 16,000 segments per hour on a single GPU, the estimated auditing cost is 30.1 GPU-hours.

Total cost comparison

All methods share the SFT checkpoint, training questions, update steps, and rollouts per question. Costs include rollout generation, rewards, auditing, and policy updates. Non-auditing RL costs ~400 GPU-hours per method; Additive Reward and RevGRPO share the same auditing workload:

  • Outcome-only GRPO: No local auditing; total cost is approximately 400 GPU-hours.
  • Additive reward: Uses the same frozen auditor and local feedback rules as RevGRPO. Auditing adds 30.1 GPU-hours, bringing the total to approximately 430.1 GPU-hours.
  • RevGRPO: Total cost is approximately 430.1 GPU-hours, including 30.1 GPU-hours for auditing, about 8% above outcome-only GRPO. Compared with additive reward, conflict suppression only changes how local feedback and outcome credit are combined and requires no additional audit calls.

At the same 430.1 GPU-hours, RevGRPO outperforms additive reward on all five benchmarks in Table 3 and surpasses outcome-only GRPO’s step-80 reward in fewer steps, as shown in Figure 6(c).

Thank you again for your constructive feedback. We welcome your further feedback and would be happy to address any remaining questions.

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer XYwK,

Thank you for your constructive feedback on our paper. As the discussion period is drawing to a close, we would like to briefly follow up to ensure our response has adequately addressed your concerns. In particular, in our response and revised manuscript, we have:

  • Clarified the auditor setup, available evidence, and abstention rules.
  • Compared three auditors against human labels on 1,926 segments and tested how noisy audit feedback affects training.
  • Reported auditing costs and compared RevGRPO with additive reward under the same compute budget.

Please let us know if any points remain unclear or require further clarification. We would be glad to address them before the deadline. If our revisions have resolved your concerns, we would greatly appreciate it if you could reconsider your assessment of our work.

Thank you very much for your time!

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer XYwK,

I hope this message finds you well.

As the discussion period is drawing to a close, with fewer than two days remaining, we wanted to kindly follow up on our rebuttal and ask whether there are any additional questions or concerns we could address. We would be grateful for any further feedback you may have, as your comments are invaluable in helping us improve our work. If our response has resolved your concerns, we would be very grateful if you could kindly consider updating your score.

Thank you very much for your time and effort in reviewing our paper.

Best regards,

The authors

AuthorsOpenReview ↗

Final Follow-up on Our Rebuttal

Dear Reviewer XYwK,

Thank you again for your valuable feedback.

With the discussion deadline approaching, we would like to kindly follow up once more. In response to your concerns, we have provided detailed auditor specifications, human validation results, noise sensitivity experiments, and training-time cost comparisons.

We hope these additions have adequately addressed your questions. We would be very grateful if you could share any remaining concerns or, if our responses have resolved them, kindly reconsider your assessment before the discussion closes.

Thank you very much for your time and consideration.

Best regards,

The authors

Reviewer_S8kEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes RevMMSearch to address incorrect entity identification in multimodal search agents. It uses an entity-evidence graph to track and revise entity hypotheses, discriminative search to verify identities, and a modified GRPO objective to improve credit assignment. The method achieves 65.9% average accuracy across five benchmarks, outperforming the strongest compared specialized search agent by 3.5 points.

Critical strengths

1.The problem is well motivated. Incorrect entity identification can mislead subsequent searches. The pilot studies provide useful evidence of this failure mode. 2.The evidence management design makes sense. Tracking which facts belong to which entities helps prevent outdated or irrelevant evidence from affecting the final answer.

Critical weaknesses

1.The novelty of the method seems somewhat limited. Maintaining entity hypotheses, verifying them through search, and updating evidence are fairly intuitive ideas. The modified GRPO objective also appears incremental. It is unclear whether the proposed complexity is necessary. 2.The evidence verification itself may be unreliable. The agent decides whether evidence supports or refutes an entity. If these judgments are wrong, the graph may still reinforce incorrect identities. The paper does not evaluate how often this happens. 3.The RL method relies on an unverified auditor. The proposed GRPO method uses an auditor to judge intermediate decisions, but its accuracy is not reported. Incorrect judgments could introduce misleading training signals.

Questions for authors

1.How does the method ensure that SUPPORT/REFUTE judgments are correct? Does the deterministic executor verify the evidence itself, or only check the graph updates? 2.How would a simpler agent that verifies entity identities through prompting perform with the same backbone, training data, and tool budget, but without the proposed graph? 3.How much of the performance improvement comes from actually correcting wrong entity predictions, rather than from the training recipe or improved search in general?

6 comments
AuthorsOpenReview ↗

Response to Reviewer S8kE (1/3)

We thank the reviewer for the careful reading, constructive feedback, and insightful questions. We are greatly encouraged by your positive feedback on our well-motivated problem formulation, informative pilot studies, and sensible entity-evidence management design. Below, we address your concerns regarding method novelty and necessity, evidence verification reliability and executor roles, comparisons with prompt-based verification and identity correction gains, and auditor reliability. All revisions are highlighted in red in the updated manuscript.

W2&Q1: Evidence Verification Reliability and Executor Roles

Thank you for the suggestion. We clarify how RevMMSearch promotes reliable SUPPORT/REFUTE judgments and distinguish the role of the policy model from that of the deterministic executor in Section 3.2 and Appendix J. We also added a manual evaluation of evidence attribution under visual ambiguity and retrieval noise in Section 4.4 (Table 4) and Appendix G.

1. Evidence-grounded attribution and reliability.

RevMMSearch improves attribution reliability by requiring the policy to ground SUPPORT/REFUTE judgments in explicit correspondence between the visual target and retrieved evidence, rather than relying on candidate mentions or retrieval relevance alone. Specifically:

  • Explicit evidence grounding: SUPPORT requires an attributable match with the visual target, while REFUTE requires an attributable inconsistency. Each judgment must identify the relevant visual cue and cite supporting evidence.
  • Conservative attribution: Evidence describing a candidate without establishing its identity is labeled DESCRIBE, while insufficient evidence is labeled UNKNOWN. Neither creates an active SUPPORT/REFUTE relation.
  • Revisable evidence relations: Competing hypotheses and conflicting evidence are retained. When subsequent evidence invalidates an earlier attribution, the corresponding relation can be deactivated through a cited update, allowing incorrect judgments to be reconsidered.

These mechanisms aim to reduce unsupported identity commitments, but do not guarantee the semantic correctness of individual judgments.

To quantify reliability, we manually selected 100 cases involving visually similar entities from MMSearch, BrowseComp-VL, and InfoSeek, and verified their target identities and evidence–identity relations. We evaluated the original retrieval results and a noisy setting in which 30% of correct results were replaced with distractors concerning visually similar but distinct entities.

We compared ReAct, MM-DeepResearch, and RevMMSearch under identical tools and interaction-round budgets. Alongside answer and identity accuracy, we report SUPPORT/REFUTE precision, measuring the fraction of judgments assigned each relation that agree with the manually verified reference. All metrics are percentages.

ConditionMethodAnswer acc. ↑Identity acc. ↑SUPPORT ↑REFUTE ↑
High similarityReAct354862.266.4
High similarityMM-DeepResearch415571.373.5
High similarityRevMMSearch577279.476.0
High similarity + retrieval noiseReAct223648.749.6
High similarity + retrieval noiseMM-DeepResearch293951.855.3
High similarity + retrieval noiseRevMMSearch466266.071.2

RevMMSearch achieves higher attribution precision and final identity and answer accuracy under both conditions. Nevertheless, retrieval noise reduces attribution precision, particularly for SUPPORT judgments. We discuss this remaining limitation in Section 4.4 and the Conclusion.

2. Role of the deterministic executor.

The policy model determines whether retrieved evidence supports or refutes a candidate relative to the visual target, providing the relation label, source, and specific matching or conflicting cues. The deterministic executor checks update structure, reference existence, and evidence dependencies. It then derives candidate states, the routing identity, and fact applicability from the accepted relations.

Importantly, the executor enforces structural and referential consistency but does not independently verify whether the cited evidence semantically supports or refutes the identity. Thus, attribution reliability depends on the evidence-grounded policy judgments evaluated above, while the executor ensures that accepted judgments are consistently reflected in the agent state.

AuthorsOpenReview ↗

Response to Reviewer S8kE (2/3)

W1: Novelty and Necessity of the Proposed Method

Thank you for the comment. Prior work has explored entity hypothesis maintenance, search-based verification, and process-level credit assignment. We have added a discussion of these similarities and differences in Appendix F (Extended Related Work). Ablations and new controlled comparisons further clarify our contributions and why the proposed components are needed.

  • Linking identity revision to fact applicability. RevMMSearch treats target identity as a revisable search routing state. Fact–source–identity dependencies allow identity changes to update both subsequent search and the applicability of historical facts. A fact can be correct for the original entity yet no longer apply after identity revision. Facts supported only through a rejected identity are therefore no longer used, while those with independent valid support under the current identity remain usable.
  • Handling conflicting credit. RevGRPO audits local decisions using decision-time evidence. When an accepted judgment conflicts with the outcome advantage, it suppresses only the outcome credit while retaining signed local supervision. This differs from simply adding process rewards or masking a segment’s entire policy update. Table 3 shows improvements over Additive Reward on all five benchmarks, averaging about 1.8 percentage points.
  • Experimental support. In Q2’s controlled comparison without additional training, RevMMSearch outperforms the strongest graph-free variant, DS+V, in both answer and final identity accuracy on all three benchmarks. This supports the value of explicit state management beyond prompt-based verification. In Table 3, removing applicability updates, discriminative search, or attribution verification reduces accuracy on all five benchmarks. Table 9 and Appendix E.2 further show that the framework retains its advantage with shared training data and standard GRPO.

Q2: Prompt-Based Identity Verification without a Graph

Thank you for the suggestion. This comparison helps assess the contribution of explicit evidence state management. Section 2.3 compares four ReAct variants using answer-oriented search (AS) or discriminative search (DS), with or without attribution verification (+V). The results show that discriminative search and verification both improve identity correction. Detailed settings are provided in Appendix C.2.

We further compare these variants with standard ReAct and RevMMSearch on MMSearch, BrowseComp-VL, and InfoSeek. All methods use the same Qwen3-VL-8B-Instruct checkpoint without additional SFT or RL, and share the same tools, token limits, and maximum interaction rounds. The variants without a graph use prompts to guide search and attribution verification, organizing evidence in a linear interaction history. We report answer accuracy (Acc.) and final entity accuracy (Ent. Acc.), both as percentages reported in Appendix C.1, Table 7.

MethodMMSearch Acc.MMSearch Ent. Acc.BC-VL Acc.BC-VL Ent. Acc.InfoSeek Acc.InfoSeek Ent. Acc.
ReAct53.464.934.042.446.256.7
ReAct + AS53.665.833.841.749.259.0
ReAct + DS54.567.634.844.350.059.5
ReAct + AS + V54.267.434.743.452.361.6
ReAct + DS + V55.468.536.346.653.962.4
RevMMSearch58.572.537.754.156.766.8
  • Prompt-based verification is effective. Adding verification improves both metrics under both search strategies. DS+V is the strongest variant without a graph, providing a competitive baseline for evaluating the full framework.

  • The full framework provides further gains. RevMMSearch outperforms DS+V on both metrics across all three benchmarks. These results support tracking identity–evidence dependencies across steps, allowing identity revisions to guide subsequent searches and update which earlier facts remain applicable.

AuthorsOpenReview ↗

Response to Reviewer S8kE (3/3)

Q3: Contribution of Identity Correction

Thank you for the suggestion. To examine the link between identity correction and answer accuracy, we report MMSearch results grouped by whether the initial identity is correct. All variants use the same experimental settings. All entries report answer accuracy (%).

MethodInitially incorrect: Acc.Initially correct: Acc.Overall Acc.
RevMMSearch51.385.472.1
w/o Discriminative Search39.090.370.3
w/o Verification43.288.670.9
  • Gains mainly reflect recovery from incorrect starting identities. The full method's advantage is concentrated on cases with an incorrect initial identity, with a larger drop when discriminative search is removed. This matches the identity recovery trend in Figure 6(a) and supports using discriminative evidence to reduce the influence of incorrect identities on later search and answering.
  • Revision benefits depend on the initial identity. When the initial identity is correct, the full method performs worse than the ablations. Thus, revision does not benefit all starting states equally. The overall gain reflects a trade-off between recovery from initial errors and losses on initially correct cases, suggesting that revision effort should match identity uncertainty.

Figure 1(b) further supports the effect of correct identity information on downstream answering. The controlled comparisons in Appendix E.2 show that framework gains cannot be explained by differences in training data or optimization alone.

W3: Auditor Reliability

Thank you for the suggestion. We added human validation and feedback-noise experiments to assess audit reliability and the impact of incorrect feedback on training.

We use a frozen Qwen3-VL-30B-A3B auditor without additional training or calibration. It uses only information available at each decision point, excluding reference answers, later observations, final answers, and outcome rewards. When evidence is insufficient, it can abstain; the original outcome credit is then retained, and the local reward is zero.

Human validation. We randomly sampled 200 search trajectories and manually labeled 1,926 action and state-update segments as supported, unsupported, or requiring abstention. We compared three candidate auditors on the same segments:

AuditorHuman agreement (%) ↑Admitted-verdict error rate (%) ↓Coverage (%)
Qwen3-VL-8B-Instruct74.421.586.6
Qwen3-VL-30B-A3B87.89.488.7
GPT-5.6-Sol93.45.094.1

Agreement is measured over all segments. The admitted-verdict error rate is the fraction of non-abstaining verdicts that disagree with human labels, including cases where humans require abstention. Coverage is the fraction of segments receiving a non-abstaining verdict.

Compared with the 8B auditor, the 30B auditor reduces errors while covering slightly more segments; GPT further improves both reliability and coverage. Auditor choice therefore affects both the quality and coverage of training feedback. We selected Qwen3-VL-30B-A3B to balance verdict quality and auditing cost during training, while recognizing that errors remain.

Impact of incorrect feedback on training. Using the same SFT initialization, training data, and number of training steps, we compared standard GRPO, original RevGRPO, and RevGRPO with 10% or 20% of admitted verdicts randomly flipped. We kept the admission gate unchanged and recomputed the conflict mask and local rewards from the flipped verdicts:

Training methodAdditional flip rateMMSearch answer accuracy (%) ↑
Standard GRPO—69.8
RevGRPO0%72.1
RevGRPO10%65.5
RevGRPO20%61.4

RevGRPO outperforms standard GRPO with the original feedback. As the flip rate increases, accuracy declines and falls below standard GRPO at both noise levels. These results show that the gains from local auditing depend on verdict quality: added errors can reverse those gains. Since audit verdicts affect both local rewards and whether outcome credit is retained or suppressed, incorrect feedback may harm optimization through both paths.

Thank you again for your constructive feedback. Your comments helped us clarify the presentation and motivated additional experiments and analyses that strengthened the paper. We believe these improvements make our work more valuable to the ICLR community. We welcome your further feedback and would be happy to address any remaining questions.

Reviewer_HDWhOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes RevMMSearch, a multimodal search framework that addresses errors caused by incorrect target identification. It maintains competing identity hypotheses through an entity-evidence graph, uses discriminative search to revise identities, and introduces revision-aware policy optimization to improve intermediate decisions. Experiments on five benchmarks show good performance. Howerver, I think this version is a drift rather than a complete essay.

Critical strengths
  1. This work addresses an important but often overlooked issue in multimodal search. The idea of revisiting target identities while keeping track of previously collected evidence is interesting and practically useful.
Critical weaknesses
  1. The improvements are not fully isolated from the effects of training data curation and post-training. A stronger baseline trained with the same data and computational budget would help clarify how much of the gain comes from the proposed framework itself.

  2. Most of the detailed ablations and analyses are limited to MMSearch. It remains unclear whether the proposed components provide similarly consistent benefits across other benchmarks, especially given the additional inference overhead.

Questions for authors
  1. It is difficult to separate the benefits of the proposed framework from those of the curated training data and post-training process. Could the authors compare against a standard search agent trained on the same SFT and RL data, using identical tools and interaction budgets?

  2. The entity-evidence graph relies on the model to determine whether retrieved evidence supports or refutes an identity. How reliable are these decisions when the visual targets are highly similar or the retrieved information is noisy? Some quantitative analysis of attribution errors would be helpful.

  3. Most of the ablation experiments are conducted on MMSearch. Do the benefits of discriminative search and evidence applicability updates generalize equally well to the other four benchmarks? It would also be interesting to understand why RevMMSearch slightly underperforms MM-DeepResearch on InfoSeek.

8 comments
AuthorsOpenReview ↗

Response to Reviewer HDWh (1/2)

We thank the reviewer for the careful reading and constructive suggestions. We are greatly encouraged by your recognition of our focus on an important yet overlooked problem, the practical value of revisiting target identities while tracking collected evidence, and our strong empirical performance across five benchmarks. Below, we address your concerns regarding manuscript completeness, isolating framework contributions from training data, the reliability of evidence attribution under ambiguity and noise, and cross-benchmark ablations and InfoSeek performance. All revisions are highlighted in red in the updated manuscript.

Summary: Manuscript Appears to Be a Draft

We appreciate the concern about presentation and completeness. To address this, we have made the following substantive revisions:

  • Framing: Rework the Abstract and Introduction to clarify failure modes and contributions.
  • Method: Formalize fact–source–identity justifications and conflict-aware credit (Sec. 3).
  • Validation: Add matched comparisons and five-benchmark ablations (Sec. 4; App. E.2).
  • Reliability: Test attribution under visual ambiguity and retrieval noise (Sec. 4.4; App. G).
  • Positioning: Expand discussion of graph and process-supervision methods (Sec. 5; App. F).

W1&Q1: Isolating Framework Contributions

Thank you for the suggestion. We added controlled comparisons with ReAct and MM-DeepResearch to separate framework gains from the effects of training data and post-training. All frameworks start from the same backbone weights and use identical tools, token limits, and maximum interaction rounds. They share RL questions and reference answers, with rollouts sampled by each policy. RevGRPO audits are excluded from the policy token budget. The results and detailed analysis are provided in Appendix E.2 (Table 9).

We compare two SFT data sources:

  • RevMMSearchData: Our shared demonstrations, reformatted for each baseline while preserving tool interactions, evidence, and decisions.
  • WorkflowData: Demonstrations generated by the same teacher on the same SFT questions using each baseline’s own workflow.

On RevMMSearchData, GRPO and RevGRPO start from the same SFT checkpoint within each framework. For baseline RevGRPO runs, we adapt local auditing to each baseline’s output format while keeping its search workflow unchanged.

IDFrameworkSFT DataTrainingMMSearchBC-VLInfoSeekAvg.
R0ReAct—No post-training53.434.046.244.5
R1ReActRevMMSearchDataSFT only60.836.455.150.8
R2ReActRevMMSearchDataSFT+GRPO62.037.356.952.1
R3ReActRevMMSearchDataSFT+RevGRPO62.437.558.652.8
R4ReActWorkflowDataSFT only61.437.657.752.2
R5ReActWorkflowDataSFT+GRPO62.138.559.853.5
M0MM-DeepResearch—No post-training57.836.152.948.9
M1MM-DeepResearchRevMMSearchDataSFT only63.840.265.056.3
M2MM-DeepResearchRevMMSearchDataSFT+GRPO65.541.368.258.3
M3MM-DeepResearchRevMMSearchDataSFT+RevGRPO65.941.869.459.0
M4MM-DeepResearchWorkflowDataSFT only64.040.068.557.5
M5MM-DeepResearchWorkflowDataSFT+GRPO66.441.369.859.2
O0RevMMSearch—No post-training58.537.756.751.0
O1RevMMSearchRevMMSearchDataSFT only68.543.268.860.2
O2RevMMSearchRevMMSearchDataSFT+GRPO69.844.068.760.8
O3RevMMSearchRevMMSearchDataSFT+RevGRPO72.145.970.862.9

Avg. is the mean accuracy across the three benchmarks.

  • Framework gains remain with shared data and the same training method. RevMMSearch outperforms both baselines on all three benchmarks. The same trend holds without additional post-training.
  • RevGRPO benefits other search workflows. Replacing GRPO with RevGRPO improves every benchmark across all three frameworks, with the largest gains in RevMMSearch.
  • WorkflowData improves the baselines but does not close the average gap. RevMMSearch retains its average lead under the same training method, although MM-DeepResearch performs better on InfoSeek under SFT + GRPO (M5 vs. O2).
AuthorsOpenReview ↗

Response to Reviewer HDWh (2/2)

Q2: Reliability of Evidence Attribution

Thank you for the suggestion. We directly evaluated evidence attribution using 100 manually selected cases with visually similar entities from InfoSeek, MMSearch, and BrowseComp-VL. We verified the target identities and reference evidence relations. We tested two settings: high similarity, using the original retrieval results, and high similarity + noise, randomly replacing 30% of correct retrieval results with results about visually similar but different entities. We have included these results in Section 4.4 and Appendix G of the revised manuscript.

We compared ReAct, MM-DeepResearch, and RevMMSearch using identical tools and interaction-round limits. Alongside final answer and identity accuracy, we report SUPPORT precision and REFUTE precision: the proportion of each type of judgment that agrees with the reference relation. For all methods, we identified these judgments from actions in their trajectories that confirmed or ruled out candidate identities. Each judgment was assessed using only information available at that decision point. Reference labels were used only for evaluation. All values are percentages.

SettingMethodAnswer acc.↑Identity acc.↑SUPPORT precision↑REFUTE precision↑
High similarityReAct354862.266.4
High similarityMM-DeepResearch415571.373.5
High similarityRevMMSearch577279.476.0
High similarity+noiseReAct223648.749.6
High similarity+noiseMM-DeepResearch293951.855.3
High similarity+noiseRevMMSearch466266.071.2
  1. Improvements in both evidence attribution and identity recognition. RevMMSearch makes more reliable support and refutation judgments and achieves better final identity and answer accuracy. These improvements in intermediate judgments and final outcomes support its effectiveness under visual ambiguity.
  2. The relative advantage remains under retrieval noise. Replacing correct results both removes useful evidence and introduces information about similar entities. All methods perform worse, but RevMMSearch retains its advantage in both attribution and final outcomes.
  3. SUPPORT judgments are more sensitive to noise. For RevMMSearch, SUPPORT precision drops more than REFUTE precision. This suggests that distinguishing valid identity support from information about similar entities remains a key challenge. The framework improves attribution reliability but does not eliminate errors.

W2&Q3: Cross-Benchmark Ablations and InfoSeek Performance

Thank you for the suggestion. We extended our ablations to the other four benchmarks using the same ablation definitions. All results below are answer accuracy. We have updated Table 3 and expanded the accompanying ablation analysis in the revised manuscript.

VariantMMSearchBC-VLFVQALiveVQAInfoSeek
RevMMSearch72.145.972.268.470.8
w/o Applicability Update70.643.969.465.767.3
w/o Discriminative Search70.343.669.565.166.8
w/o Verification70.944.269.966.167.6
Additive Reward70.244.469.166.570.0
Outcome-only GRPO69.844.068.965.468.7
SFT only68.543.267.863.568.8

Removing discriminative search or evidence applicability updates lowers accuracy on all five benchmarks, showing that both components help beyond MMSearch. Discriminative search helps distinguish competing identities, while applicability updates adjust how existing evidence is used after an identity revision. Removing attribution verification also lowers accuracy, as does replacing revision-aware optimization with additive or outcome-only rewards. These results further support the effectiveness of these designs across benchmarks.

On MMSearch, RevMMSearch uses 7.0% more model tokens per question on average than ReAct and takes 18.6% longer end to end. Answer accuracy increases from 53.4% to 72.1%, a gain of 18.7 percentage points, as detailed in Appendix E.1 and Table 8.

On InfoSeek, RevMMSearch achieves 70.8% accuracy, below MM-DeepResearch’s 72.3% in Table 2. We discuss this result from two perspectives:

  • Different training data distributions. MM-DeepResearch used InfoSeek questions to build SFT trajectories, while our training data did not include InfoSeek. This may help MM-DeepResearch better adapt to InfoSeek’s task distribution and partly explain the original performance gap.
  • Controlled comparisons and ablations support our method. In W1&Q1, both methods start from the same backbone weights and use the same tools and interaction budgets. They also share RevMMSearchData and the same RL questions and reference answers. With SFT + RevGRPO, RevMMSearch outperforms MM-DeepResearch on InfoSeek by 1.4%. The InfoSeek ablations above also support the effectiveness of discriminative search and evidence applicability updates.

Thank you again for your constructive feedback. We welcome your further feedback and would be happy to address any remaining questions.

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer HDWh,

Thank you for your constructive feedback on our paper. As the discussion period is drawing to a close, we would like to briefly follow up to ensure our response has adequately addressed your concerns. In particular, in our response and revised manuscript, we have:

  • Improved the presentation and clarified the method and contributions.
  • Added controlled comparisons with ReAct and MM-DeepResearch using shared training data.
  • Tested evidence attribution under visual ambiguity and retrieval noise.
  • Extended ablations to all five benchmarks and discussed InfoSeek performance and inference costs.

Please let us know if any points remain unclear or require further clarification. We would be glad to address them before the deadline. If our revisions have resolved your concerns, we would greatly appreciate it if you could reconsider your assessment of our work.

Thank you very much for your time!

Comments on the paper

1 comment
AuthorsOpenReview ↗

Rebuttal Summary

Dear PCs, SACs, ACs, and Reviewers,

We sincerely thank all chairs and reviewers for their valuable feedback and constructive reviews, which have greatly helped us improve our paper. We are encouraged by their recognition of our important and well-motivated problem (Reviewers HDWh, S8kE, and XYwK), informative pilot studies (Reviewer S8kE), interesting and practically useful idea of revisiting target identities while tracking collected evidence (Reviewer HDWh), sensible and coherent evidence-management design (Reviewers S8kE and XYwK), good performance across five benchmarks (Reviewer HDWh), and targeted ablations and diagnostic analyses (Reviewer XYwK).

In our rebuttal, we addressed the following raised concerns/misunderstandings:

  • We conducted controlled comparisons against ReAct and MM-DeepResearch under zero post-training, each baseline’s own workflow trajectories (WorkflowData), and shared training data, proving that gains stem from the framework itself rather than data curation and clarifying the InfoSeek comparison.

  • We evaluated graph-free prompt-based verification baselines and stratified results by initial identity correctness, demonstrating that explicit state management outperforms prompting and primarily helps recover from initially misidentified targets.

  • We extended all ablations to all five benchmarks, verifying consistent cross-benchmark generalization.

  • We clarified the policy and executor roles and evaluated evidence attribution under visual ambiguity and noise on manually verified cases, confirming higher attribution precision and robustness to distractors.

  • We detailed the frozen auditor setup and conducted human validation and noise-sensitivity tests across candidate auditors, verifying reliable audit quality while transparently characterizing sensitivity to audit errors.

  • We quantified training-time auditing overhead and compared RL objectives under matched compute budgets, showing that RevGRPO outperforms additive process supervision at identical compute with only minor overhead over outcome-only GRPO.

  • We strengthened the manuscript’s formalization and positioned RevMMSearch against prior structured search methods, clearly establishing our distinct contributions in revisable identity routing, coupled fact-applicability updates, and conflict-aware post-training.

We would like to highlight the core novelty and contributions of our work. A key bottleneck in multimodal search agents is that factually correct information still leads to wrong answers when tied to a misidentified visual target. Rather than letting an early visual error lock in later searches and leave stale facts active, RevMMSearch provides a unified framework for evidence-grounded identity revision:

  • Target identity as a revisable routing state: Motivated by pilot studies showing stale evidence, hypothesis reinforcement, and misleading outcome credit, we treat visual target identity not as a fixed premise, but as a revisable routing state that governs both future retrieval and the validity of earlier facts.

  • Coupled identity revision and fact management: Using an entity-evidence graph with explicit fact, source, and identity dependencies, RevMMSearch goes beyond prior structured search workflows by directly linking identity revision to historical fact applicability. Within a unified loop of discriminative search and attribution verification, facts supported only by rejected identities become stale, while independently supported facts remain usable.

  • Conflict-aware credit assignment: Standard outcome rewards and additive process rewards struggle when final outcomes conflict with intermediate evidence. RevGRPO uses decision-time evidence audits to suppress only the conflicting outcome credit while keeping signed local supervision, offering a distinct alternative to additive reward shaping or masking an entire segment's update.

  • Strong empirical gains and broad generalization: Across five benchmarks, RevMMSearch achieves higher average accuracy than strong multimodal search agents and keeps its lead under controlled backbone, data, and budget comparisons. Moreover, RevGRPO also improves other search frameworks such as ReAct and MM-DeepResearch.

We believe these insights, combined with our principled framework and comprehensive empirical validation, provide a valuable contribution to the field of multimodal search agents. We hope our responses have addressed all questions and concerns, and we sincerely appreciate your time and consideration of our submission.

Best regards,

Authors of Submission 111

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.