acceptodds
Desk rejected at ICLR 2027

OneSearch-VL: Unified Multimodal Deep Research Agent for Image and Video

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-image, multi-image, and video deep research require different visual operations but share a workflow of visual grounding, external retrieval, and fact composition. A key challenge is to preserve the dependencies linking localized visual anchors, entity relations, source-supported facts, and answer-producing operations. We introduce OneSearch-VL, a unified agent centered on the Visually Grounded Evidence Graph (VGEG), which encodes these dependencies as a shared task-level reference for data construction, process supervision, and operation-level evaluation. Our VGEG-based data engine constructs and verifies multi-image and video questions and filters expert trajectories. Using these data, We assemble OneSearch-VL-SFT-110K and OneSearch-VL-RL-10K for SFT and RL, respectively. We further derive the Evidence-aware Visual-Grounded Rubric reward (EVGR) from VGEG annotations to supervise evidence traceability and visual grounding during RL. For finegrained evaluation, we construct OneSearch-MI-Bench and OneSearch-Video-Bench, organizing questions by the research operations encoded in their VGEGs. Experiments show that OneSearch-VL-8B improves over Qwen3-VL-8B with tool access by 20.2 and 17.6 percentage points on the two new benchmarks, respectively, while also achieving substantial gains across 7 image benchmarks and VideoDR.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
multimodal deep research, multimodal understand, agentic tool use

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_7d2HOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces OneSearch-VL, a unified multimodal deep-research agent that handles single-image, multi-image, and video inputs through a unified tool-use framework. The paper proposes the Visually Grounded Evidence Graph (VGEG), which connects visual anchors, real-world entities, source-supported facts, and the operations required to produce final answers. Building on this representation, the authors construct training data and expert trajectories and introduce a reward mechanism based on the Evidence-aware Visual-Grounded Rubric (EVGR) to supervise evidence traceability and visual grounding during reinforcement learning. Experimental results show that the proposed method outperforms existing baselines on several established and newly constructed benchmarks.

Critical strengths

The paper studies how multimodal agents can connect visual observations with external knowledge and combine evidence across images and videos. This is a valuable research direction that goes beyond traditional visual question answering.

Meanwhile, VGEG provides a unified representation for data construction, trajectory filtering, reward design, and operation-oriented evaluation. The experiments also provide meaningful preliminary evidence for the effectiveness of the proposed approach.

Critical weaknesses

The paper introduces several interrelated concepts and presents them as an integrated framework, but the methodological innovations themselves remain insufficiently distinctive. The overall approach appears more like a combination of existing techniques than a clearly differentiated new methodology. The current ablation studies demonstrate that the overall system is effective, but they do not sufficiently establish that the specific design choices are necessary or superior to simpler alternatives. The authors should therefore provide a more explicit comparison of VGEG and EVGR with the most relevant evidence-graph and process-supervision methods.

The evaluation also relies on LLM-based judgments of answer correctness and trajectory quality. Since the proposed reward mechanism likewise depends on structured assessments of evidence support and visual grounding, systematic errors in the evaluator could affect both training and final evaluation.

The paper introduces new benchmarks, but it remains unclear whether these benchmarks independently assess general multimodal research capabilities or favor the specific operations and evidence structures emphasized during training. The authors should further analyze potential overlap between the training and test sets, the diversity of data sources, and the model's generalization to out-of-distribution tasks.

Questions for authors
  1. What are the most important technical differences between VGEG and existing evidence-graph or knowledge-grounding representations?

  2. How reliable is the LLM-based EVGR evaluator? Have the authors assessed the agreement between its scores and human annotations or independent evaluators? Could systematic biases cause the evaluator to assign higher scores to trajectories that conform more closely to the proposed rubric?

  3. How do the authors control for potential overlap between the training data and the two newly introduced benchmarks?

4 comments
AuthorsOpenReview ↗

Response to Reviewer 7d2H (1/2) — What VGEG represents

VGEG’s technical differences, evaluator reliability, and cross-benchmark generalization

Thank you for asking us to clarify what VGEG represents and what our evaluation covers.

Q1. Technical differences between VGEG and existing evidence-graph or knowledge-grounding representations

In video and multi-image deep research tasks, relevant entities often appear in different frames or images. Reasoning depth is the number of hops along entity relations needed to retrieve a fact. Reasoning breadth is the number of visual anchors whose evidence must be combined to answer the question. VGEG records both kinds of dependencies in one graph per question.

Existing evidence graphs and knowledge grounding. SearchEyes builds graphs from entities and relations already present in knowledge bases such as Wikidata5M. CaRR identifies the entities and facts to verify in a text question. STAMP starts with entities from Wikipedia and follows relations over multiple hops to build its graph. In video and multi-image deep research tasks, a question built only around entities and relations retrieved in advance can still require many retrieval hops. However, it may leave out other entities in the visual inputs and their relations, reducing the need to combine evidence across frames or images.

Task representation in existing video deep research methods. VideoSearcher uses entities from existing image QA pairs to find videos and then adapts the questions to the video content. This can preserve the original multi-hop retrieval requirements, but the breadth of each new question still depends largely on the entities in the original question. Other objects in the video and their relations may therefore play no role in the reasoning. Video-DeepResearch localizes objects in videos and uses verified keyframes, bounding boxes, entities, and search summaries to construct QA pairs. Selecting multiple entities can broaden the scope of a question. Its representation, however, neither traces the multi-hop evidence chain from each anchor nor specifies which facts from different anchors feed each answer operation.

Although VGEG may initially seem like a combination of existing ideas, our core contributions include building question-specific evidence graphs from visual inputs to satisfy both reasoning depth and breadth, and designing the full data construction pipeline based on the graph. The graph captures reasoning depth through evidence chains that can span multiple hops along entity relations, linking each visual anchor to the facts needed for the answer. It captures reasoning breadth by specifying which facts retrieved from different anchors are combined in each answer operation. Using the same graph, we can both verify each evidence chain and check whether facts from different frames or images can be combined to derive the answer.

We also evaluated VideoSearcher-8B on both OneSearch benchmarks. The comparison shows that OneSearch-VL-8B has a clear advantage on questions that require combining evidence across visual anchors.

OneSearch-MI-Bench (accuracy, %)

ModelSAMHCountJoinArith.Comp.All
VideoSearcher-8B39.333.361.431.024.638.837.9
OneSearch-VL-8B46.443.875.456.944.363.355.8

OneSearch-Video-Bench (accuracy, %)

ModelSAMHCountJoinArith.Comp.All
VideoSearcher-8B52.017.034.419.211.124.724.8
OneSearch-VL-8B44.038.336.134.622.239.035.5
AuthorsOpenReview ↗

Response to Reviewer 7d2H (2/2) — Evaluator reliability and benchmark generalization

Q2. Reliability of the LLM-based EVGR evaluator and potential rubric bias

1. Agreement between EVGR scores and human ratings

For each of Trace and Ground, the human evaluation protocol groups rollouts by their original Seed 2.0 Pro scores using ten equal-width bins over the 0–1 range. We randomly sample 50 unmodified RL rollouts from each bin, for a total of 500 per dimension. Five annotators take part. Two independently score each rollout on a 0–1 scale using the question, source media, and tool records. Neither sees the judge’s original score or the other annotator’s rating. The human reference score is normally the average of the two ratings.

For each rollout, we pair the judge’s original score with the human reference score. We report the mean absolute error (MAE) and the share of valid pairs whose scores differ by at most 0.10. The shares were 85.4% for Trace and 84.4% for Ground, with MAEs of 0.046 and 0.050, respectively. The similar agreement rates and low MAEs indicate that Seed 2.0 Pro generally scores both Trace and Ground close to the independent human ratings on the sampled rollouts.

2. Could systematic biases cause the evaluator to assign higher scores to trajectories that conform more closely to the proposed rubric?

To test whether the evaluator favors wording that more closely matches the proposed rubric, we sample up to 500 saved RL rollouts for each of Trace and Ground as version A. For B, we give the proposed rubric to an LLM and ask it to rewrite the wording to match the rubric more closely while preserving the original meaning and evidence. C introduces an error in one key piece of evidence, and D applies both changes. All other content stays fixed, and the same judge scores all four versions.

EvidenceOriginal wordingRewritten wording
SufficientA. Original rollout with sufficient evidence.B. Keep A’s evidence and meaning, and rewrite its wording to more closely match the proposed rubric.
One key errorC. Keep the original wording and introduce an error in one key piece of evidence.D. Keep C’s evidence and use the rewritten wording from B.

Paired score differences

MetricTraceGround
Wording (A − B)+0.0040.0
Wording (C − D)+0.011+0.014
Evidence (A − C)+0.196+0.228
Evidence (B − D)+0.203+0.241

The score differences are on the original 0–1 scale. In these paired comparisons, rewriting with reference to the proposed rubric does not noticeably raise scores when the evidence and meaning stay fixed. Introducing an error in key evidence while keeping the wording fixed substantially lowers scores.

Q3. Controlling for potential overlap between training data and the two new benchmarks

Training and benchmark separation. Neither benchmark shares source YouTube videos or visual entities used to construct questions with SFT-110K or RL-10K. Before building the training sets, we used YouTube video IDs, upload channels, and the visual entities used to construct questions to split the source videos into a training pool of about 65K videos and a benchmark pool of about 5K. We kept all videos from the same channel in the same pool. Each clip, extracted frame, and derived multi-image question stayed in the pool of its source video.

Diversity of benchmark sources. Together, the two benchmarks contain 608 questions across six domains: daily life (222), entertainment (153), geography (84), technology (58), sports (51), and news (40). All six research operations appear in every domain, though their proportions vary.

Cross-benchmark generalization. To test whether the model’s research capabilities transfer beyond the two OneSearch benchmarks, we also evaluated it on external image and video benchmarks. Across seven existing single-image deep research benchmarks, OneSearch-VL-8B achieves an average score of 58.3, compared with 56.6 for OpenSearch-VL-8B. On the independently developed VideoDR benchmark, OneSearch-VL-8B achieves 57.0% accuracy, compared with 47.0% for OpenSearch-VL-8B and 30.0% for tool-augmented Qwen3-VL-8B.

AuthorsOpenReview ↗

Sincere Thanks and Follow-up to Reviewer 7d2H

Dear Reviewer 7d2H,

Thank you again for taking the time to review our manuscript and share your feedback. We have responded to your comments and revised the manuscript accordingly. We hope that our responses and revisions address your concerns. If any questions or concerns remain, we would be happy to address them during the discussion period.

Best regards,

The Authors

AuthorsOpenReview ↗

Follow-up on our rebuttal

Dear Reviewer 7d2H,

As the discussion period is drawing to a close, we would like to gently follow up on our rebuttal and the additional experiments addressing your comments on OneSearch-VL. We would appreciate your feedback on whether our responses have addressed your remaining concerns, and would be happy to provide any further clarification before the deadline.

Thank you for your time and constructive feedback.

Best regards, The Authors

Reviewer_2mjdOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

OneSearch-VL is a unified deep-research agent over single-image, multi-image, and video inputs, organized around the Visually Grounded Evidence Graph (VGEG) — a task-level structure linking localized visual anchors to entities, source-supported facts, and answer-producing operations. A VGEG-based data engine builds OneSearch-VL-SFT-110K and OneSearch-VL-RL-10K, filters expert trajectories, and yields the EVGR process reward (evidence traceability plus visual grounding), used in GRPO as a format-gated weighted sum of answer-correctness, query-quality, and EVGR rewards. Two operation-oriented benchmarks (OneSearch-MI-Bench 301, OneSearch-Video-Bench 307) are introduced. The 8B model beats Qwen3-VL-8B by 20.2 / 17.6 / 27.0 points on MI / Video / VideoDR.

Critical strengths

1.Coherent, well-engineered system. VGEG genuinely unifies four concerns (data construction, trajectory filtering, RL reward, evaluation) rather than being decorative; the tool suite and action–observation protocol are clean. 2.Comprehensive experiments: seven external single-image benchmarks plus two new ones plus VideoDR, many baselines, and ablations on both data mixture and reward. 3.EVGR is a sensible process reward, and the ablation (57.3 to 59.0/59.3 to 61.1) shows each dimension contributes. 4.Strong external video result: +27.0 on VideoDR (30.0 to 57.0) is convincing precisely because VideoDR is not author-built. 5.Detailed appendix (data-engine stages, tool semantics, full SFT/RL configs) aids reproducibility of the training recipe.

Critical weaknesses

1.Self-benchmark contamination is not ruled out. The two new benchmarks are produced by the same pipeline and the same 70,781-video source pool as the training data. The paper never explicitly states that benchmark questions/sources are disjoint from SFT-110K/RL-10K. Given that gains on the self-built benchmarks (+20.2/+17.6) are an order of magnitude larger than on external single-image benchmarks (+1.7 average over OpenSearch-VL), distribution overlap is a live concern. Please confirm disjointness by source video and ideally report a train-held-out variant. 2.The "unification" benefit is marginal. Joint SFT averages 55.8 vs. video-only 55.1 — a 0.7 gain. Most value comes from video trajectories alone, which undercuts the claim that jointly training all three input types is meaningfully better and invites the simpler recommendation "just train on video." 3.No error bars or significance tests anywhere. Benchmark sizes are small (608 total; per-operation cells as low as 53), scoring is a binary GPT-4o judgment, and results depend on live web retrieval. Per-operation deltas of a few points are within plausible noise; confidence intervals / multiple seeds are needed. 4.Heavy proprietary dependence at both construction and evaluation. Seed 2.0 Pro is the expert model and the EVGR trajectory judge; GPT-4.1 does routing/QC; GPT-4o judges final-answer correctness. This creates reproducibility risk and a circularity where the model is trained to imitate, and rewarded by, the same proprietary judge. No human-agreement statistic for EVGR is reported. 5.Human review is thin: only 5 annotators, "joint inspection," and no inter-annotator agreement reported for the benchmark labels (operation type, difficulty, grounding). The structural difficulty score is admittedly not human-calibrated. 6.Missing strong agentic baselines on the new benchmarks. Frontier APIs appear only under Direct Reasoning; there is no Gemini/GPT-with-tools or video-specific deep-research agent on OneSearch-MI/Video-Bench, making it hard to situate the 8B model against frontier agentic systems on the authors' own evaluation.

Questions for authors

1.Are OneSearch-MI-Bench / OneSearch-Video-Bench strictly disjoint (by source video and by VGEG) from SFT-110K and RL-10K? Please state the split procedure. 2.Can you report seed variance / confidence intervals for the main tables, at least on the two new benchmarks? 3.How reliable is EVGR (Seed 2.0 Pro judge) against human ratings of traceability/grounding? 4.Why not compare against a video-only-trained agent, given video-only is roughly equal to joint?

6 comments
AuthorsOpenReview ↗

Response to Reviewer 2mjd (1/3) — Benchmark separation and inference variation

Evaluation reliability and training comparisons

Thank you for the questions about benchmark separation, uncertainty, and the video-only control.

Q1. Benchmark separation

We deliberately defined explicit criteria for the data split to ensure that no original YouTube video appears in more than one of the SFT, RL, and benchmark pools.

After screening video sources, we retained 70,781 candidate YouTube video records. We first split the original videos into a training pool of about 65K and a benchmark pool of about 5K, based on YouTube video ID, upload channel, and the visual entities used to construct questions. Videos from the same channel stayed in the same pool. To verify each entity, we first located the object in the image or video, then identified the corresponding real-world entity using image search or OCR followed by text search (Figure 1(c)).

Clips, extracted frames, and multi-image questions derived from the same source video, along with the VGEG instances linked to those visual inputs, stayed in the same pool as the source video. Using verified visual entities as a split criterion also helps avoid overlap in the entities used in training and benchmark questions. We then split the training pool by YouTube video ID, allocating about 45K original videos to SFT and 20K to RL, and selected samples separately within each pool. We checked both public training datasets against each benchmark for exact question matches and found none.

Q2. Variation across inference runs

The repeated-inference setup uses five runs per benchmark, a fixed checkpoint, decoding temperature 0, and the same tool configuration and budget. The correct count ranges span three questions on MI and two on Video.

Five inference runs (mean ± sample SD)

BenchSAMHCountJoinArith.Comp.All
OneSearch-MI45.71 ± 1.6043.75 ± 1.4775.79 ± 1.4756.90 ± 1.2244.59 ± 0.7363.67 ± 1.7155.95 ± 0.38
OneSearch-Video43.20 ± 1.7938.30 ± 1.5036.72 ± 0.9034.62 ± 1.3622.22 ± 1.5738.96 ± 0.9235.57 ± 0.27

Each column reports mean accuracy and sample standard deviation across the same five runs. For each run, we sum the correct counts across the six operations and divide by the total number of questions to obtain overall accuracy. The All column reports the mean and sample standard deviation of these five overall accuracies.

AuthorsOpenReview ↗

Response to Reviewer 2mjd (2/3) — Human agreement and training controls

Q3. Human agreement for EVGR

Our human evaluation protocol uses stratified samples from saved RL training rollouts. For Trace and Ground separately, we divide the 0–1 range of the original Seed 2.0 Pro scores into ten equal-width bins. We randomly sample 50 rollouts per bin, with a quota of 500 per dimension. Each comparison pairs the human reference score with the judge's original score for the same rollout.

Five annotators take part in the human evaluation, with two independently scoring each sampled rollout. They review the question, the facts and visual objects to verify, the source images or video, the images or frames the model actually used, and the tool calls and outputs. They cannot see the judge's scores or explanations, or each other's ratings. Trace assesses whether the retrieved evidence supports the key facts needed to answer the question. Ground assesses whether the model correctly identifies and locates the object and uses it in subsequent retrieval. Annotators score the assigned dimension from 0 to 1 for the entire rollout and record the relevant web excerpts, images, or timestamps. The scores 0, 0.25, 0.50, 0.75, and 1 serve as reference points, and intermediate scores are allowed.

We use the mean of the two ratings as the human reference score. If the ratings differ by more than 0.20 points, or if one is at least 0.80 and the other is below 0.80, a third annotator reviews the evidence and assigns the human reference score.

We include only rollouts with valid scores from both the human annotators and the judge. For Trace and Ground separately, we report the mean absolute error (MAE) and the share of rollouts with a human–judge score difference of at most 0.10. We also report two error rates using 0.80 as the pass threshold. For the false pass rate, the denominator is the number of valid rollouts with a human score below 0.80, and the numerator counts those with a judge score of at least 0.80. For the false fail rate, the denominator is the number of valid rollouts with a human score of at least 0.80, and the numerator counts those with a judge score below 0.80.

JudgeWithin 0.10MAEFalse passFalse fail
Trace85.4%0.0462.2%8.1%
Ground84.4%0.0503.7%11.5%

The relatively low false-pass and false-fail rates at the 0.80 threshold support the validity of the scoring rubric for assessing Trace and Ground on the sampled rollouts.

Q4. Video-only training

To compare the models after RL, we added image-only and video-only controls. Each control uses data from only its corresponding modality in both SFT and RL. As shown below, joint training achieves a mean accuracy of 61.1%, 3.0 percentage points higher than video-only training and 4.1 points higher than image-only training. The gap between joint and video-only training is larger than in the SFT-only comparison in Table 4(a).

SettingSimpleVQAInfoSeekFVQAMulti-imageVideoVideoDRAvg.
Image-only SFT + RL72.271.672.948.827.749.057.0
Video-only SFT + RL69.866.270.052.833.956.058.1
Joint SFT + RL72.472.373.455.835.557.061.1
AuthorsOpenReview ↗

Response to Reviewer 2mjd (3/3) — Benchmark annotation and tool-augmented baselines

W5. Benchmark annotation and difficulty

Five annotators jointly reviewed the source images or videos, reference answers, and supporting evidence. We rechecked the operation types and visual-anchor annotations for all 608 questions across the two benchmarks, including 301 in OneSearch-MI-Bench and 307 in OneSearch-Video-Bench. These checks found no errors in either type of annotation.

We considered the annotators’ feedback when designing the difficulty score and grouped questions by operation type, number of reasoning hops, and number of required facts.

W6. Tool-augmented frontier baselines

To address the reviewer’s request for tool-augmented baselines, we evaluated Gemini 3.8 Flash and GPT-5.6 Sol with tools on both OneSearch benchmarks. These were the frontier models we could access and evaluate with tools during the rebuttal period. We also evaluated the video research agent VideoSearcher-8B on both benchmarks.

OneSearch-MI-Bench (accuracy, %)

ModelSAMHCountJoinArith.Comp.All
Gemini 3.8 Flash39.352.171.946.637.742.949.2
GPT-5.6 Sol46.452.178.950.034.453.152.8
VideoSearcher-8B39.333.361.431.024.638.837.9
OneSearch-VL-8B46.443.875.456.944.363.355.8

OneSearch-Video-Bench (accuracy, %)

ModelSAMHCountJoinArith.Comp.All
Gemini 3.8 Flash64.042.624.634.615.636.433.9
GPT-5.6 Sol80.059.627.946.231.145.545.0
VideoSearcher-8B52.017.034.419.211.124.724.8
OneSearch-VL-8B44.038.336.134.622.239.035.5

On the multi-image benchmark, OneSearch-VL-8B ranks first among the evaluated models at 55.8%, 3.0 percentage points ahead of GPT-5.6 Sol. GPT-5.6 Sol leads the video benchmark at 45.0%, where OneSearch-VL-8B scores 35.5%. OneSearch-VL-8B outperforms Gemini 3.8 Flash and VideoSearcher-8B in overall accuracy on both benchmarks. OneSearch-VL-8B shows a particular advantage on questions that require integrating evidence across multiple visual anchors for more complex reasoning.

Reviewer_DMWPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

OneSearch-VL trains a single agent for deep research over single images, image collections, and videos. A Visually Grounded Evidence Graph (VGEG) links visual anchors to retrieved facts and the operations needed to derive an answer. The graph supports data construction and an Evidence-aware Visual-Grounded Rubric reward (EVGR). The agent is trained on about 110K SFT trajectories and 10K RL tasks, and evaluated on existing benchmarks and two new benchmarks containing 608 questions in total.

Critical strengths
  1. VGEG provides a coherent reference across task construction, trajectory filtering, and reward design. Keeping the dependencies between visual anchors, retrieved facts, and answer operations is useful for multi-image questions in particular, where identifying the right objects is only one part of the task. The unified treatment of image and video research is a meaningful extension of modality-specific systems.

  2. The results are supported by existing benchmarks as well as the new ones. Relative to OpenSearch-VL, the reported average over seven image benchmarks rises from 56.6 to 58.3, and VideoDR accuracy rises from 47.0 to 57.0 (Tables 2-3). The reward ablation is also informative: adding Trace and Ground separately improves the six-benchmark average, and using both raises it from 57.3 to 61.1 over answer and query rewards (Table 4b).

Critical weaknesses
  1. The train-test separation for the new benchmarks is not sufficiently specified. Training and test examples are produced by the same data engine, and multi-image inputs are derived from source-video frames (Sections 3-4 and Appendices B.4 and C.1). The paper describes deduplication and quality review, but I could not find a protocol that groups each source video and its derived image examples into a single split, or overlap statistics across SFT, RL, and evaluation. Shared construction does not imply leakage. Still, without these checks, the large gains on the new benchmarks are difficult to interpret as generalization to unseen visual sources and evidence paths.

  2. The claim about process quality is less well supported than the accuracy claim. Appendix E.3 states that EVGR receives visual handles and location records rather than the raw media. These records can support checks against annotated locations, but do not independently establish that an object was correctly recognized or that a claim is supported by the pixels. Table 4b shows a downstream accuracy benefit, yet the paper does not report independent human calibration of Trace/Ground scores or whether EVGR reduces unsupported claims in the trained agent's outputs. The qualitative cases in Appendix F.2 illustrate why answer matching alone is insufficient: answers that match the reference can still contain conflicting facts. These are illustrative trajectories, not a controlled evaluation of the full EVGR-trained model.

Questions for authors
  1. How were source videos and their multi-image derivatives assigned to SFT, RL, and test splits? Please report cross-split checks for duplicate or near-duplicate visual inputs and repeated questions/evidence paths, and the effect of excluding any overlaps.

  2. On an independently annotated sample, how often do the Trace and Ground judges accept unsupported claims or incorrect visual identifications? Comparing RL with answer and query rewards against full EVGR using the same independent assessment of evidence quality would directly test this claim.

  3. What is the exact rule for combining the Trace and Ground scores into EVGR? Appendix E.4 gives the outer reward weights, but I could not find the combination rule for these two components.

4 comments
AuthorsOpenReview ↗

Response to Reviewer DMWP (1/2) — Training and benchmark split

Data separation and evidence quality under EVGR

Thank you for asking us to clarify the data split and the EVGR reward.

Q1. Training and benchmark split

After screening video sources, we retained 70,781 candidate YouTube video records. We first split the source videos into a training pool of about 65K and a benchmark pool of about 5K. The split used three criteria: YouTube video ID, upload channel, and the visual entities used to create questions. Videos from the same channel stayed in the same pool. To verify an entity, we first located the object in the image or video, then identified the real-world entity using image search or OCR followed by text search (Figure 1(c)).

Video clips, extracted frames, derived multi-image questions, and VGEG instances tied to those visual inputs inherited the source video's assignment. Grouping by verified entity identity also helps keep the visual entities used in benchmark questions and any identical VGEG instances tied to them out of the training data.

We then divided the training pool by YouTube video ID, allocating about 45K source videos to SFT and 20K to RL. We selected samples separately within each pool to build the final datasets. The SFT, RL, and benchmark pools share no source videos, so samples derived from the same source video cannot cross pools.

To select RL questions, we generated eight independent answers with Qwen3-VL-8B per candidate, kept questions with one to seven correct answers, and then deduplicated them. We also checked about 110K SFT questions and 10K RL questions for exact matches against OneSearch-MI-Bench and OneSearch-Video-Bench. Neither training dataset contained an exact question match with either benchmark.

AuthorsOpenReview ↗

Response to Reviewer DMWP (2/2) — Evidence quality and EVGR rewards

Q2. False acceptance and evidence quality

Table 4(b) shows that adding Full EVGR to the answer and query rewards raises mean answer accuracy across six benchmarks from 57.3% to 61.1%. To compare evidence quality directly, we pair one reasoning trajectory from the Answer + Query model with one from the Full EVGR model for each question. GPT-5.6 Sol and Gemini 3.8 Flash each independently evaluate the same 900 questions, with 300 each for single-image, multi-image, and video inputs. For human evaluation, we randomly sample 450 of these questions, 150 per modality. Both trajectories for each question receive Trace and Ground scores.

Before scoring, we prepare the VGEG for each question and extract an evidence ledger from each trajectory. Five annotators take part in the human evaluation, with two independently scoring each trajectory. For each dimension, we average the two scores if they differ by at most 0.2 points. If the difference exceeds 0.2 points, a third annotator reviews the evidence and assigns the final score for that dimension. The human annotators and both models use the same scoring criteria and review the question, tool calls, and actual tool outputs.

Evaluators score each trajectory separately on Trace and Ground, on a scale from 0 to 1. Trace measures how well the retrieved evidence supports the facts specified by the VGEG. Ground measures whether the agent correctly identifies and locates the visual objects and uses them as retrieval cues when required. Both models and the human annotators receive the same scoring criteria, with five anchors at 0, 0.25, 0.50, 0.75, and 1. Evaluators may assign intermediate scores. When scoring, they record the relevant tool output, image ID, or video timestamp.

Mean Trace and Ground scores (0–1)

TraceGround
InputSettingGPT-5.6 SolGemini 3.8 FlashHumanGPT-5.6 SolGemini 3.8 FlashHuman
Single-imageAnswer + Query0.6700.6390.6080.7110.6870.648
Full EVGR0.8320.8110.7910.8640.8470.824
Δ+0.162+0.172+0.183+0.153+0.160+0.176
Multi-imageAnswer + Query0.6160.5950.5470.6580.6330.591
Full EVGR0.7890.7760.7380.8170.8010.775
Δ+0.173+0.181+0.191+0.159+0.168+0.184
VideoAnswer + Query0.5430.5220.4820.6070.5760.533
Full EVGR0.7240.7120.6780.7720.7530.722
Δ+0.181+0.190+0.196+0.165+0.177+0.189
OverallAnswer + Query0.6100.5850.5460.6590.6320.591
Full EVGR0.7820.7660.7360.8180.8000.774
Δ+0.172+0.181+0.190+0.159+0.168+0.183

False acceptance rates for Trace and Ground

We use the same 450 questions from the human evaluation to estimate Seed 2.0 Pro’s false acceptance rates. Without seeing Seed 2.0 Pro’s judgments, the annotators independently check whether the retrieved evidence supports each fact and whether the agent correctly identified each object. Once their assessment is complete, the annotators compare their judgments with the judge’s item by item and count only items that both could assess. Among claims that annotators found unsupported or contradicted by the evidence, the Trace judge marked 10.3% as supported. Among object identifications that annotators found incorrect, the Ground judge marked 8.2% as correct.

Q3. EVGR aggregation

Full EVGR is the arithmetic mean of the two trajectory scores, REVGR=(rtrace+rground)/2R_{\mathrm{EVGR}}=(r_{\mathrm{trace}}+r_{\mathrm{ground}})/2. In the full reward, the format gate multiplies the weighted sum of answer correctness, query quality, and EVGR, with weights 0.6, 0.2, and 0.2, respectively.

Reviewer_DMWPOpenReview ↗

Post-Rebuttal Assessment

I have carefully read the authors’ response and appreciate their efforts to clarify and strengthen the paper. The rebuttal has addressed some of my concerns.

After considering the response and the other reviewers’ comments, I have decided to maintain my current score. My main reservation concerns the overall significance of the contribution. In my assessment, the methodological novelty remains limited, and the paper offers relatively few generalizable insights into search-agent design beyond the specific system presented. Although the response provides useful clarification, it has not sufficiently changed my assessment on these points.

I appreciate the authors’ work, but on balance, I do not find the contribution sufficiently compelling to recommend acceptance.

AuthorsOpenReview ↗

Clarification on the broader methodological contribution

Dear Reviewer DMWP,

Thank you for clarifying your remaining concern about the broader contribution. We would like to briefly highlight the design insight discussed in our response to Reviewer 7d2H (Q1).

VGEG explicitly represents both multi-hop evidence chains and the composition of facts across visual anchors, using a common task-level reference for data construction, trajectory filtering, and process supervision. The broader lesson we aim to offer is to supervise evidence acquisition together with its grounding in the relevant visual input. The reward ablation supports this supervision choice: adding Trace and Ground to Answer + Query improves mean accuracy across six benchmarks from 57.3% to 61.1% (Table 4(b)).

We hope this clarifies the methodological insight underlying our framework. Thank you again for your time and consideration.

Best regards, The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Updated manuscript and a quick follow-up

Thanks for taking the time to read and discuss our paper. We’ve uploaded a revised PDF that brings the main updates from our replies into the manuscript. It includes more detail on the data split, the human evaluation of EVGR, and the new baseline and training comparisons. We’ve marked the additions in the PDF so they’re easy to find.

  • Data separation and benchmark review. We clarify the training–benchmark split and report the review of operation labels and visual anchors for all 608 benchmark questions.
  • EVGR evaluation. We clarify how Trace and Ground are combined and add the paired evidence-quality evaluation, the comparison with independent human ratings, and the wording and evidence controls.
  • Baselines and additional experiments. Table 3 adds tool-augmented results for Gemini 3.8 Flash, GPT-5.6 Sol, and VideoSearcher-8B. Appendix G reports five repeated inference runs and the image-only and video-only training controls.

If you’ve had a chance to read our replies, we’d appreciate a quick note on whether they address your main concerns. If there’s a point you’d still like us to clarify, please let us know before the discussion closes on November 18 so we have time to follow up.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.