acceptodds
Under review as a conference paper at ICLR 2027

Beyond Single Videos: Benchmarking and Active Evidence Seeking for E-Commerce Cross-Video Reasoning

Abstract

E-commerce videos are information-dense and frequently compared in practice, as consumers compare products and merchants evaluate marketing tactics, making cross-video analysis highly valuable for real-world business analytics. Yet, existing multimodal models primarily focus on single video understanding, with limited ability to perform the cross-video comparisons needed for e-commerce video analysis. To bridge this gap, we introduce **AdsCVR**, the first e-commerce cross-video reasoning benchmark. After filtering, AdsCVR contains 2,483 videos and 6,110 QA pairs across six reasoning dimensions. Extending reasoning across multiple videos fundamentally magnifies the challenge of pinpointing fine-grained evidence amid massive redundant frames, especially in e-commerce videos where visual details, speech, and on-screen text are combined into densely packed content. We propose **AdSeek**, an agentic framework that dynamically allocates visual and audio tools during multi-turn exploration, replacing static uniform sampling with active evidence grounding. While reinforcement learning (RL) can optimize sequential tool use, sparse outcome rewards struggle with long-horizon credit assignment, providing limited guidance on where a trajectory goes wrong or how to correct it. To address this, we propose an **offline trajectory rectification mechanism** that identifies and corrects reasoning errors and missing multimodal evidence in RL-generated trajectories. It then uses them for SFT to strengthen the model’s evidence gathering and reasoning abilities, thereby mitigating biases learned during RL. This mechanism anchors our **rectified bootstrapping pipeline**: an initial RL phase exposes reasoning bottlenecks, the subsequent SFT phase uses rectified data to calibrate model's thinking and action biases, and a final RL phase achieves optimal convergence. AdSeek achieves 74.30% accuracy on the AdsCVR test split, improving upon its backbone (Qwen3-VL-8B-Instruct) by 27.90%. It also generalizes well to the open-domain CrossVid benchmark, validating its effectiveness in active evidence gathering.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-Video Reasonig, Multimodal Large Language Models, Agentic Video Understanding, E-commerce Video

Reviews

Among:
Shape:
1234this paper, Bottom 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.25 over 4 reviews, above 49% of the others.
Reviewer_n5qNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces AdsCVR, a benchmark for cross-video comparison of e-commerce advertisements, and AdSeek, which actively invokes visual and audio tools to gather evidence. The method uses a training pipeline consisting of RL, trajectory-rectification SFT, and a second RL stage.

Critical strengths
  1. I think cross-video comparison of e-commerce advertisements is practically valuable, and AdsCVR provides a useful evaluation resource for this setting.
  2. The idea of actively combining visual and audio evidence is reasonable, and the tool ablations provide support for this design.
Critical weaknesses
  1. The gain from trajectory-rectification SFT in Table 3 is quite small, while most of the improvement appears after the second RL stage. I am therefore not fully convinced that the reported gain can be attributed to trajectory rectification without comparable continued-RL or ordinary-success-trajectory SFT baselines.
  2. Since Gemini generates the questions, answers, and textual clues, and GPT mainly checks the answers against those clues, I worry that errors in the initial video descriptions may carry through the verification process. The paper does not report the scale or error rate of independent human checks against the original videos, so the reliability of the dataset remains uncertain to me.
  3. The CrossVid results do not seem fully consistent with the claim of robust cross-domain generalization: average accuracy increases from 29.47% to 31.55%, but product identification drops from 62.5% to 45.4% and open-ended QA from 31.2% to 20.4%. I would appreciate an analysis explaining these task-specific declines.
Questions for authors
  1. I would like to know whether the authors have results from continued RL with a comparable training budget or SFT using ordinary successful trajectories, as these comparisons would help me assess the gains attributable to trajectory rectification itself.
  2. For the reliability of the dataset answers, I would like to better understand the scale and sampling procedure of the representative-sample checks, whether the original videos were inspected directly, and how many errors or disagreements were found.
  3. The marked declines on product identification and open-ended QA on CrossVid are somewhat puzzling to me. Do the authors have an error analysis indicating whether these issues mainly arise from evidence acquisition or from subsequent reasoning and answering?
3 comments
AuthorsOpenReview ↗

Response to Reviewer n5qN(1/2)

We thank the reviewer for recognizing the value of cross-video analysis for e-commerce advertising and the effectiveness of our agentic framework. Below, we address your constructive comments in detail.

1. Evaluating the Effectiveness of Trajectory Rectification (weakness1 & question1)

AdSeek uses a three-stage training pipeline. We collect the RL trajectories from Stage 1, filter and rectify them to construct the SFT data for Stage 2, and thereby correct errors in the model's reasoning. Stage 3 then resumes RL exploration. This pipeline yields successive accuracy improvements of 71.27% → 71.53% → 74.30%.

To isolate the contribution of trajectory rectification in Stage 2, we provide the following ablations:

  1. WeakCheck: Use only Stage 1 trajectories with correct final answers for SFT, without further filtering or rectification.
  2. Teacher distillation: Use all trajectories generated by the teacher model as SFT data.
  3. Continued RL only: Remove the SFT stage and use continued RL throughout the remaining training.

The training settings for each applicable stage are kept consistent with those of AdSeek.

With WeakCheck, Stage 2 performs similarly to AdSeek after Stage 1, while Stage 3 incurs a slight decline of 0.5 percentage points. Because the biases in these trajectories remain unrectified, this result supports the effectiveness of the trajectory filtering and rectification used in AdSeek's Stage 2.

Direct teacher distillation improves accuracy by 0.8 percentage points in Stage 2, but its subsequent Stage 3 gain (+1.7 percentage points) is smaller than AdSeek's (+2.77 percentage points). AdSeek rectifies reasoning errors in trajectories drawn from the student's own distribution, enabling the second RL stage to better realize the model's potential.

Continued RL alone performs similarly to AdSeek after Stage 1, indicating that trajectory rectification provides an advantage for subsequent reinforcement learning in the original three-stage pipeline.

Together, these results support the effectiveness of trajectory rectification within the three-stage training pipeline.

MethodTSCSIDICODVCAvg
AdSeek (Stage1)74.672.465.872.872.669.471.27
AdSeek (Stage2)75.067.668.273.674.470.471.53
AdSeek (Stage3)77.073.470.875.674.874.274.30
WeakCheck (stage2)74.272.666.669.471.872.671.20
WeakCheck (stage3)75.268.468.870.869.871.270.70
Teacher distillation (stage2)71.472.270.871.673.672.872.07
Teacher distillation (stage3)76.874.070.274.075.871.873.77
Continued RL75.871.268.071.072.470.671.50

2. Benchmark Reliability (weakness2 & question2)

We took several measures during the construction of AdsCVR to reduce generation errors and potential bias:

  1. We described each video from four perspectives(events, objects, creative structure, and commercial logic) to reduce information omissions associated with a single descriptive perspective.
  2. We used different models for generation and verification, reducing the risk of relying on hallucinations from a single model through cross-model checks.
  3. We also conducted blind tests without video input and verified the textual clues to strengthen the benchmark's reliability and factual accuracy.

We further clarify the human audit conducted during dataset construction. Using a fixed random seed, we drew a random sample of 100 questions without replacement, covering all six evaluation dimensions. Two annotators first answered the sampled questions independently without access to the reference answers. They then recorded (1) reference-answer correctness, (2) answer unambiguity, (3) sufficiency of video evidence, and (4) multi-video dependency , retaining evidence notes for each question. The audit results are as follows:

Reference-answer correctnessAnswer unambiguityEvidence sufficiencyMulti-video dependency
Annotator 194%93%95%78%
Annotator 293%95%96%79%

This type of human spot check has precedent in video-understanding research. VideoSAVi randomly audited 200 QA pairs from its generalization test set and reported video-based answerability, category-assignment correctness, and answer uniqueness of 97.5%, 94.0%, and 98.0%, respectively( VideoSAVi, Appendix A.5.4). The results of manual verification demonstrate that our benchmark is reliable, with the vast majority of questions requiring evidence from multiple videos to be answered.

AuthorsOpenReview ↗

Response to Reviewer n5qN(2/2)

3. Clarifying the CrossVid Evaluation Results (weakness3 & question 3)

On CrossVid, AdSeek improves over the baseline by 2.08 percentage points on average, with substantial gains on tasks including BU (+15.4 percentage points), NC (+11.3 percentage points), and CC (+7 percentage points).

Some tasks, such as PI (plot inference) and MSR (multiview spatial reasoning), involve specialized scenarios requiring narrative or spatial reasoning. These differ substantially from e-commerce advertising videos, which are fast-paced, commercially focused, and lack complex plots; this domain difference accounts for the performance declines. Meanwhile, improvements on general tasks such as BU (behavioral understanding), CC (culinary comparison), and NC (narrative comprehension) support the transferability of AdSeek's capabilities. Overall, AdSeek achieves a substantial lead on e-commerce cross-video comparison tasks while also delivering overall performance gains on more general tasks in CrossVid, demonstrating strong generalization capability.

AuthorsOpenReview ↗

Response to Reviewer n5qN

Dear Reviewer,

As the discussion deadline approaches, we would appreciate it if you could let us know whether our rebuttal has addressed your concerns. If any questions remain, we would be happy to clarify. If our response has addressed your concerns, we would be grateful if you would consider revisiting your initial score in light of the clarifications.

Thank you for your time and thoughtful feedback. We hope you have a pleasant week.

Best regards, The Authors

Reviewer_r5QBOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AdsCVR, a benchmark for cross-video reasoning in e-commerce advertising, comprising 2,483 videos and 6,110 multiple-choice questions across six reasoning dimensions. It also proposes AdSeek, an agentic framework that iteratively acquires visual and speech evidence, and a Rectified Bootstrapping Pipeline (RBP) consisting of initial reinforcement learning, teacher-guided trajectory rectification followed by supervised fine-tuning, and further reinforcement learning. The application is well motivated, and the reported results suggest that the overall system improves performance over its original backbone. However, the current experiments do not sufficiently isolate the contribution of trajectory rectification. Additional concerns involve benchmark validation, evaluation comparability, and the strength of the generalization claims.

Critical strengths

Clear and reasonable motivation. Comparing product demonstrations, marketing strategies, and narrative structures across advertisements is a practically relevant task. The proposed benchmark addresses a meaningful application of cross-video reasoning.

Convincing evidence of multimodal complementarity. The full system achieves 74.30% accuracy, compared with 63.97% for visual-only access and 65.03% for speech-only access. These results support the usefulness of combining visual and speech evidence for the proposed task.

Critical weaknesses

The benefit of the RL → SFT → RL pipeline is not independently established. Table 3 reports 71.27% after initial RL, 71.53% after rectification and SFT, and 73.57% after subsequent RL with in-domain data. These results are compatible with rectification being helpful, but do not distinguish its contribution from additional training or teacher supervision. A compute-matched baseline that continues RL directly from Stage 1 is particularly important. Comparisons against SFT on successful student trajectories and direct teacher-trajectory distillation would further establish whether targeted rectification provides a distinct benefit. The overall improvement over the original backbone cannot, by itself, validate this specific training design.

Model-dependent difficulty filtering may introduce substantial selection bias. The benchmark construction procedure removes 50% of questions correctly answered by Qwen3-VL-8B-Instruct using uniformly sampled frames. The same model serves as the backbone and a principal baseline in the evaluation. Consequently, the test distribution is explicitly conditioned on this baseline’s weaknesses, which may amplify the apparent advantage of the proposed system. Difficulty filtering is not inherently inappropriate, but its effect on model rankings and reported gains should be quantified. Results on an unfiltered held-out set, together with sensitivity analyses using different filtering models or criteria, would help determine whether the conclusions remain stable.

Answer verification is not sufficient. The verification procedure primarily checks whether GPT-5.5 can recover the reference answer from generated textual clues. This can establish consistency between the clues and the answer, but does not independently establish that those clues faithfully describe the videos. Errors in generated descriptions could therefore persist through verification.

The CrossVid results require a more qualified interpretation and a clearer baseline comparison. Although average performance improves from 29.47% to 31.55%, substantial declines occur on PI (62.5% → 45.4%), MSR (34.5% → 26.4%), and CCQA (31.2% → 20.4%). An analysis of these regressions would help explain which capabilities transfer and which deteriorate. Results for the AdsCVR-only model would also help separate domain transfer from the effect of mixed-source training. Furthermore, the original AgentCVR paper reports 32.06% for AgentCVR-4B and 42.03% for AgentCVR-8B, both numerically above AdSeek’s 31.55%. The present paper instead reports 28.24% for its own reproduction. The substantial discrepancy should be explained before concluding that AdSeek outperforms AgentCVR on CrossVid.

Questions for authors
  1. Dataset provenance, distribution, and availability. Although the paper provides aggregate statistics and filtering procedures, more detail on the original video sources, sampling process, and dataset composition would help readers assess its coverage and representativeness. Additional examples in the supplementary material would make the benchmark easier to assess. Since AdsCVR is a central contribution, the authors should also clarify whether the videos, annotations, train/test splits, and evaluation code will be released, and specify any access restrictions.

  2. Does the proposed pipeline outperform continued RL under matched training budgets and data? How does it compare with simpler trajectory-based SFT controls?

  3. How much does difficulty filtering change model rankings and performance gaps? Do the main conclusions hold on an unfiltered test set?

  4. What proportion of questions has been independently checked against the original videos, and what are the observed error and ambiguity rates?

  5. What explains the declines on PI, MSR, and CCQA, and the gap between the reproduced and originally reported AgentCVR results?

6 comments
AuthorsOpenReview ↗

Response to Reviewer r5QB(1/2)

We thank the reviewer for acknowledging the motivation and application value of our benchmark, as well as the efficacy of our multimodal agentic framework. Below, we address your constructive comments in detail.

1. Evaluating the Effectiveness of Trajectory Rectification (weakness1,question 2)

AdSeek uses a three-stage training pipeline. We collect the RL trajectories from Stage 1, filter and rectify them to construct the SFT data for Stage 2, and thereby correct errors in the model's reasoning. Stage 3 then resumes RL exploration. This pipeline yields successive accuracy improvements of 71.27% → 71.53% → 74.30%.

To isolate the contribution of trajectory rectification in Stage 2, we provide the following ablations:

  1. WeakCheck: Use only Stage 1 trajectories with correct final answers for SFT, without further filtering or rectification.
  2. Teacher distillation: Use all trajectories generated by the teacher model as SFT data.
  3. Continued RL only: Remove the SFT stage and use continued RL throughout the remaining training.

The training settings for each applicable stage are kept consistent with those of AdSeek.

With WeakCheck, Stage 2 performs similarly to AdSeek after Stage 1, while Stage 3 incurs a slight decline of 0.5 percentage points. Because the biases in these trajectories remain unrectified, this result supports the effectiveness of the trajectory filtering and rectification used in AdSeek's Stage 2.

Direct teacher distillation improves accuracy by 0.8 percentage points in Stage 2, but its subsequent Stage 3 gain (+1.7 percentage points) is smaller than AdSeek's (+2.77 percentage points). AdSeek rectifies reasoning errors in trajectories drawn from the student's own distribution, enabling the second RL stage to better realize the model's potential.

Continued RL alone performs similarly to AdSeek after Stage 1, indicating that trajectory rectification provides an advantage for subsequent reinforcement learning in the original three-stage pipeline.

Together, these results support the effectiveness of trajectory rectification within the three-stage training pipeline.

MethodTSCSIDICODVCAvg
AdSeek (Stage1)74.672.465.872.872.669.471.27
AdSeek (Stage2)75.067.668.273.674.470.471.53
AdSeek (Stage3)77.073.470.875.674.874.274.30
WeakCheck (stage2)74.272.666.669.471.872.671.20
WeakCheck (stage3)75.268.468.870.869.871.270.70
Teacher distillation (stage2)71.472.270.871.673.672.872.07
Teacher distillation (stage3)76.874.070.274.075.871.873.77
Continued RL75.871.268.071.072.470.671.50

2. Clarifying Benchmark Difficulty Filtering (weakness 2, question3)

Starting from the dataset before difficulty filtering, we excluded all examples in the existing training set to construct an expanded-test set of 6,063 examples. We evaluated the baselines and AdSeek on this set, as shown below. Difficulty filtering increases the overall difficulty, while the model ranking remains unchanged (AdSeek > Qwen3.5-9B > Qwen3-VL-8B), and AdSeek retains a substantial gain from training.

Test setTSCSIDICODVCAvg
Qwen3.5-9Bexpanded-test73.471.772.271.273.671.972.33
test68.964.667.164.064.964.565.67
Qwen3-VL-8Bexpanded-test64.563.260.962.366.863.263.48
test52.246.841.643.648.246.046.40
AdSeek(stage3)expanded-test81.578.777.678.179.978.279.00
test77.073.470.875.674.874.274.30

We additionally replaced the difficulty-filtering model with Qwen3.5-9B, yielding a test set of 3,000 examples. The corresponding results are shown below. AdSeek still maintains a substantial performance advantage on the new test set (+20.55 percentage point to Qwen3-VL-8B). This demonstrates that although changing the filter model alters the relative difficulty of the test set for different models, AdSeek has learned more generalizable cross-video comparison capabilities in the e-commerce domain, enabling it to perform consistently well across test sets of varying difficulty.

TSCSIDICODVCAvg
Qwen3.5-9B61.157.657.758.461.459.359.25
Qwen3-VL-8B56.155.256.957.160.757.457.23
AdSeek(stage3)80.574.975.679.678.377.877.78
AuthorsOpenReview ↗

Response to Reviewer r5QB(2/2)

3. Benchmark Reliability (weakness3 & question4)

We took several measures during the construction of AdsCVR to reduce generation errors and potential bias:

  1. We described each video from four perspectives(events, objects, creative structure, and commercial logic) to reduce information omissions associated with a single descriptive perspective.
  2. We used different models for generation and verification, reducing the risk of relying on hallucinations from a single model through cross-model checks.
  3. We also conducted blind tests without video input and verified the textual clues to strengthen the benchmark's reliability and factual accuracy.

We further clarify the human audit conducted during dataset construction. Using a fixed random seed, we drew a random sample of 100 questions without replacement, covering all six evaluation dimensions. Two annotators first answered the sampled questions independently without access to the reference answers. They then recorded (1) reference-answer correctness, (2) answer unambiguity, (3) sufficiency of video evidence, and (4) multi-video dependency , retaining evidence notes for each question. The audit results are as follows:

Reference-answer correctnessAnswer unambiguityEvidence sufficiencyMulti-video dependency
Annotator 194%93%95%78%
Annotator 293%95%96%79%

This type of human spot check has precedent in video-understanding research. VideoSAVi randomly audited 200 QA pairs from its generalization test set and reported video-based answerability, category-assignment correctness, and answer uniqueness of 97.5%, 94.0%, and 98.0%, respectively( VideoSAVi, Appendix A.5.4). The results of manual verification demonstrate that our benchmark is reliable, with the vast majority of questions requiring evidence from multiple videos to be answered.

4. Clarifying the CrossVid and AgentCVR Evaluation Results (weakness4 & question 5)

On CrossVid, AdSeek improves over the baseline by 2.08 percentage points on average, with substantial gains on tasks including BU (+15.4 percentage points), NC (+11.3 percentage points), and CC (+7 percentage points).

Some tasks, such as PI (plot inference) and MSR (multiview spatial reasoning), involve specialized scenarios requiring narrative or spatial reasoning. These differ substantially from e-commerce advertising videos, which are fast-paced, commercially focused, and lack complex plots; this domain difference accounts for the performance declines. Meanwhile, improvements on general tasks such as BU (behavioral understanding), CC (culinary comparison), and NC (narrative comprehension) support the transferability of AdSeek's capabilities. Overall, AdSeek achieves a substantial lead on e-commerce cross-video comparison tasks while also delivering overall performance gains on more general tasks in CrossVid, demonstrating strong generalization capability.

Regarding the AgentCVR results, the authors have not provided the trained model checkpoints. We therefore reproduced AgentCVR's tool-use environment within ROLL, the same framework used for AdSeek, and trained it following the original paper, with training configurations consistent with ours (see footnote on p.8 of the main text). We will release the AgentCVR reproduction code alongside our own implementation.

5. Additional Benchmark Details (question 1)

AdsCVR is constructed from active advertising videos on Taobao, a Chinese e-commerce platform. It contains 6,110 QA pairs across six dimensions: TS: 1,038; CS: 1,003; ID: 986; IC: 1,134; OD: 993; and VC: 956. The numbers of questions paired with two, three, and four videos are 1,823 (29.84%), 2,048 (33.52%), and 2,239 (36.64%), respectively. The videos have a mean duration of 33.82 s, a median duration of 29.93 s, and a duration range of 6.47–259.47 s, totaling 23.32 hours. The paper's appendix provides two examples for each dimension, for a total of 12, including the questions, answer choices, reference answers, and sampled video frames.

Evaluation code and the dataset will be released upon acceptance of the paper.

Reviewer_r5QBOpenReview ↗

Response to Author

Thank you for the additional experiments and clarifications, which address several of my earlier concerns. However, some concerns regarding the training data used in the ablations, the handling of issues identified in the human audit, and the CrossVid comparison remain unresolved. Further clarification on the following points would be helpful.

  • Training data: Please clarify the data sources used at each training stage

  • Human audit: The audit reports 93–94% reference-answer correctness and 78–79% multi-video dependency. How were questions identified as incorrect, ambiguous, or not requiring multiple videos handled? Were they corrected, filtered, or retained in the benchmark?

  • CrossVid comparison: Could you explain the substantial decline on CCQA, which is not covered by the discussion of PI and MSR? Regarding AgentCVR, I understand that missing checkpoints necessitated reproduction. What are the possible reasons for the large gap between your reproduced results and those reported in the original paper?

Reviewer_GLrNOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces AdsCVR, an e-commerce cross-video reasoning benchmark containing 6,110 multiple-choice questions over 2,483 videos, and AdSeek, an agent that acquires visual and audio evidence through tools. Its Rectified Bootstrapping Pipeline combines initial RL, teacher-guided repair of failed student trajectories followed by SFT, and resumed RL.

Critical strengths

The benchmark addresses a practically relevant setting. Comparing multiple advertisements requires combining product details, presentation strategies, and audiovisual evidence. AdsCVR makes this setting concrete through six evaluation dimensions and a documented construction pipeline, offering a useful complement to general-domain cross-video benchmarks.

Critical weaknesses
  1. The results do not show that repairing failed trajectories beats ordinary continued training. Failure-specific repair is the claimed advance over existing bootstrapping and refinement. Yet Table 3 moves from 71.27% after RL to 71.53% after repair-based SFT, then 73.57% after more RL. Most additional gain appears after RL resumes. Compare continued RL and successful- or teacher-trajectory SFT followed by RL, matching training examples and total optimization budgets.

  2. The comparison mixes better training with access to more evidence and computation. The 27.90-point gain compares a single-pass 8B backbone with an agent using an extra 4B visual model, ASR, and repeated tool calls. Add a prompt-only agent with the same backbone and tools. The agentic comparison is limited to AgentCVR; include other relevant baselines, such as VideoAgent. Compare accuracy, end-to-end latency, calls, and frames under matched evidence budgets.

Questions for authors

See weakness. And the reported author checks do not quantify benchmark label reliability. The AI Use Statement reports checks on representative examples, but gives no sample size or error rate. GPT-5.5 verifies answers from generated textual clues (§3.3), so errors shared by clues and answers can pass. Report a human review of a random sample against the original videos, including answer correctness, ambiguity, and whether multiple videos are actually required.

4 comments
AuthorsOpenReview ↗

Response to Reviewer GLrN(1/2)

We thank the reviewer for the positive assessment of the practical value, construction process, and academic contribution of AdsCVR. Below, we address your constructive comments in detail.

1. Evaluating the Effectiveness of Trajectory Rectification (weakness1)

AdSeek uses a three-stage training pipeline. We collect the RL trajectories from Stage 1, filter and rectify them to construct the SFT data for Stage 2, and thereby correct errors in the model's reasoning. Stage 3 then resumes RL exploration. This pipeline yields successive accuracy improvements of 71.27% → 71.53% → 74.30%.

To isolate the contribution of trajectory rectification in Stage 2, we provide the following ablations:

  1. WeakCheck: Use only Stage 1 trajectories with correct final answers for SFT, without further filtering or rectification.
  2. Teacher distillation: Use all trajectories generated by the teacher model as SFT data.
  3. Continued RL only: Remove the SFT stage and use continued RL throughout the remaining training.

The training settings for each applicable stage are kept consistent with those of AdSeek.

With WeakCheck, Stage 2 performs similarly to AdSeek after Stage 1, while Stage 3 incurs a slight decline of 0.5 percentage points. Because the biases in these trajectories remain unrectified, this result supports the effectiveness of the trajectory filtering and rectification used in AdSeek's Stage 2.

Direct teacher distillation improves accuracy by 0.8 percentage points in Stage 2, but its subsequent Stage 3 gain (+1.7 percentage points) is smaller than AdSeek's (+2.77 percentage points). AdSeek rectifies reasoning errors in trajectories drawn from the student's own distribution, enabling the second RL stage to better realize the model's potential.

Continued RL alone performs similarly to AdSeek after Stage 1, indicating that trajectory rectification provides an advantage for subsequent reinforcement learning in the original three-stage pipeline.

Together, these results support the effectiveness of trajectory rectification within the three-stage training pipeline.

MethodTSCSIDICODVCAvg
AdSeek (Stage1)74.672.465.872.872.669.471.27
AdSeek (Stage2)75.067.668.273.674.470.471.53
AdSeek (Stage3)77.073.470.875.674.874.274.30
WeakCheck (stage2)74.272.666.669.471.872.671.20
WeakCheck (stage3)75.268.468.870.869.871.270.70
Teacher distillation (stage2)71.472.270.871.673.672.872.07
Teacher distillation (stage3)76.874.070.274.075.871.873.77
Continued RL75.871.268.071.072.470.671.50

2. Additional Controlled Comparisons (weakness2)

ModelTSCSIDICODVCAvg
Qwen3-VL-8B52.246.841.643.648.246.046.40
AdSeek (Stage3)77.073.470.875.674.874.274.30
Qwen3-VL-8B (agentic env)64.660.458.062.662.263.061.80
VideoAgent60.857.455.759.256.256.457.62

We supplement the evaluation with the following comparisons:

  1. Evaluation of the baseline model, Qwen3-VL-8B-Instruct, in the same agentic environment as AdSeek.
  2. An additional relevant baseline, VideoAgent.
  3. Additional operational metrics for the methods.

Qwen3-VL-8B-Instruct in the agentic environment improves by 15.4 percentage points over single-pass inference. This supports the effectiveness of our multi-turn agentic framework, which allows flexible context augmentation and access to multimodal information. Within the same environment, AdSeek further outperforms Qwen3-VL-8B-Instruct(agentic env) by 12.5 percentage points, supporting the effectiveness of our training strategy.

We also include VideoAgent (ECCV 2024). To improve comparability and reduce performance differences introduced by external models, we configure its main components as follows:

  • Backbone: Qwen3-VL-8B.
  • Video-captioning model: Qwen3-VL-4B.

This configuration is similar to ours. VideoAgent achieves 57.62% on AdsCVR, providing further support for the effectiveness of our agentic environment and the improvements enabled by our training strategy.

Evaluated in our environment,AdSeek's mean end-to-end latency is 63.9 s, with visual and audio tools accounting for 50.75% and 49.25% of tool calls, respectively. VideoAgent's mean end-to-end latency is 145.6 s; its tool-call distribution is caption retrieval (26.46%), segment localization (21.24%), visual question answering (23.44%), and object-memory querying (28.86%).

AuthorsOpenReview ↗

Response to Reviewer GLrN(2/2)

3. Benchmark Reliability (question 1)

We took several measures during the construction of AdsCVR to reduce generation errors and potential bias:

  1. We described each video from four perspectives(events, objects, creative structure, and commercial logic) to reduce information omissions associated with a single descriptive perspective.
  2. We used different models for generation and verification, reducing the risk of relying on hallucinations from a single model through cross-model checks.
  3. We also conducted blind tests without video input and verified the textual clues to strengthen the benchmark's reliability and factual accuracy.

We further clarify the human audit conducted during dataset construction. Using a fixed random seed, we drew a random sample of 100 questions without replacement, covering all six evaluation dimensions. Two annotators first answered the sampled questions independently without access to the reference answers. They then recorded (1) reference-answer correctness, (2) answer unambiguity, (3) sufficiency of video evidence, and (4) multi-video dependency , retaining evidence notes for each question. The audit results are as follows:

Reference-answer correctnessAnswer unambiguityEvidence sufficiencyMulti-video dependency
Annotator 194%93%95%78%
Annotator 293%95%96%79%

This type of human spot check has precedent in video-understanding research. VideoSAVi randomly audited 200 QA pairs from its generalization test set and reported video-based answerability, category-assignment correctness, and answer uniqueness of 97.5%, 94.0%, and 98.0%, respectively( VideoSAVi, Appendix A.5.4). The results of manual verification demonstrate that our benchmark is reliable, with the vast majority of questions requiring evidence from multiple videos to be answered.

Reviewer_GLrNOpenReview ↗

Response to Author

Thank you for the additional experiments and clarifications. The new baselines and human audit address several of my concerns. I encourage the authors to include these results in the revised paper and provide more details on tool calls and processed frames. I maintain my positive score.

AuthorsOpenReview ↗

Response to Reviewer GLrN

We thank the reviewer for the constructive suggestions and positive score, and for recognizing the practical value of AdsCVR and our additional experiments validating trajectory rectification, comparing alternative methods, and assessing benchmark reliability.

We will incorporate these results into the main text and provide further details on tool calls and processed frames.

Reviewer_zBf1OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces AdsCVR, a benchmark of 6,110 questions for comparing e-commerce videos, and AdSeek, an agent that selectively acquires visual and speech evidence. AdSeek combines reinforcement learning with teacher-guided trajectory rectification and supervised fine-tuning, achieving 74.30% average accuracy on AdsCVR.

Critical strengths
  1. The benchmark addresses a practically meaningful comparison task. Comparing product advertisements requires identifying and aligning evidence distributed across videos, temporal segments, speech, and visual details. The six reasoning dimensions cover useful aspects of commercial video analysis, including creative strategy, information density, and visual composition. This provides a relevant setting for studying cross-video reasoning.

  2. The active evidence-acquisition framework is well motivated and supported by several useful experiments. Selecting video segments and modalities according to the question is a reasonable approach to locating brief but decisive evidence. The modality ablations support the complementary value of visual and speech information: restricting the system to visual or audio tools yields 63.97% and 65.03% accuracy, respectively, compared with 74.30% for the full system. The paper also includes training-stage and training-data ablations, CrossVid evaluation, and a video-overlap audit, providing useful evidence about the system’s behavior.

Critical weaknesses
  1. The reliability of the benchmark’s ground-truth answers requires stronger independent validation. Section 3.3 generates questions and answers from model-produced descriptions and metadata, then checks answer verifiability using generated textual clues. Although the automatic filtering addresses several quality concerns, consistency among descriptions, clues, and model judgments does not establish their correctness against the original videos. The reported validation does not quantify answer correctness, ambiguity, or genuine multi-video dependence through an independent human audit of the source videos. Since AdsCVR is a central contribution, uncertainty in these properties directly affects confidence in its evaluation results.

  2. The contribution of trajectory rectification is not sufficiently isolated from other training effects. Table 3 reports 71.27% after initial RL, 71.53% after rectification and SFT, 73.57% after subsequent in-domain RL, and 74.30% with mixed training data. These results demonstrate improvement across the complete pipeline, but do not isolate the benefit of targeted rectification from additional optimization, teacher supervision, or additional data. The small immediate SFT gain does not rule out a benefit during subsequent RL; however, that possibility requires controlled comparisons. Without such controls, the evidence for the proposed rectification mechanism remains limited.

Questions for authors
  1. Independent validation of benchmark labels: Do you have a human audit in which independent annotators directly inspect the original videos without seeing the model-generated answers or clues? Could you report answer correctness, ambiguity, and whether answering genuinely requires multiple videos, stratified across the six reasoning dimensions? Annotator agreement and the proportion of questions requiring revision would help establish the reliability of the benchmark.

  2. Controlled evaluation of trajectory rectification: Under matched training budgets and data, how does the proposed pipeline compare with continued RL, SFT on successful trajectories without rectification, and distillation from successful teacher trajectories? Please report and control teacher usage where applicable. Evaluating the alternatives after the subsequent RL stage would be particularly informative, because rectification may primarily benefit later optimization. Do the gains persist under these controls?

2 comments
AuthorsOpenReview ↗

Response to Reviewer zBf1

We thank the reviewer for recognizing the practical value of AdsCVR and the effectiveness of our agentic framework based on active evidence acquisition. Below, we address your constructive comments in detail.

1. Benchmark Reliability (weakness1 & question1)

We took several measures during the construction of AdsCVR to reduce generation errors and potential bias:

  1. We described each video from four perspectives(events, objects, creative structure, and commercial logic) to reduce information omissions associated with a single descriptive perspective.
  2. We used different models for generation and verification, reducing the risk of relying on hallucinations from a single model through cross-model checks.
  3. We also conducted blind tests without video input and verified the textual clues to strengthen the benchmark's reliability and factual accuracy.

We further clarify the human audit conducted during dataset construction. Using a fixed random seed, we drew a random sample of 100 questions without replacement, covering all six evaluation dimensions. Two annotators first answered the sampled questions independently without access to the reference answers. They then recorded (1) reference-answer correctness, (2) answer unambiguity, (3) sufficiency of video evidence, and (4) multi-video dependency , retaining evidence notes for each question. The audit results are as follows:

Reference-answer correctnessAnswer unambiguityEvidence sufficiencyMulti-video dependency
Annotator 194%93%95%78%
Annotator 293%95%96%79%

This type of human spot check has precedent in video-understanding research. VideoSAVi randomly audited 200 QA pairs from its generalization test set and reported video-based answerability, category-assignment correctness, and answer uniqueness of 97.5%, 94.0%, and 98.0%, respectively( VideoSAVi, Appendix A.5.4). The results of manual verification demonstrate that our benchmark is reliable, with the vast majority of questions requiring evidence from multiple videos to be answered.

2. Evaluating the Effectiveness of Trajectory Rectification (weakness2 & question2 )

AdSeek uses a three-stage training pipeline. We collect the RL trajectories from Stage 1, filter and rectify them to construct the SFT data for Stage 2, and thereby correct errors in the model's reasoning. Stage 3 then resumes RL exploration. This pipeline yields successive accuracy improvements of 71.27% → 71.53% → 74.30%.

To isolate the contribution of trajectory rectification in Stage 2, we provide the following ablations:

  1. WeakCheck: Use only Stage 1 trajectories with correct final answers for SFT, without further filtering or rectification.
  2. Teacher distillation: Use all trajectories generated by the teacher model as SFT data.
  3. Continued RL only: Remove the SFT stage and use continued RL throughout the remaining training.

The training settings for each applicable stage are kept consistent with those of AdSeek.

With WeakCheck, Stage 2 performs similarly to AdSeek after Stage 1, while Stage 3 incurs a slight decline of 0.5 percentage points. Because the biases in these trajectories remain unrectified, this result supports the effectiveness of the trajectory filtering and rectification used in AdSeek's Stage 2.

Direct teacher distillation improves accuracy by 0.8 percentage points in Stage 2, but its subsequent Stage 3 gain (+1.7 percentage points) is smaller than AdSeek's (+2.77 percentage points). AdSeek rectifies reasoning errors in trajectories drawn from the student's own distribution, enabling the second RL stage to better realize the model's potential.

Continued RL alone performs similarly to AdSeek after Stage 1, indicating that trajectory rectification provides an advantage for subsequent reinforcement learning in the original three-stage pipeline.

Together, these results support the effectiveness of trajectory rectification within the three-stage training pipeline.

MethodTSCSIDICODVCAvg
AdSeek (Stage1)74.672.465.872.872.669.471.27
AdSeek (Stage2)75.067.668.273.674.470.471.53
AdSeek (Stage3)77.073.470.875.674.874.274.30
WeakCheck (stage2)74.272.666.669.471.872.671.20
WeakCheck (stage3)75.268.468.870.869.871.270.70
Teacher distillation (stage2)71.472.270.871.673.672.872.07
Teacher distillation (stage3)76.874.070.274.075.871.873.77
Continued RL75.871.268.071.072.470.671.50
AuthorsOpenReview ↗

Response to Reviewer zBf1

Dear Reviewer,

As the discussion deadline approaches, we would appreciate it if you could let us know whether our rebuttal has addressed your concerns. If any questions remain, we would be happy to clarify. If our response has addressed your concerns, we would be grateful if you would consider revisiting your initial score in light of the clarifications.

Thank you for your time and thoughtful feedback. We hope you have a pleasant week.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.