acceptodds
Withdrawn from ICLR 2027

MAVER: Multi-Anchor Verification and Evidence Recovery for Training-Free Video Reasoning Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Reasoning video object segmentation identifies and segments objects from implicit queries about actions, relations, or temporal events. Although training-free methods use multimodal large language models (MLLMs) to interpret such queries, a plausible interpretation can still yield a coherent track of the wrong object. In this paper, we propose MAVER (Multi-Anchor Verification and Evidence Recovery), a training-free framework that treats inferred object descriptions as hypotheses and checks candidate regions against the original query before using them as tracking prompts. Multi-anchor verification searches candidate frames under a fixed hypothesis, providing alternative grounding opportunities. Early stopping ends the search at the first confirmed anchor and uses its frame and box as the single initial prompt. When an entire grounding round confirms no anchor, evidence recovery revises the hypothesis and repeats grounding within a preset round limit, keeping the original query unchanged. If verification remains inconclusive, memory-based fallback can supply an explicitly unconfirmed prompt from the retained candidates. MAVER coordinates frozen reasoning, detection, and segmentation models without task-specific fine-tuning. Extensive experiments on four referring and reasoning video segmentation benchmarks demonstrate that MAVER achieves the best overall performance among the compared methods. Moreover, MAVER improves performance while maintaining inference efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Multimodal Large Language Model, Reasoning Video Object Segmentation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_a7bxOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MAVER, a training-free framework for reasoning video object segmentation that aims to improve target selection. MAVER first uses an MLLM to generate a description of the target object, then checks candidate objects against the original query. Multi-Anchor Verification searches multiple frames and uses the first verified candidate to initialize video segmentation. If no candidate is verified, Evidence Recovery revises the target description and repeats the search. If all attempts fail, the method selects a fallback candidate from previously collected results.

Critical strengths
  1. Clear motivation. The paper identifies a key problem in existing training-free methods: a plausible object description may lead to tracking the wrong target.
  2. Strong performance. MAVER achieves strong results across four benchmarks without additional training, outperforming existing training-free methods.
  3. Convincing ablation studies. The paper provides detailed ablations on multi-anchor verification, evidence recovery, different reasoning models.
Critical weaknesses
  1. Limited contribution of hypothesis revision. Table 5 shows that most of the gains from evidence recovery come from the fallback mechanism, while hypothesis revision provides only a small additional improvement.
  2. Figure 3 is difficult to understand at first glance. A clearer presentation of the overall workflow would improve readability.
Questions for authors

see weakness

Reviewer_Guy3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes MAVER, a training-free framework for reasoning video object segmentation that combines frozen reasoning, detection, and segmentation models. It verifies candidate regions against the original query across multiple anchors under a fixed hypothesis, using the first confirmed anchor to initialize mask propagation. When an entire grounding round fails, it revises the hypothesis within a bounded number of rounds; if verification remains inconclusive, memory-based fallback supplies an explicitly unconfirmed prompt. The framework is evaluated on four existing referring and reasoning video segmentation benchmarks.

Critical strengths
  1. The paper addresses an important failure mode: consistently tracking an object that does not satisfy the original query.
  2. MAVER reports improvements over the compared methods across four referring and reasoning video segmentation benchmarks.
Critical weaknesses
  1. The main reasoning and verification process is similar to Refer-Agent. The changes mainly concern the order of search and hypothesis revision, and the addition of fallback. Please explain why these changes are a meaningful new contribution.

  2. In Table 5, fallback alone achieves a J&F score of 73.5, close to 74.1 for the full method. Please show which cases benefit from hypothesis revision and whether this gain is worth the extra cost.

  3. For the same failed queries, please compare memory-based selection with choosing the candidate with the highest detector score. This would show whether the gain comes from better candidate selection or simply avoiding empty predictions.

  4. In Table 3, MAVER uses fewer MLLM calls but takes 57 seconds per query, compared with 44 seconds for Refer-Agent using the same reasoning model. Please report the time spent in each stage and distinguish fewer calls from faster processing.

Questions for authors
  1. What evidence shows that checking other anchors before revising the hypothesis is better than Refer-Agent's reflection strategy when the same model components are used?

  2. In Table 5, the performance difference between the full method and fallback alone is only 0.6 in J&F. How often does hypothesis revision correct a wrong prediction, and how much extra time does it require?

  3. When verification fails, how does memory-based fallback compare with simply selecting the candidate with the highest detector score?

  4. In Table 3, MAVER requires more inference time despite using fewer MLLM calls. Does the efficiency improvement mainly refer to fewer calls, or does the method also reduce the total computational cost?

Reviewer_FzvoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work prsents MAVER, a training-free framework for reasoning video object segmentation. It treats an MLLM-generated target description as a retrieval hypothesis, searches multiple anchors under that fixed hypothesis, and verifies candidate regions against the original query. The first confirmed frame–box pair initializes video segmentation. A failed grounding round triggers hypothesis revision, while failure across all rounds triggers an unconfirmed fallback from cross-round candidate memory. The paper reports strong results on four referring and reasoning segmentation benchmarks, supported by component ablations and comparisons across multiple reasoners.

Critical strengths
  1. Well-motivated problem formulation and targeted method design. The paper addresses the important failure mode of consistently tracking an incorrectly identified object. It separates description-based retrieval from original-query verification and handles different levels of grounding failure through multi-anchor search and evidence recovery, yielding a coherent and practically useful inference pipeline.
  2. Broad empirical evaluation with competitive results. MAVER achieves the strongest overall results among the compared methods on four referring and reasoning video segmentation benchmarks. Comparisons across multiple reasoners and component ablations further support its applicability and demonstrate the contributions of original-query verification, multi-anchor search, and candidate fallback.
  3. Training-free deployment with modular adaptability. The framework coordinates frozen reasoning, detection, and segmentation models through structured interfaces, without additional task annotations or fine-tuning. Experiments with alternative reasoners and trackers provide empirical support for its adaptability across components.
Critical weaknesses
  1. The contribution of the core policy remains confounded by differences in the underlying components. Although Table 6 matches the reasoner, it retains method-specific detection and segmentation components, preventing performance differences from being directly attributed to multi-anchor search and recovery. The tracker replacement in Table 7 only partially addresses this issue. Without comparisons using identical candidate pools, segmenters, and inference budgets, the policy’s independent contribution over existing reflection-based frameworks remains unclear.
  2. Verification is central to the paper, yet its ability to reliably distinguish correct from incorrect targets is not directly established. Equation (4) combines attribute judgments and query matching produced by the same base MLLM into an acceptance rule, but the paper does not report false-acceptance rates, rejection rates for correct candidates, or the individual contributions of the gating conditions. Final segmentation scores cannot substitute for direct evaluation of verification quality. Consequently, “verified” remains a procedurally defined status whose actual reliability is unclear.
Questions for authors
  1. Stopping at the first confirmation is a reasonable efficiency-oriented design. However, since anchors are ordered by detection relevance, an early false acceptance could prevent subsequent correct candidates from being examined. Could the authors report the early false-acceptance rate and compare this stopping rule with multi-candidate selection or cross-frame consistency strategies under comparable computational budgets? The current anchor-count ablation does not isolate the accuracy–efficiency trade-off of the stopping rule.
  2. In Table 5, fallback alone achieves 73.5, close to the full method’s 74.1. Could the authors compare it against a single global selection and highest-scoring detector fallback using the same candidate memory and comparable inference budgets, to distinguish the contribution of cross-round evidence recovery from the benefit of simply providing a backup prediction?
  3. What fractions of queries are resolved through first-round confirmation, confirmation after revision, and fallback? What is the target localization accuracy for each path? Please specifically quantify false acceptance and rejection of correct candidates.
1 comment
Reviewer_FzvoOpenReview ↗

Post Rebuttal

Since there is no respose, I will keep my rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.