acceptodds
Under review as a conference paper at ICLR 2027

Toward Counterfactual Learning for Selective Re-Examination in Agentic Visual Search

Abstract

Visual search over high-resolution images requires a vision-language model (VLM) not only to determine where to inspect, but also whether another observation is warranted before committing to an answer. Existing prompting, demonstrations, and outcome-based reinforcement learning can encourage re-examination, yet do not directly supervise whether it is beneficial at a particular search state. We introduce RESEE, a benchmark of re-examination decisions comprising 3,227 visual-search trajectories and 8,827 explicit commit-or-re-examine decisions, with counterfactual adjudication that distinguishes states where additional inspection is warranted from those where commitment is preferable. Building on this formulation, we propose CARE (Counterfactual Advantage for Re-Examination), a decision-level reinforcement learning method that compares paired continuations under commit and re-examine verdicts from the same search state. Their cost-aware return difference estimates the counterfactual value of an additional observation and is assigned only to the corresponding verdict tokens, providing localized credit for whether search should continue. Experiments expose limited re-examination selectivity in existing VLMs and show that CARE significantly improves accuracy and decision selectivity over GRPO across two backbones, with higher accuracy and fewer visual requests on six unseen out-of-distribution benchmarks.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Search, Vision-Language Models, Selective Re-Examination, Counterfactual Credit Assignment, Reinforcement Learning, Multimodal Agents

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_ZBZeOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies the important question: "Should the vision-language model re-examine a part of the input images or directly commit to the answer at a certain decision state"? The paper first introduces the ReSee benchmark containing visual search trajectories with re-examination or commitment decisions, and baseline results show that VLMs can be bad at making such decisions even with fair answer accuracy. To alleviate this issue, the paper proposes the CARE method, which employs reinforcement learning to strengthen the model's capability to determine if a further localized visual inspection should be carried out to collect more visual information while deriving the answer. The method assigns specialized rewards for the "verdict" token spans for the decision, which empowers the model to make better evaluations of whether current evidence is sufficient and valid. Extensive experiments show consistent performance gains over visual perception/search and OOD benchmarks, and ablation studies validate the main components in the method.

Critical strengths
  1. The question studied is well chosen. Modern visual search agents suffer from the problem of making the final answer wrong even if at some intermediate state the model has the correct evidence, as well as deriving premature answer without retrieving sufficient visual evidence. The paper's solution targets directly at this problem.
  2. The method is natural and logical. Expanding both re-examination and commitment trajectories provide contrasts between two possible outcomes, which supplies a natural direction for the model to improve the decision over whether to collect more visual evidence.
  3. The experiments are extensive and generally convincing. The paper evaluates a large number of VLMs and visual search agents over the ReSee benchmark to emphasize the current models' inabilities, and compare CARE with many other methods (SFT, GRPO, etc.) to demonstrate the advantage of the proposed method. The paper also performs ablations on paired verdict contrast, hyperparameter sensitivity, counterfactual utility and decision quality at equal visual cost. Studies on OOD generalization and scaling behaviors further consolidate the paper's claims.
Critical weaknesses
  1. The benchmarking strategy on ReSee is unclear. From section 5, appendix D and the anonymous repository, I infer that the authors let VLMs generate trajectories on their own instead of performing teacher forcing on ReSee trajectories. However, it remains unexplained why transferring the counterfactual necessity label from Qwen3-VL-Plus (for generating trajectories in ReSee) to the VLM being tested is reasonable.
  2. The authors seem to have mixed "whether the visual evidence is sufficient at the current decision state" and "whether the re-examination can lead to a better answer". It is possible that the visual evidence is genuinely insufficient at the current decision state, and the model decides to re-examine, but the re-examination fails due to perception failure. In this case, the decision is considered over-examination by the paper's criterion, but in fact it has chance to improve the results. In other words, the authors do not disentangle decision error and perception error, which undermines the validity of some experimental results, such as Table 1.
  3. ReSee trajectories may be post-hoc rationalization given the reference answer instead of honest perception-based reasoning. Appendix A.1 mentions "Qwen3-VL-Plus (Bai et al., 2025a) writes each trajectory from the image, question and reference answer, which anchors the search to the target evidence." This means that the model has access to the target answer in the beginning. However, in the uploaded samples, the model responds "All visual evidence confirms the woman with backpack is to the right of the beanie-wearer, yet the answer is B (left) ... Given the ground truth is B, I must have missed the correct pair", which demonstrates that the model may initiate re-examination based on the discrepancy between current evidence and the ground truth, instead of the insufficiency of visual evidence itself. Although the authors describe filtering against explicit mention of reference answer in Appendix A.2 ("the 459 trajectories whose reasoning quotes the reference answer leave supervised training"), it is still not guaranteed that the reference answer does not interfere with re-examination decisions.
  4. The authors do not compare the training efficiency of CARE with GRPO. CARE shows higher accuracy than GRPO at the same visual cost, which implies supreme efficiency at inference time; but how much more computational resource CARE costs than GRPO at training time is still unknown.
Questions for authors
  1. What is the composition of the improvement of CARE models compared to GRPO models, i.e. what are the proportions of corrected over-examination and premature commitment?
  2. What is the precise definition of the reward R(ζ)R(\zeta), and how is it computed?
4 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/2)

Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.


W1 [Benchmarking strategy on ReSee and transfer of counterfactual labels to tested models]

Every evaluated model searches on its own without teacher forcing, and whether it re-examines on each evaluation question is compared with the adjudicated label of that question. ReSee applies the same labels to all models, which keeps decision metrics comparable. The policy-dependent utility τ^π enters only training, where CARE estimates it with its own continuations.

To check that these labels also hold for the evaluated policies, we conduct two additional adjudications of the 288 evaluation decisions. In the first, GRPO and CARE are adjudicated at their own decision states with eight self-generated continuations per verdict. In the second, Qwen3-VL-Plus regenerates each trajectory without the reference answer, and each decision state is adjudicated with four reference-blind continuations per verdict. Table R1 below shows the resulting labels and decision quality.

Table R1. Adjudication of the 288 evaluation decisions with Qwen3-VL-8B. Paired entries are GRPO/CARE.

AdjudicationWarranted decisionsAgreement with ReSee (Cohen's κ)GRPO Bal. acc.CARE Bal. acc.
ReSee labels52100% (1.00)54.865.9
Each policy's own continuations41 / 4793.4% (0.76) / 94.8% (0.82)56.768.2
Reference-blind regeneration4694.4% (0.80)55.164.9

Labels from each policy's own search agree with the ReSee labels of the same questions at Cohen's κ = 0.76 and 0.82, and labels from trajectories regenerated without the reference answer at 0.80. CARE leads GRPO in balanced accuracy under every set of labels, so evaluating these policies against the ReSee labels yields the same conclusion as evaluating each against its own labels. We have stated this evaluation protocol in App. C.1 of the revised manuscript and added Table R1 to App. A.3.


W2 [Disentangling decision error and perception error]

Thank you for this comment. Decision metrics and answer accuracy separate the two errors. In adjudication, the re-examining branch is the demonstrated continuation, which reaches the reference answer in 99.3% of trajectories, so a warranted label marks a state where committing with the gathered evidence yields an incorrect answer. Decision metrics compare the executed verdict with this fixed label (App. C.1). A re-examination at a warranted state therefore counts toward recall rather than over-examination, whether or not the model then perceives the evidence correctly, and a perception failure lowers answer accuracy instead. Tab. 1 in our paper shows that decision quality does not track answer accuracy: Qwen3.8-27B has the highest macro accuracy (70.1) but 49.3 balanced accuracy, below the 55.7 of the less accurate GLM-4.6V.

We further decompose the outcomes of GRPO and CARE on the 288 adjudicated decisions by label and executed verdict, and record the final answer after each re-examination. Table R2 below reports the results.

Table R2. Outcomes of GRPO and CARE with Qwen3-VL-8B on the 52 warranted and 236 unwarranted evaluation decisions.

LabelOutcomeGRPOCARE
WarrantedRe-examination1222
WarrantedPremature commitment4030
UnwarrantedOver-examination3225
UnwarrantedSafe commitment204211
WarrantedRe-examination ending with a correct answer7 (58.3%)14 (63.6%)
UnwarrantedOver-examination ending with an incorrect answer128

CARE re-examines at 10 more warranted decisions and 7 fewer unwarranted ones, and 63.6% of its warranted re-examinations end with a correct answer, compared with 58.3% for GRPO.


W3 [Possible post-hoc rationalization of ReSee trajectories given the reference answer]

Reasoning that invokes the reference answer, as in the cited trajectory, is excluded from supervised training by the filtering step of App. A.2, does not determine the labels, and does not explain the gains of CARE. Specifically, we address your concerns from three aspects:

(1) Construction and Filtering of Answer-Aware Reasoning. The generator receives the reference answer so that its intermediate search steps reach the target evidence instead of reflecting its own perception errors. Its reasoning can therefore state a conflict with the reference answer, as in the cited trajectory. ReSee trajectories are used only for benchmarking and the SFT cold start (App. B.2), and the 459 trajectories that quote the reference answer, including this one, leave supervised training. The labels and evaluation of the benchmark do not use the reasoning text.


Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.

AuthorsOpenReview ↗

Authors' Rebuttal (2/2)

Continued: Authors’ Rebuttal (2/2)


(2) Labels Independent of the Reasoning. As stated in the Counterfactual Adjudication paragraph of Sec. 3, labels compare the outcomes of commitment and re-examination from the same state rather than following the generator's verdict or reasoning. To test whether answer conditioning still shapes the labels, we regenerate trajectories without the reference answer and adjudicate them with reference-blind continuations. The resulting labels agree with the ReSee labels of the same questions at Cohen's κ = 0.80, and CARE still leads GRPO. Please refer to Table R1 in our response to W1.

(3) Gains of CARE Independent of the Demonstrations. No trained or evaluated policy receives the reference answer as input: RL training uses only the questions, CARE's verdict credit comes from the policy's own paired continuations, and evaluated models search on their own. Tab. 10 in our paper shows that CARE leads GRPO when the cold start contains no ReSee re-examination (Stripped, 45.1 vs. 43.9) or no ReSee trajectory (External, 44.0 vs. 42.8). The cold start alone reaches only 22.1% precision, compared with 18.1% for always re-examining, so CARE's selective verdicts come from RL rather than from the demonstrations (App. C.1).


W4 [Training efficiency of CARE compared with GRPO]

Thank you for this suggestion. We measure the training cost of GRPO and CARE on 8 NVIDIA H100 GPUs and train two compute-matched GRPO baselines. Table R3 below reports the comparison.

Table R3. Training cost and single-seed compute-matched comparisons on Qwen3-VL-8B.

MethodOutput tokensGPU hoursMacroBal. acc.Looks ↓
GRPO1.00×6246.054.82.09
GRPO, group size 101.25×7646.455.52.08
GRPO, 1.25 epochs1.25×7846.255.12.06
CARE1.24×7548.565.91.97

CARE requires only 21% more GPU hours than GRPO. Compute-matched GRPO gains at most 0.4 macro and 0.7 balanced-accuracy points, and CARE still leads the stronger variant by 2.1 and 10.4 points. We have added Table R3 to App. F of the revised manuscript.


Q1 [Composition of the improvement over GRPO]

Fewer premature commitments account for 59% of the reduction in decision errors (premature commitments plus over-examinations) on Qwen3-VL-8B and 53% on InternVL3-8B, and fewer over-examinations account for the rest. The rates of Tab. 2 on the 52 warranted and 236 unwarranted decisions give the following counts. On Qwen3-VL-8B, decision errors fall from 72 to 55, with premature commitments falling from 40 to 30 and over-examinations from 32 to 25. On InternVL3-8B, they fall from 76 to 59, with premature commitments from 38 to 29 and over-examinations from 38 to 30. In answer accuracy, CARE ends 7 more warranted re-examinations with a correct answer and 4 fewer over-examinations with an incorrect answer. Please refer to Table R2 in our response to W2.


Q2 [Precise definition and computation of the reward R(ζ)]

R(ζ) is the answer reward plus a grounding reward, and all RL methods, rollouts, and replays share it (App. C.3):

R(ζ)=1[a^(ζ)≡a⋆]+wg G(ζ),G(ζ)=1∣B⋆∣∑b∈B⋆max⁡b^∈B^(ζ)IoU(b,b^).R(\zeta)=\mathbb{1}\big[\hat a(\zeta)\equiv a^{\star}\big]+w_g\,G(\zeta),\qquad G(\zeta)=\frac{1}{|B^{\star}|}\sum_{b\in B^{\star}}\max_{\hat b\in\hat B(\zeta)}\mathrm{IoU}(b,\hat b).

Here a^(ζ)\hat a(\zeta) is the final answer of ζ\zeta, a⋆a^{\star} is the reference answer, and ≡\equiv is the rule-based match of App. C.1. B^(ζ)\hat B(\zeta) is the set of boxes inspected in ζ\zeta, and B⋆B^{\star} is the set of target boxes. G(ζ)G(\zeta) enters only for training questions whose target boxes are independently annotated, with wg=0.5w_g=0.5. R(ζ) contains no format, length, or observation-cost term. The observation cost of CARE enters only the paired contrast of Eq. (3), and each replay is scored by the same R. We have added this definition to Sec. 4 of the revised manuscript.


We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

Reviewer_ZBZeOpenReview ↗

Rebuttal Acknowledgement

Many thanks to the authors for the detailed response and additional experiments. The reference-blind and policy-specific adjudication results, together with the compute-matched comparison and the additional clarifications, have adequately addressed my main concerns. Although the faithfulness of individual reference-conditioned trajectories cannot be fully established, the authors have shown that this issue does not materially affect the main conclusions. I therefore continue to support acceptance and will maintain my current score.

Dear Reviewer ZBZe,

Thank you for your thoughtful assessment. We are pleased that our response and additional experiments have addressed your main concerns, and we sincerely appreciate your continued support for acceptance.

Best regards,
The Authors

Reviewer_ioaYOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper studies when a VLM should re-examine visual evidence rather than commit to an answer. The authors introduce ReSee, a benchmark with 3,227 trajectories and 8,827 decisions, and propose CARE, which estimates the value of re-examination through paired counterfactual rollouts and assigns the resulting advantage to decision tokens. Experiments show improvements over GRPO in answer accuracy and decision selectivity, with fewer visual requests.

Critical strengths
  1. The paper addresses an interesting problem that is often overlooked in visual search. ReSee provides a useful way to distinguish necessary re-examination from unnecessary repeated inspection.

  2. CARE is a reasonable approach to this problem. Comparing opposite decisions from the same state provides a more direct learning signal than trajectory-level rewards. The ablations in Table 3 support the importance of paired replay and localized credit assignment.

Critical weaknesses
  1. My main concern is the construction of ReSee's counterfactual labels. Appendix A.1 states that the reference trajectories are generated with access to ground-truth answers, and 99.3% of the demonstrated re-examination continuations reach the correct answer. Since re-examination utility is policy-dependent, it is unclear whether these labels reflect the benefit of re-examination for independently acting models. A reference-blind validation would help address this concern.

  2. The observation cost c(h)c(h) is not clearly defined. The paper lists several cost-related annotations but does not explain how they are converted into the scalar cost used during training, especially for newly sampled states. This matters because removing the cost term substantially reduces performance.

Questions for authors
  1. How were the 1,862 adjudicated states selected from all 8,827 decisions? Could this selection affect the evaluation distribution, and have the authors checked label agreement with reference-blind continuations?

  2. Could the authors clarify how c(h)c(h) is computed during training and whether it uses the same definition as benchmark adjudication?

  3. How much additional training compute does CARE require? Could the authors also provide uncertainty estimates for the decision metrics?

5 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/2)

Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.


W1 [Construction of the counterfactual labels and reference-blind validation]

The generator sees the reference answer only during trajectory construction, where conditioning on the question and its reference answer enables it to write gold intermediate search steps anchored to the target evidence. Labels compare the outcomes of re-examination and commitment from the same state, with the reference answer used only to grade these outcomes. ReSee scores every model against these fixed labels, whereas the policy-dependent utility τ^π enters only CARE's training, which estimates it with the policy's own continuations.

To validate the labels directly, we conduct two additional adjudications of the 288 evaluation decisions. In the first, Qwen3-VL-Plus regenerates each trajectory without the reference answer, and each decision state is adjudicated with four reference-blind continuations per verdict. In the second, GRPO and CARE are adjudicated at their own decision states with eight self-generated continuations per verdict, so the labels depend on each policy. Table R1 below shows the resulting labels and decision quality.

Table R1. Adjudication of the 288 evaluation decisions with Qwen3-VL-8B policies. Paired entries are GRPO/CARE, and the ReSee success rate covers all trajectories.

AdjudicationSuccessful re-examinationsWarranted decisionsAgreement with ReSee (Cohen's κ)GRPO Bal. acc.CARE Bal. acc.
ReSee labels99.3%52100% (1.00)54.865.9
Reference-blind regeneration63.9%4694.4% (0.80)55.164.9
Each policy's own continuations55.4% / 60.8%41 / 4793.4% (0.76) / 94.8% (0.82)56.768.2

These adjudications include realistic search failures, with 36.1% to 44.6% of the re-examination continuations answering incorrectly. Their labels agree with the ReSee labels of the same questions at Cohen's κ = 0.80 without the reference answer and at 0.76 and 0.82 with each policy's own continuations, and CARE leads GRPO in balanced accuracy by 9.8 to 11.5 points under all of them, including when each policy is scored against its own labels. We have added Table R1 to App. A.3 of the revised manuscript.


W2 & Q2 [Definition of the observation cost c(h) in training and adjudication]

In Eq. (1) and Eq. (2), c(h) is the cost of the additional observation itself rather than a conversion of the annotated cost estimates. It is therefore defined for every newly sampled state, and benchmark adjudication uses the same definition.

In training, c(h_j) is the number of visual tokens of the observation that re-examination adds at h_j, priced at λ = 2×10⁻⁴. It follows directly from the requested region in whichever branch re-examines, the on-policy rollout or the replay, so it needs no annotation. The cost estimates in Tab. 5 are descriptive annotations and do not enter training. Adjudication prices re-examination with the same c(h) and scores answer correctness.

Each observation is bounded at 1,024 pixels on its longer side (App. A.2), so c(h) stays below 1/λ = 5,000 visual tokens and λc(h) below the unit answer reward. App. A.1 in our paper states that, for any priced cost λc(h) in (0, 1), a state is warranted exactly when re-examination answers correctly and commitment does not, so the ReSee labels are identical under any other cost in this range. In training, λc(h) enters only the paired contrast of Eq. (3). A re-examination that corrects the answer therefore keeps a positive contrast, whereas one that leaves the return unchanged receives −λc(h). Without the price, unnecessary re-examinations receive zero credit, which explains why over-examination nearly doubles (Tab. 3). We have stated in Sec. 4 of the revised manuscript that c(h) counts the visual tokens of the observation.


Q1 [Selection of the 1,862 adjudicated states and its effect on the evaluation distribution]

The 1,862 states form a source-stratified random sample in which re-examinations are oversampled, since they make up only 12.8% of all decisions. This yields comparable numbers of re-examinations and commitments (883 and 979), both of which are required to measure selectivity. The selection does not affect answer accuracy, which uses all 980 questions, and the 288 adjudicated evaluation decisions include warranted cases from all six sources.


Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.

AuthorsOpenReview ↗

Authors' Rebuttal (2/2)

Continued: Authors’ Rebuttal (2/2)


To check whether the selection shifts the label rates, we additionally adjudicate a verdict-stratified random sample of 150 re-examinations and 450 commitments outside the set with the same protocol. Table R2 below compares the two sets.

Table R2. Label rates within each verdict for the adjudicated set and for a random sample of decisions outside it.

Decision setRe-examinationsWarrantedCommitmentsSafe
Adjudicated set88331.6%97993.8%
Random sample outside the set15029.3%45095.1%

The Wilson 95% intervals of the sample, [22.6, 37.1] and [92.7, 96.7], contain the rates of the adjudicated set, so the selection introduces no detectable shift in the label rate within each verdict. Label agreement with reference-blind continuations is Cohen's κ = 0.80. Please refer to Table R1 in our response to W1. We have stated the selection rule in App. A.1 of the revised manuscript.


Q3 [Additional training compute and uncertainty of the decision metrics]

Thank you for these questions. We would like to address your concerns from two aspects below:

(1) Additional Training Compute. We measure the training cost of GRPO and CARE on 8 NVIDIA H100 GPUs and train GRPO with matched sampling or GPU hours. Table R3 below reports the comparison.

Table R3. Training cost and single-seed compute-matched comparisons on Qwen3-VL-8B.

MethodOutput tokensGPU hoursMacroBal. acc.Looks ↓
GRPO1.00×6246.054.82.09
GRPO, group size 101.25×7646.455.52.08
GRPO, 1.25 epochs1.25×7846.255.12.06
CARE1.24×7548.565.91.97

CARE requires only 21% more GPU hours than GRPO, because replays are much shorter than full rollouts: each starts at a decision state, and a replay under imposed commitment ends with an answer. Compute-matched GRPO gains at most 0.4 macro and 0.7 balanced-accuracy points, and CARE still leads the stronger variant by 2.1 and 10.4 points.

(2) Uncertainty of the Decision Metrics. We train GRPO and CARE with three seeds per backbone and compute two-level paired bootstrap intervals with 10,000 resamples of the seeds and of the 288 adjudicated decisions. Table R4 below reports the results.

Table R4. Three-seed mean ± standard deviation of the decision metrics, with paired bootstrap 95% confidence intervals for the differences between CARE and GRPO.

BackboneMethodRecallPrec.Over-ex. ↓Bal. acc.
Qwen3-VL-8BGRPO23.7 ± 1.927.0 ± 1.414.1 ± 1.054.8 ± 1.1
Qwen3-VL-8BCARE41.0 ± 2.245.3 ± 1.810.9 ± 0.865.1 ± 1.2
Qwen3-VL-8BDifference [95% CI]+17.3 [3.7, 31.2]+18.3 [6.3, 30.5]−3.2 [−6.3, −0.5]+10.3 [4.8, 15.9]
InternVL3-8BGRPO26.5 ± 1.126.9 ± 1.515.9 ± 0.955.3 ± 0.7
InternVL3-8BCARE43.6 ± 1.943.3 ± 1.612.6 ± 0.765.5 ± 1.0
InternVL3-8BDifference [95% CI]+17.1 [3.3, 30.4]+16.4 [4.6, 29.0]−3.3 [−6.4, −0.4]+10.2 [4.4, 15.7]

On both backbones, every paired 95% confidence interval of the difference excludes zero, and the standard deviation of balanced accuracy across seeds is at most 1.2 points, compared with a gap of 10.2 to 10.3 points. We have added Tables R3 and R4 to App. F of the revised manuscript.


We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

Dear Reviewer ioaY,

Thank you for your positive assessment of our work and your constructive feedback.

As the discussion period draws to a close, we would appreciate knowing whether our responses have adequately addressed your concerns. We would also be happy to provide any further clarification during the remaining discussion period.

Best regards,
The Authors

Reviewer_7U7QOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the problem of deciding when an agentic vision-language model (VLM) should re-examine previously inspected image regions rather than commit to an answer based on the current evidence. The authors introduce ReSee, a benchmark containing 3,227 visual-search trajectories, 8,827 explicit commit-or-re-examine decisions, and 1,862 counterfactually adjudicated cases. They further propose CARE, a reinforcement learning method that estimates the value of re-examination by comparing paired continuations under opposite decisions from the same search state and assigns the resulting cost-aware advantage specifically to the verdict tokens.

Experiments with Qwen3-VL-8B and InternVL3-8B show improvements over GRPO in both answer accuracy and re-examination selectivity. The paper also reports favorable results on six out-of-distribution benchmarks, together with ablation studies, parameter sensitivity analyses, and a theoretical analysis of the proposed decision-level credit assignment mechanism.

Overall, the paper addresses an important problem in agentic visual search and presents a coherent combination of benchmark construction, counterfactual estimation, and reinforcement learning.

Critical strengths
  1. The paper explicitly formulates commit-or-re-examine as a state-dependent decision rather than treating additional inspection as an intrinsically beneficial behavior. This is a meaningful conceptual contribution: an agent should gather more evidence when its expected benefit exceeds its cost, rather than simply maximizing the frequency of verification. The distinction between answer correctness and decision quality is particularly valuable for evaluating interactive visual agents.

  2. CARE combines paired counterfactual continuations, observation-cost adjustment, and localized credit assignment to the decision tokens. The paper includes relevant ablations and sensitivity analyses, and evaluates the approach using two VLM backbones. On the reported six-source evaluation split, CARE improves macro accuracy from 46.0 to 48.5 for Qwen3-VL-8B and from 44.6 to 46.9 for InternVL3-8B, while also improving balanced decision accuracy. The reported gains on six additional benchmarks provide encouraging evidence that the approach may transfer beyond the benchmark used for training.

Critical weaknesses
  1. According to Appendix A.1, the trajectories are generated by Qwen3-VL-Plus using the image, question, and reference answer. The paper also reports that the demonstrated re-examination continuation reaches the reference answer in 99.3% of trajectories. For counterfactual adjudication, answer correctness is used as the reward, together with a priced observation cost that is smaller than one.
    Under this setup, a successful re-examination has positive utility precisely when immediate commitment produces an incorrect answer. Consequently, the adjudicated label may largely reflect whether the current answer is wrong, rather than a more general assessment of whether additional inspection is worth its cost under realistic, potentially unsuccessful search behavior. The fact that the trajectory generator has access to the reference answer may further influence the choice of inspected regions and the quality of the re-examination continuation.
    The paper does include human evaluation, which is helpful, but the reported adjudication-agreement assessment involves 120 decisions and achieves 83.3% agreement with the adjudicated labels. This does not fully establish that the benchmark captures the intended counterfactual utility in less curated settings. A clearer separation between answer-aware trajectory construction, counterfactual outcome evaluation, and independent validation would strengthen the benchmark's validity.

  2. The decision metrics are evaluated on 288 adjudicated decisions, including 52 warranted re-examinations in the utility-ranking evaluation. Although the results are promising, this evaluation set is relatively small, and the paper does not report confidence intervals or significance tests for the principal decision-quality metrics. The reported run-to-run standard deviations for macro accuracy are useful, but they do not by themselves establish robustness to training-seed variation.
    Furthermore, CARE requires additional counterfactual rollouts, with up to three decision states considered per trajectory in the main configuration. The paper demonstrates that CARE can reduce the number of visual requests during evaluation, but it does not sufficiently quantify the additional training-time computation or establish whether its gains persist against compute-matched baselines. This distinction matters when assessing the practical efficiency of the method: reducing inference-time observations may come at the cost of substantially more training-time sampling.

Questions for authors
  1. How robust are the ReSee labels to the use of reference-answer-aware trajectory generation?

Since the trajectory generator receives the reference answer, could the authors report results on an independently constructed subset in which trajectories are generated without access to the reference answer, or provide additional human validation of whether the counterfactual labels reliably reflect the marginal utility of re-examination? In particular, how much of the benchmark's decision signal is explained by whether the current candidate answer is incorrect?

  1. Are the decision-quality improvements statistically robust, and how do they compare under matched training computation?

Could the authors provide uncertainty estimates for recall, precision, over-examination, and balanced accuracy, preferably across independent training seeds? Additionally, could CARE be compared with GRPO under a matched rollout or training-compute budget, accounting for the extra opposite-verdict replays? This would help establish whether the observed gains arise from localized counterfactual credit assignment rather than additional sampling alone.

  1. How sensitive are the results to the observation-cost definition?

The cost-adjusted utility is central to both the benchmark labels and the training objective. Could the authors clarify how the cost function and its scale are calibrated relative to the binary answer reward, and whether the conclusions remain stable under alternative cost definitions or cost distributions? This would help establish the extent to which the method learns general selective re-examination behavior rather than a decision boundary specific to the chosen cost parameterization.

3 comments
AuthorsOpenReview ↗

Authors' Rebuttal (1/2)

Thank you for providing us with your valuable feedback. We have carefully considered your comments and would like to address them as below.


W1 & Q1 [Robustness of ReSee labels to reference-answer-aware trajectory generation]

ReSee keeps construction, adjudication, and validation separate. Its labels remain stable under two additional adjudications, one without the reference answer and one with each policy's own continuations, and CARE leads GRPO under every adjudication. Their dependence on answer correctness is the measured utility itself rather than a proxy. We would like to address your concerns from three aspects below:

(1) Separation of Construction, Adjudication, and Validation. The generator sees the reference answer only during trajectory construction. This access is needed for gold trajectories: given the question and its reference answer, the generator writes intermediate search steps that reach the target evidence instead of reflecting its own perception errors. Labels compare the outcomes of re-examination and immediate commitment from the same state, and the reference answer serves only to grade these outcomes. Validation is independent of both, as the raters of Tab. 8 see neither adjudication labels nor reference answers.

(2) Two Additional Adjudications. In the first, Qwen3-VL-Plus regenerates each trajectory without the reference answer, and each decision state is adjudicated with four reference-blind continuations per verdict. In the second, GRPO and CARE are adjudicated at their own decision states with eight self-generated continuations per verdict. Table R1 below shows the resulting labels and decision quality.

Table R1. Adjudication of the 288 evaluation decisions with Qwen3-VL-8B policies. Paired entries are GRPO/CARE, and the ReSee success rate covers all trajectories.

AdjudicationSuccessful re-examinationsWarranted decisionsAgreement with ReSee (Cohen's κ)GRPO Bal. acc.CARE Bal. acc.
ReSee labels99.3%52100% (1.00)54.865.9
Reference-blind regeneration63.9%4694.4% (0.80)55.164.9
Each policy's own continuations55.4% / 60.8%41 / 4793.4% (0.76) / 94.8% (0.82)56.768.2

These adjudications include realistic search failures, with 36.1% to 44.6% of the re-examination continuations answering incorrectly, and their labels agree substantially with the ReSee labels of the same questions at Cohen's κ = 0.76 to 0.82. CARE leads GRPO in balanced accuracy by 9.8 to 11.5 points under the new labels.

(3) Role of Answer Correctness. Under the answer reward of Eq. (1), an additional observation has value only if it corrects a commitment that would fail, and the policy never observes correctness, so it must recognize such states from the evidence alone. An incorrect commitment is necessary but not sufficient for a warranted label, since re-examination can also fail. Under the two new adjudications, 61.7% to 70.3% of the states with an incorrect commitment are labeled unwarranted for this reason. We have added the protocols of the two adjudications and Table R1 to App. A.3 of the revised manuscript.


W2 & Q2 [Statistical robustness of the decision-quality improvements and comparison under matched training computation]

Thank you for these suggestions. We would like to address your concerns from two aspects below:

(1) Statistical Robustness of the Decision-Quality Gains. The 288 adjudicated decisions are only the subset with the counterfactual labels that decision metrics require. Tab. 2 and Tab. 4 in our paper evaluate answer accuracy and visual requests on all 980 questions of the six sources and on 4,943 questions from six unseen benchmarks. We train GRPO and CARE with three seeds per backbone in the setting of Tab. 2. We compute the intervals with a two-level paired bootstrap of 10,000 resamples, first over the three seeds and then over the 980 questions or the 288 adjudicated decisions. Table R2 below reports the results.

Table R2. Three-seed mean ± standard deviation, with paired bootstrap 95% confidence intervals for the differences between CARE and GRPO.

BackboneMethodMacroRecallPrec.Over-ex. ↓Bal. acc.
Qwen3-VL-8BGRPO45.9 ± 0.723.7 ± 1.927.0 ± 1.414.1 ± 1.054.8 ± 1.1
Qwen3-VL-8BCARE48.3 ± 0.841.0 ± 2.245.3 ± 1.810.9 ± 0.865.1 ± 1.2
Qwen3-VL-8BDifference [95% CI]+2.4 [1.2, 3.5]+17.3 [3.7, 31.2]+18.3 [6.3, 30.5]−3.2 [−6.3, −0.5]+10.3 [4.8, 15.9]
InternVL3-8BGRPO44.7 ± 0.626.5 ± 1.126.9 ± 1.515.9 ± 0.955.3 ± 0.7
InternVL3-8BCARE46.6 ± 0.843.6 ± 1.943.3 ± 1.612.6 ± 0.765.5 ± 1.0
InternVL3-8BDifference [95% CI]+1.9 [0.6, 3.1]+17.1 [3.3, 30.4]+16.4 [4.6, 29.0]−3.3 [−6.4, −0.4]+10.2 [4.4, 15.7]

Please see the subsequent Authors' Rebuttal (2/2) for remaining responses.

AuthorsOpenReview ↗

Authors' Rebuttal (2/2)

Continued: Authors’ Rebuttal (2/2)


The decision-quality gains hold across the three seeds, and even on the 288 adjudicated decisions every paired 95% confidence interval of the difference excludes zero. The standard deviation of balanced accuracy is at most 1.2 points, compared with a gap of 10.2 to 10.3 points, and a paired permutation test over decisions, with verdicts pooled across seeds, gives p < 0.001 on both backbones.

(2) Training Computation and Compute-Matched Baselines. We measure the training cost of GRPO and CARE on 8 NVIDIA H100 GPUs and train two compute-matched GRPO baselines. Table R3 below reports the comparison.

Table R3. Training cost and single-seed compute-matched comparisons on Qwen3-VL-8B.

MethodOutput tokensGPU hoursMacroBal. acc.Looks ↓
GRPO1.00×6246.054.82.09
GRPO, group size 101.25×7646.455.52.08
GRPO, 1.25 epochs1.25×7846.255.12.06
CARE w/o verdict flip1.24×7544.253.82.12
CARE1.24×7548.565.91.97

CARE incurs only 21% additional GPU hours over GRPO, as counterfactual replays start from intermediate decision states and commitment replays proceed directly to an answer. Raising GRPO to CARE's sampling or GPU-hour budget improves its macro and balanced accuracy by at most 0.4 and 0.7 points, respectively, compared with 2.5 and 11.1 points for CARE. At the same replay budget, the ablation without verdict flip (Tab. 3 of our paper), which compares each verdict with itself, performs below GRPO. As established in Cor. 2, same-verdict replay cancels the difference in decision values in expectation, indicating that CARE's gains arise from opposite-verdict credit assignment rather than additional sampling. We have added Tables R2 and R3 to App. F of the revised manuscript.


Q3 [Sensitivity of the results to the observation-cost definition]

The ReSee labels do not depend on the cost definition, and CARE retains its advantage under alternative costs. We would like to address your concerns from three aspects below:

(1) Calibration Relative to the Answer Reward. The cost c(h) is the number of visual tokens of the additional observation. Each observation is bounded at 1,024 pixels on its longer side (App. A.2), so c(h) stays below 1/λ = 5,000 visual tokens and the priced cost λc(h) below the unit reward of a correct answer. A re-examination that corrects the answer therefore keeps a positive contrast in Eq. (3), whereas one that leaves the return unchanged receives −λc(h). Removing the price gives unnecessary re-examinations zero credit, and over-examination nearly doubles (Tab. 3).

(2) Labels Independent of the Cost Scale. App. A.1 in our paper states that, with answer correctness as the reward and any priced cost λc(h) in (0, 1), a state is warranted exactly when re-examination answers correctly and commitment does not. Every ReSee label is therefore identical under any cost definition or distribution in this range.

(3) Alternative Cost Definitions. Fig. 6 and Tab. 14 in our paper show that CARE exceeds GRPO trained at the same price, in both macro and balanced accuracy, at every price across the 25-fold range from 2×10⁻⁴ to 5×10⁻³. We conduct new experiments with three alternative cost definitions, and Table R4 below reports the results.

Table R4. CARE under alternative observation-cost definitions on Qwen3-VL-8B. Visual-token cost is the default, and alternative costs are calibrated to the same mean priced cost.

Cost definitionMacroBal. acc.Looks ↓
GRPO46.054.82.09
Visual tokens of the observation48.565.91.97
Constant cost per observation48.164.71.99
Fraction of the image covered by the requested region48.265.21.95
Price λ drawn log-uniformly from [5×10⁻⁵, 5×10⁻⁴] per decision48.064.91.94

All alternatives stay within 0.5 macro points of the default and lead GRPO by at least 2.0 macro and 9.9 balanced-accuracy points, so CARE learns selective re-examination rather than a boundary tied to one cost parameterization. We have stated in Sec. 4 of the revised manuscript that c(h) counts the visual tokens of the observation and added Table R4 to App. F.


We hope this fully addresses your questions and comments, and remain happy to engage in further discussion for the rest of the rebuttal period.

Dear Reviewer 7U7Q,

Thank you for your positive assessment of our work and your constructive feedback.

As the discussion period draws to a close, we would appreciate knowing whether our responses have adequately addressed your concerns. We would also be happy to provide any further clarification during the remaining discussion period.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

Dear Reviewers, AC, SAC, and PC,

Thank you for your time and effort in reviewing and handling our paper. We have posted point-by-point responses to each reviewer and summarize the main points below.


Paper Overview and Strengths

[Paper Overview]: This paper proposes CARE, a decision-level RL method that assigns the cost-aware return difference between paired continuations under the commit and re-examine verdicts only to the verdict tokens, together with ReSee, a benchmark of counterfactually adjudicated commit-or-re-examine decisions. Across six source benchmarks and two backbones, CARE improves accuracy and decision selectivity over GRPO, with higher accuracy and fewer visual requests on six unseen benchmarks.

[Strengths]: The reviewers consider the problem important and well chosen (7U7Q, ZBZe) and often overlooked (ioaY), recognize the paired counterfactual contrast of CARE (7U7Q, ioaY, ZBZe) as a more direct signal than trajectory-level rewards (ioaY), and note that ReSee separates necessary from unnecessary re-examination (ioaY) and decision quality from answer correctness (7U7Q). They also find the ablations supportive of the design (ioaY, ZBZe), the evaluation extensive and generally convincing (ZBZe), and the gains on six unseen benchmarks encouraging evidence of transfer (7U7Q, ZBZe).


Additional Experiments for Concerns

We conduct the following experiments.

  • [Reference-Blind and Policy-Specific Adjudication] (7U7Q, ioaY, ZBZe): We re-adjudicate the 288 evaluation decisions with reference-blind regenerated trajectories and with the continuations of GRPO and CARE (Table R1 for each reviewer). They agree with the ReSee labels at Cohen's κ = 0.76 to 0.82, and CARE leads GRPO in balanced accuracy by 9.8 to 11.5 points under all labels.
  • [Seeds and Confidence Intervals] (7U7Q, ioaY): With three seeds per backbone (Table R2 for 7U7Q, Table R4 for ioaY), every paired 95% confidence interval excludes zero, with a balanced-accuracy gap of 10.2 to 10.3 points against a seed standard deviation of at most 1.2 points.
  • [Training Compute] (7U7Q, ioaY, ZBZe): CARE uses 21% more GPU hours than GRPO (Table R3 for each reviewer). Compute-matched GRPO gains at most 0.4 macro and 0.7 balanced-accuracy points and still trails CARE by 2.1 and 10.4 points.
  • [Alternative Cost Definitions] (7U7Q): Three alternative cost definitions stay within 0.5 macro points of the default and lead GRPO by at least 2.0 macro and 9.9 balanced-accuracy points (Table R4 for 7U7Q).
  • [Selection of the Adjudicated States] (ioaY): For 600 random decisions outside the adjudicated set, the within-verdict label rates have Wilson intervals that contain those of the adjudicated set (Table R2 for ioaY).
  • [Decision and Perception Errors] (ZBZe): CARE re-examines at 10 more warranted and 7 fewer unwarranted decisions than GRPO, and 63.6% of its warranted re-examinations end correctly, compared with 58.3% for GRPO (Table R2 for ZBZe).

Summary of Responses to Reviewers' Questions

  • [Role of the Reference Answer] (7U7Q, ioaY, ZBZe): The generator sees the reference answer only during trajectory construction, and labels compare the outcomes of re-examination and commitment from the same state. No trained or evaluated policy receives it, the 459 trajectories that quote it leave supervised training, and CARE leads GRPO without ReSee trajectories in the cold start (Tab. 10).
  • [Observation Cost c(h)] (7U7Q, ioaY): c(h) is the number of visual tokens of the additional observation, priced at λ = 2×10⁻⁴ with λc(h) below the unit answer reward. Training and adjudication use the same definition, and the ReSee labels are identical for any priced cost in (0, 1).
  • [Evaluation Protocol and Error Attribution] (ZBZe): Every evaluated model searches on its own and is scored question by question against the adjudicated labels. A re-examination at a warranted state counts toward recall even if the model then misperceives the evidence, which lowers answer accuracy instead. Fewer premature commitments account for 59% and 53% of the reduced decision errors on the two backbones.
  • [Reward R(ζ)] (ZBZe): R(ζ) is answer correctness plus 0.5 times the grounding IoU on independently annotated boxes, shared by all RL methods, rollouts, and replays.

Revision During Rebuttal

  • [Sec. 4]: the definitions of c(h) and R(ζ).
  • [App. A.1]: the selection rule of the adjudicated states.
  • [App. A.3]: the two additional adjudications (Table R1).
  • [App. C.1]: the evaluation protocol described above.
  • [App. F]: the seed, training-cost, and alternative-cost results (Tables R2 to R4 for 7U7Q).

All changes are highlighted in blue in the revised manuscript.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.