acceptodds
Under review as a conference paper at ICLR 2027

ProVIC: Provenance-Aware Credit Assignment for Visual Tool Reasoning

Abstract

Group-based reinforcement learning (RL) trains visual-tool agents from final-answer rewards, but typically assigns the same trajectory-level credit to tool calls with different roles in evidence acquisition. Local group-based methods distinguish these calls by comparing returns under matched intermediate conditions. For visual-tool calls, however, neither ordered histories nor returned values reliably identify equivalent evidence. In particular, different acquisition orders can yield equivalent prior evidence, while identical values can arise from different image regions or calculations. To address this matching problem, we introduce ProVIC (Provenance-aware Visual Interaction Credit), which converts final-answer rewards into call-specific learning signals for visual-tool segments. By using a persistent evidence graph, ProVIC matches equivalent prior evidence across acquisition orders while preserving observation provenance. It then uses matched final-answer rewards to give each tool segment a call-specific advantage. Experiments with Qwen3-VL-8B-Instruct on five visual reasoning benchmarks show that ProVIC achieves superior performance over all the compared methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Visual tool reasoning, credit assignment, tool-augmented agents

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_Z9PzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

Paper Summary ProVIC assigns call-specific advantages to visual-tool actions using final-answer rewards. It represents acquired observations and their provenance in a persistent evidence graph, then compares returns among rollouts with matched prior evidence and tool outcomes. On five benchmarks with Qwen3-VL-8B-Instruct, it improves the mean score from 56.4 (GRPO) and 57.1 (BiPACE) to 58.4.

Critical strengths
  1. Well-motivated matching rule: Provenance distinguishes identical values obtained from different regions or calculations, while the evidence set allows matching across acquisition orders. This directly addresses a weakness of history- and value-based grouping.
  2. Consistent controlled gains: ProVIC leads all matched RL baselines on all five benchmarks and in each of five paired training seeds. The crossed ablation and replay diagnostic strengthen the case that grouping and credit estimation both contribute.
Critical weaknesses
  1. Unclear effective coverage of local credit: The paper does not report how often calls receive content-, region-, or source-level matches versus falling back to trajectory-level GRPO. Without these rates and group sizes, it is difficult to tell how broadly the proposed credit mechanism operates.
  2. Limited transfer evidence: All training results use one 8B backbone, one executor that supplies structured provenance, and an eight-call limit. The method’s utility with less reliable provenance or longer interactions remains untested.
Questions for authors
  1. What fraction of training calls use each matching level or GRPO fallback, and what are their context and transition group-size distributions?
  2. How do performance and match coverage change when executor provenance is missing or noisy?
Reviewer_eWTUOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Addressing the issue that trajectory-level credit assignment in reinforcement learning for visual tool-use is too coarse-grained, this paper proposes ProVIC. ProVIC utilizes a "persistent evidence graph" to record both the content and the origin of observations. It matches equivalent prior evidence across different collection sequences, distinguishing between observations that share the same value but originate from different sources, and subsequently estimates tool-specific advantages based on the final outcome reward through context/transition matching groups. The motivation behind this approach is clear, offering a distinct alternative to trajectory-level credit assignment methods such as GRPO.

Critical strengths

The paper correctly identifies that trajectory-level credit assignment fails to differentiate between individual visual tool calls and that relying solely on ordered history or return values ​​is insufficient for reliably determining whether pieces of evidence are equivalent.

The persistent evidence graph records both content and origin, enabling the matching of equivalent prior evidence across different collection sequences, while also distinguishing between observations that have the same value but arise from different regions or computational dependencies.

Critical weaknesses

If information regarding origins, regions, operands, or states is missing or incorrect, the quality of the matching degrades. A fallback to coarser-grained matching may inadvertently conflate observations that ought to be kept distinct.

In the absence of transition-based matching, the system may fall back to trajectory-level or context-only baselines, where the granularity of credit assignment for tool calls may be insufficient. Such coarse matching increases the support (sample size) but reduces discriminative power.

Even after matching prior evidence and accounting for remaining budget, the policy’s ordered history and earlier reasoning steps may still differ. Consequently, the resulting comparisons may reflect correlations rather than strictly causal relationships between actions and their effects.

Questions for authors

Please refer to the weaknesses.

Reviewer_HW8uOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This article points out that group-based reinforcement learning typically assigns the same trajectory-level credit to tool calls with different roles in evidence acquisition. On this basis, the ProVIC method was proposed, which converts final-answer rewards into call-specific learning signals for visual-tool segments to address this issue.

Critical strengths
  1. ProVIC identifies and addresses the genuine mismatch between visual-tool evidence and conventional local grouping
Critical weaknesses
  1. The experimental results are based only on the Qwen3-VL-8B-Instruction model, which makes the experimental conclusions unreliable
  2. Related works omit several directly relevant visual-tool RL and credit-assignment researches. The paper needs a direct comparison against prior works related to step-level credit assignment and relevant visual-tool RL methods.
Questions for authors
  1. Can you conduct experiments on more types of MLLMs to validate your method?
1 comment

thank you for your reply.

My problem is partially solved, but the contribution of this work is not sufficient to support a positive score

I will keep my score.

Comments on the paper

1 comment

Dear Reviewers,

The authors have submitted their rebuttal and tried to address the concerns raised in your reviews. If you haven't already, we would appreciate it if you could take the time to read their responses and engage in the discussion, particularly to assess whether your concerns have been adequately addressed.

Thank you for your time and valuable contributions to the review process!

Best, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.