- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes TACT (Tool-Aware Credit Transfer), an advantage-correction mechanism that alleviates "credit misassignment" in group-based policy optimization (e.g., GRPO) for multimodal search agents. Existing methods broadcast a single trajectory-level advantage to every token, assuming all steps contribute equally. This breaks down in multi-step tool use: failed trajectories often contain tool calls that actually retrieved the needed information, yet they are penalized together with useless steps. Their key insight is that, for information-acquisition tools, the call parameters encode the retrieval intent, so successful matched calls serve as a counterfactual reference to compensate wrongly-penalized steps. Experiments on Qwen3-VL across 7 multimodal benchmarks show consistent gains.
Critical strengths
- Well-motivated problem. The misassignment phenomenon is real and quantified, making the diagnosis convincing.
- Plug-and-play and zero extra cost. TACT requires no annotation, no extra model.
- Broad evaluation: 7 multimodal benchmarks plus text-only validation, with fairly complete ablations.
Critical weaknesses
- Sensitivity of key hyperparameters (θ, α_min) and their cross-task transferability is not systematically analyzed.
- How matching is decided (rule/embedding/LLM) strongly affects results; matching errors propagate directly. Authors should discuss about it.
Questions for authors
How does the method degrade when successful trajectories are scarce and the reference bank is sparse?
Thank you for recognizing the motivation, mechanism, and experiments of our work, and for suggesting equal-budget controls. The 79.4% versus 27.5% result in the independent tool-response evaluation supports the informational value of the selected calls. Eq. (2) analyzes the credit discrepancy, while Eq. (4) provides heuristic compensation, with the original advantage of the failing trajectory still retained. In the newly added equal-budget controls, TACT achieves an average accuracy of 59.71%, a relative improvement of 5.27% / 4.74% over Uniform / Random, with higher accuracy on all seven benchmarks. These results support the additional value of selective compensation; simply attenuating negative advantages does not reproduce all the gains. We explain each point below.
Q1: Does parameter similarity imply an incorrect penalty?
Brief summary: TACT matches information-seeking intent, rather than whether subsequent reasoning succeeds; higher similarity and greater support across successful trajectories provide a stronger structural basis for attenuating the negative advantages of actions in failing trajectories.
Parameter similarity does not necessarily imply an inappropriate penalty, but failed reasoning does not negate the value of necessary information. For the same question and tool type, parameters specify the information-acquisition target and can provide an intent-based reference for cross-trajectory comparison, rather than assuming identical contributions across contexts.
The independent tool-response evaluation shows that TACT can preferentially select useful calls. The proportions of triggered and non-triggered steps containing question-necessary information are 79.4% versus 27.5%; the proportion is 83.0% on the 265-step author-annotated subset, and the F1 score between the LLM and author annotations is 0.87. Figure 2 illustrates the matching phenomenon, while the tool-response evaluation examines informational value; neither is causal proof for each individual match.
To address retrieval noise or redundant evidence, TACT constrains compensation through weighting, gating, and a clamp at zero, only attenuating negative advantages without assigning positive advantages. In the evaluated GRPO, GSPO, and SAPO settings, average accuracy improves by approximately 4%–6% relative to the respective baselines.