- Rating
- 2(+1)
- Confidence
- 4(+1)
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This work explores how multimodal shopping agents can obtain relevant visual information efficiently when evaluating products on e-commerce platforms. Rather than requiring an agent to examine every image on a product detail page, the authors aim to enable targeted inspection of visual content needed to verify specific product properties. To study this problem, they establish two evaluation settings: Fixed-Item Question Answering, which measures the ability to interpret multimodal information about a given product, and Open-Ended Product Recommendation, which examines the agent's performance throughout the broader product discovery and selection process. The proposed system separates the main agent from a dedicated visual evidence component, allowing it to locate potentially useful images through coarse indexing and extract specific supporting information through more detailed inspection.
The authors further introduce a post-training approach that integrates a progressively organized supervised fine-tuning curriculum with reinforcement learning under restricted interaction budgets. Supervised training first develops the model's ability to interpret individual products before extending to complete recommendation tasks, while the subsequent RL stage encourages effective evidence gathering with limited opportunities for tool interaction. Results obtained with 4B and 9B models indicate improvements in both product question answering and recommendation performance. Under a four-turn interaction limit, the optimized 9B model reaches 46.8% Qualified@4, slightly surpassing the reported 45.6% achieved by GPT-5.6-Sol without tool-use restrictions. The paper also includes ablations investigating the effects of curriculum organization, interaction-budget scheduling, and access to visual evidence.
Critical strengths
1. A relevant research problem accompanied by a well-structured evaluation setting.
A notable strength of this work is its focus on a practical limitation of multimodal shopping agents: important product characteristics are frequently communicated through images rather than structured descriptions, making selective visual inspection essential for reliable decisions. The two proposed tasks examine this challenge at different levels. Fixed-Item QA isolates the agent's ability to understand and verify information about individual products, whereas Open-Ended Recommendation evaluates how those capabilities translate into a complete shopping workflow. Another valuable aspect is the explicit separation of questions that can be answered from text and those requiring visual evidence. This distinction reduces the risk that strong results merely reflect textual shortcuts and makes the evaluation more informative about genuine multimodal capabilities. Together, these design choices establish a useful experimental setting for investigating visual evidence gathering in e-commerce applications.
2. Convincing performance gains with experiments that clarify the value of the training procedure.
The empirical findings indicate that the proposed training approach can considerably strengthen the capabilities of relatively compact models. In the recommendation setting, the 9B agent increases its unconstrained Qualified@4 score from 16.9% to 49.5%, and retains a competitive 46.8% even when restricted to four interaction turns. Beyond these overall gains, the ablations offer useful evidence about which training decisions matter. In particular, organizing supervised fine-tuning into successive stages yields better results than combining all training examples from the beginning, while RL with gradually tightened interaction limits helps the agent operate more effectively under constrained conditions. These observations support the practical value of the proposed training procedure for developing smaller multimodal shopping agents with stronger task performance.
Critical weaknesses
1. The methodological contribution is not sufficiently differentiated from prior work.
While the overall framework is coherent, the individual techniques largely build upon ideas that have already received considerable attention in related research. Separating visual perception from agent reasoning, retrieving evidence through coarse-to-fine processing, arranging supervised training into stages, and optimizing tool usage under budget constraints are not fundamentally new directions. Existing work, including EcomMMMU, MMShopBench, MindFlow, and AdaTurn, has addressed closely connected problems involving multimodal shopping, visual evidence processing, agent-based search, and resource-aware perception. However, the experimental evaluation does not establish a clear performance comparison with representative specialized methods from these lines of research. Instead, the reported improvements are mainly demonstrated against general-purpose models or alternative configurations of the authors' own pipeline. This makes it challenging to distinguish the benefits of the proposed technical design from those attributable to additional training on the target tasks. As a result, the extent of the paper's methodological originality and its advantages over existing specialized approaches remain uncertain.
2. The efficiency claims are not adequately validated by direct resource measurements.
Although reducing the cost of acquiring visual information is presented as a primary motivation, the experiments do not convincingly establish that the proposed approach requires fewer computational resources. The main efficiency indicator is the number of interaction turns, which captures sequential tool-use depth but does not directly measure total inference workload. An agent may complete a task in fewer turns while simultaneously issuing more visual processing requests. Indeed, Table 11 indicates that applying RL increases the mean number of visual tool calls in the unrestricted setting from 6.4 to 12.8. This observation suggests that some performance improvements may be accompanied by greater visual processing demand rather than actual computational savings. The absence of systematic measurements of inference latency, token usage, and overall processing cost makes it difficult to assess this trade-off. Furthermore, the paper does not sufficiently evaluate whether selective
Questions for authors
1. Performance relative to specialized multimodal shopping methods.
Most reported baselines involve general-purpose models, leaving the relationship between this framework and existing specialized approaches insufficiently explored. Could the authors evaluate representative systems or techniques associated with MMShopBench, MindFlow, and AdaTurn in a comparable experimental setting? Such results would help clarify whether the reported advantages stem from the proposed framework itself or from its task-specific post-training procedure.
2. Computational cost beyond interaction-turn counts.
Using the number of tool-use turns as the primary efficiency metric may obscure differences in actual resource consumption, particularly when visual requests are processed concurrently. Moreover, Table 11 reports an increase in average visual calls from 6.4 to 12.8 following RL in the unrestricted setting. Could the authors provide measurements of total inference time, token usage, and visual processing overhead for the evaluated approaches? It would be important to establish whether the method truly reduces computational expenditure or primarily achieves fewer sequential steps by performing more operations in parallel.
3. Advantages of the coarse-to-fine visual evidence mechanism.
While the ablation results indicate that access to visual information benefits agent performance, they do not fully isolate the advantages of the particular evidence acquisition mechanism introduced here. How would the proposed method perform against exhaustive image examination or standard image retrieval pipelines when all approaches are given equivalent computational resources or image-processing limits? The embedding-based retrieval experiment in Appendix B offers some relevant evidence, but a broader end-to-end comparison would provide a stronger basis for assessing the effectiveness of the proposed selective inspection strategy.
4. Separating RL policy improvements from additional visual tool usage.
The RL stage yields substantial improvements in recommendation performance, but it also leads to more frequent visual evidence requests. Can the authors determine how much of the improvement is attributable to more effective evidence selection and tool-use planning, as opposed to simply checking more images within the same number of interaction turns? In particular, would the gains persist if the compared policies were restricted to identical total numbers of visual tool calls or matched computational budgets? This analysis would help establish whether RL improves the quality of evidence acquisition decisions rather than merely increasing the amount of visual exploration.
I would be open to revisiting my evaluation, including the possibility of assigning a higher rating, if the authors can adequately address these questions with convincing explanations or additional experimental evidence.
3 comments
Weakness 2 / Question 2: Interaction Efficiency and Computational Cost
We appreciate the reviewer for highlighting this ambiguity. In particular, the phrase "High computation cost" in Figure 1 may suggest a broader claim about reducing total computational expenditure. We agree that this presentation could be clearer and will refine it in the revised version.
Our central claim has consistently focused on interaction efficiency: achieving better recommendation quality under limited tool-turn budgets. This objective runs throughout the paper, including the Introduction, Section 4.3, and the experimental analysis. Crucially, Table 1 and Figure 3 directly validate this claim. For example, under the same four-turn budget, RL improves Qualified@4 from 34.16% to 46.77% over SFT.
The increased visual parallelism in Table 11 further illustrates how RL learns to utilize limited interaction opportunities more effectively. We thank the reviewer for identifying presentation details that could obscure this distinction.
Question 3: Coarse-to-Fine Visual Evidence Acquisition
We have already provided a controlled comparison with standard embedding-based image retrieval in Appendix B.1, Table 9, using the same image selection limit and visual Reader. Our approach improves accuracy from 55.00% to 67.27% and evidence-grounded accuracy from 48.64% to 60.91%.
These results support our visual tool design. Further exhaustive comparisons of visual retrieval pipelines primarily concern the visual tool itself, rather than our central contribution: training agents to perform effective search and visual verification under constrained interaction budgets. We therefore regard such comparisons as valuable extensions rather than essential validations of our core claim.
Question 4: RL Policy Improvements Beyond Additional Visual Calls
We thank the reviewer for this valuable suggestion. To investigate whether RL improves tool-use decisions beyond simply increasing visual exploration, we conducted an additional query-level matched analysis using existing evaluation trajectories under the same four-turn budget.
For each query, we match SFT and RL trajectories with identical or similar numbers of visual calls and compare their query-averaged Qualified@4 scores:
| Model | Call Difference | Matched Pairs | SFT Avg. Calls | RL Avg. Calls | SFT Q@4 (%) | RL Q@4 (%) | Gain (pp) |
|---|---|---|---|---|---|---|---|
| 4B | 0 | 96 | 5.85 | 5.85 | 31.25 | 43.04 | +11.79 |
| 4B | ≤1 | 158 | 5.89 | 5.94 | 28.05 | 37.93 | +9.88 |
| 4B | ≤2 | 198 | 5.82 | 6.07 | 28.05 | 36.28 | +8.23 |
| 9B | 0 | 39 | 6.41 | 6.41 | 36.64 | 40.80 | +4.17 |
| 9B | ≤1 | 69 | 6.28 | 6.49 | 39.74 | 41.92 | +2.18 |
| 9B | ≤2 | 98 | 6.13 | 6.99 | 40.22 | 48.43 | +8.21 |
4B and 9B refer to Qwen3.5-4B and Qwen3.5-9B, respectively.
RL consistently outperforms SFT across all matching thresholds. Notably, with exactly matched visual call counts, RL achieves gains of +11.79 pp on 4B and +4.17 pp on 9B. These results provide additional evidence that RL improves search and verification behavior beyond simply increasing the number of visual tool calls.
We hope our clarifications and additional analysis have addressed your concerns and better demonstrated the value of our work. We would greatly appreciate your reconsideration of our score. Thank you again for your time and constructive feedback, and we wish you a wonderful day!
The rebuttal addresses some of my concerns, particularly through the additional matched analysis showing that RL brings performance gains even when the number of visual calls is controlled. This provides useful evidence of improved tool-use decisions, and I am therefore inclined to raise my score modestly.
However, my main concerns remain regarding the methodological novelty and the lack of direct measurements of computational costs. The clarification of interaction efficiency does not fully establish the overall computational benefits of the proposed approach. Therefore, despite the improvements in the rebuttal, I still lean toward rejection.
We sincerely thank the reviewer for the detailed feedback and thoughtful questions. We appreciate the opportunity to further clarify our contributions and provide additional experimental evidence addressing your concerns.
Weakness 1: Methodological Novelty and Contribution
We respectfully disagree that the use of established techniques diminishes the novelty or significance of our contribution.
The cited works address related but distinct problems, including multimodal product understanding, shopping benchmarks, multimodal tool use, budget-aware perception, and efficient web search. Our work tackles a different, underexplored problem: efficient visual evidence acquisition in real-world e-commerce search. This setting presents two interconnected challenges: searching across numerous candidate products, each containing extensive visual information, and verifying which products satisfy user requirements under limited interaction budgets. Furthermore, open-ended recommendation requires identifying a set of qualified products rather than a single predefined target.
To address these challenges, we introduce modality-aware evaluation tasks that explicitly distinguish text-answerable from visually dependent requirements, together with an integrated post-training framework for learning effective search and verification strategies. Our experiments demonstrate substantial improvements, while controlled ablations validate the contributions of the proposed training strategies.
As emphasized in the ICLR guidelines, research contributions can arise from addressing valuable problems and establishing meaningful empirical findings, without requiring every individual technique to be fundamentally new. We believe our work makes such a contribution.
Question 1: Comparisons with Specialized Methods
We believe meaningful comparisons require consistent task objectives and evaluation environments.
EcomMMMU and MMShopBench are primarily benchmarks, while MindFlow focuses on multimodal e-commerce customer support and AdaTurn investigates budget-aware active visual perception. Their task formulations, tool environments, and evaluation objectives differ substantially from our open-ended product search and recommendation setting.
Our experiments instead compare multiple foundation models and post-training strategies within the same real-world shopping environment, with controlled ablations demonstrating the effectiveness of our approach. We believe these experiments provide direct evidence for our central claims, whereas adapting the cited systems would address a different, broader comparison question.