- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduce ACTIVELOOK, which casts visual perception as a decision-making problem. Rather than making a dedicated glance at first, it starts from the coarse glance, and learns a policy that actively decides where to look and how much to look. To achieve this, this paper suggests to conduct distillation based SFT with reinforcement learning, with some techniques like trajectory augmentation and using annotated trajectories for SFT, and reward scheduling for RL.
Critical strengths
S1. Well-motivated and compelling approach to visual perception. The paper presents a compelling framework for training VLMs to selectively acquire visual information, learning both where to look and how much visual detail to acquire.
S2. Strong empirical performance across diverse visual tasks. ACTIVELOOK achieves strong results across a broad range of image, document, and video benchmarks, demonstrating the effectiveness and generality of the proposed approach.
Critical weaknesses
W1. Insufficient explanation of important implementation details
The paper omits several low-level details that are important for understanding and reproducing the proposed method. Providing additional clarification on the following points would substantially improve the technical completeness of the paper.
- Action omission by the base model. It is unclear how the base model decides to omit an action. If this mechanism relies on the model's native tool-calling functionality, the authors should describe the implementation in more detail, including how the action arguments are defined.
- Distribution of the coarse-glance parameter . The paper does not explain how the distribution of the coarse-glance parameter is selected during training, thought it might be important to understand the evaluation results. The authors should clarify the sampling strategy and provide the rationale behind this choice.
- Selection of and . The paper should explain how these hyperparameters are chosen and why the reported values are appropriate. In addition, I believe a sensitivity analysis with respect to and would be valuable for demonstrating the robustness of the method.
W2. Insufficient justification for several design choices
Several components of the proposed training and scheduling pipeline are introduced without sufficient empirical justification.
- Effect of additional SFT techniques. The paper does not clearly isolate the contribution of the additional techniques used during SFT, such as annotation and data augmentation. Ablation studies would help quantify the individual contribution of these components.
- Necessity of the scheduling technique. I am not fully convinced that the proposed scheduling technique is necessary based on the current evidence. Although Figure 6 is intended to demonstrate its effectiveness, the curves appear to overlap substantially, and even seems to provide a better trade-off in the low-budget regime. At minimum, the authors should provide separate plots for each benchmark so that the effect of scheduling can be examined more clearly. A more explicit quantitative comparison would also help establish whether the additional scheduling mechanism provides a consistent benefit.
W3. Additional baselines are needed
The paper discusses conventional efficiency approaches such as token pruning and token merging, and argues that ACTIVELOOK is more effective because it can operate along multiple axes. However, this claim is not sufficiently supported by the current experiments.
I recommend including representative token-pruning and/or token-merging methods for each benchmarks as additional baselines under comparable computational budgets. Such comparisons would more directly demonstrate the advantages of ACTIVELOOK over existing efficiency-oriented approaches and substantiate the paper's claim regarding its multi-axis adaptation capability.
Questions for authors
Q1. What is the role of setting in the experimental setup?
Q2. Could the authors report the Figure 6 ablation results separately for each benchmark? This would make it easier to understand how the scheduling technique affects performance across different benchmarks and whether its impact is consistent.
Q3. Could the authors further clarify the fundamental differences between ACTIVELOOK and existing tool-calling approaches that incorporate a cost penalty? Conceptually, ACTIVELOOK appears related to training a model to call a specific vision tool while penalizing tool usage, with a bag of tricks. It would therefore be helpful for the authors to articulate more clearly what is fundamentally novel about ACTIVELOOK beyond this formulation, and how it differs from prior cost-aware tool-calling methods.