- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Marvo, a self-evolution framework that trains visual agents using question–answer pairs generated from unlabeled images. A challenger and a solver are alternately optimized, with the challenger rewarded by the solver’s accuracy difference with and without tools to encourage questions that benefit from tool use. Experiments across two model backbones and seven benchmarks show improvements over the initial checkpoints and the R-Zero baseline, while extended training reveals limitations from declining generated-data quality.
Critical strengths
This work presents a well-motivated reward for tool-useful task generation. The margin reward directly measures the solver’s benefit from tool access, providing a more targeted signal than uncertainty alone. Using this signal to train the challenger offers a simple and interpretable way to align generated tasks with the goal of improving tool use.
Critical weaknesses
-
Limited methodological contribution and incomplete isolation of the reward’s value. The challenger–solver framework largely follows R-Zero, making the margin reward the central methodological contribution. Although comparisons with R-Zero are useful, the paper does not establish whether the full margin formulation offers substantial benefits over simpler objectives, such as rewarding tool-assisted success or combining uncertainty with tool necessity. This leaves the significance of the specific reward design insufficiently established.
-
Generated-data reliability limits sustained self-evolution. Tables 6–7 show that additional evolution rounds lead to performance degradation alongside declining question quality: the external judge’s estimated accuracy of generated questions falls from 68.5% in Round 1 to 43.6% in Round 8. Although reporting this limitation is valuable, the method lacks an effective mechanism to prevent unreliable self-generated supervision, limiting the evidence for scalable, sustained improvement.
Questions for authors
- Contribution of the tool-free baseline. While Table 1 compares against uncertainty-based rewards and Table 4 changes the reference harness, can the authors isolate the contribution of subtracting tool-free performance by comparing against tool-assisted success alone, with filtering and training budgets held fixed?
- Scalability of self-evolution. The later-round degradation reported in Appendix F.2 raises concerns about whether Marvo can sustain improvement as evolution continues. Is this primarily limited by the models’ initial capabilities, or by errors accumulating and being reinforced through self-training? Can the authors also investigate whether stronger models mitigate this degradation, or whether stricter filtering with independent checks of question and answer validity enables more sustained improvement?
- Consistency of solver rewards. Section 2.4 describes the solver reward as binary answer correctness, while Table 5 also lists a tool-use bonus of +0.2 and a format penalty of −0.5. Can the authors clarify how these components are combined, which experiments use them, and whether Marvo and R-Zero use identical solver rewards? This clarification would help establish whether the comparison isolates the effect of the challenger’s margin reward.