- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
- Method. The paper proposes V-Gym, a framework that improves a fixed-parameter multimodal agent for tool-assisted visual reasoning by co-evolving a hierarchical skill bank and a practice data bank from execution trajectories.
- Two components. (1) Skill evolution reflects on grouped trajectories to propose skill patches, which are accepted only if they improve performance on a validation set. (2) Data evolution attributes skill-update gains to practice instances, selects seeds with UCB-style scoring, and generates targeted new instances that preserve core task challenges.
- Results. Experiments on TIRBench, MMSearch-Plus, and MMBrowseComp show gains over existing skill-evolution methods, and the evolved skills transfer to unseen domains and weaker models.
Critical strengths
- Novel skill-data co-evolution. Using the same execution trajectories to both refine skills and generate targeted practice data is a well-motivated idea, and the internal comparisons show that targeted generation is more effective than generic synthesis under the same budget.
- Transferable skills. The evolved skills improve performance on unseen benchmarks and on weaker models without further updates, suggesting that the learned procedural knowledge is reusable.
Critical weaknesses
-
Insufficient ablations. (1) The utility-based UCB selection is only compared with random selection. A simple heuristic, such as prioritizing failed or low-accuracy instances, would be a stronger control for justifying the utility attribution. (2) The self-check in data generation is not ablated; it remains unclear how much it contributes compared with admitting generated data directly. (3) The validation gate uses a large in-distribution labeled validation set (e.g., 630 instances vs. 195 training instances on TIRBench), whereas the w/o Validation Gate variant practices only on the training set. Adding the validation data to practice without the gate would help separate the effect of the gating mechanism from that of the additional labeled data.
-
Limited baseline comparison. The skill-evolution baselines (XSkill, Ace-Skill, and SkillOPT) are evaluated only with GPT-5.5. With Gemini-3.5-Flash and Qwen-3.7-Flash, V-Gym is compared only with Vanilla Tools, which is insufficient to support the claim that it consistently outperforms baselines across backbones.
-
Missing cost analysis. Only the training rollout budget (i.e., during skill evolution) is matched across methods, while the additional costs of V-Gym, including validation reruns, reflection, and data generation with self-checks, are not reported. (1) Time cost: the time cost of both training and test-time inference should be reported and compared with the baselines. (2) Computational cost: the number of API calls and token consumption in both phases are not reported.
Questions for authors
See weakness 1-3.