- Rating
- 1
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 1
- Soundness
- 2
Read review
Paper summary
This paper proposes AgriSEED, a learning-to-teach framework that generates the most effective question-answer pairs by leveraging real-world learning gains and visual quality assessments to increase the VLM's learning efficiency in agriculture.
Critical strengths
- The authors propose a new framework to solve the adaptation challenge for agricultural tasks.
- The models trained via AgriSEED outperformed the baseline models, demonstrating the effectiveness of the framework.
Critical weaknesses
- I think the novelty of AgriSEED is limited. This is because existing research has already explored approaches to fine-tuning teacher models or data generators based on the learning outcomes of student models (i.e., learning-to-teach framework). While the authors highlight the framework as a key contribution (I believe), it appears that they have not fully articulated a clear distinction from existing methodologies. I think the main contribution of this study is not in proposing a new learning framework, but in applying the existing learning-to-teach approach to the agricultural domain and experimentally verifying its effectiveness.
- To emphasize the authors' proposed framework as a core contribution, a more depth and comprehensive analysis of the roles and effects of each component is required. To gain a deeper understanding, it's essential to systematically investigate how the cross-image probe specifically enhances learning performance, how its performance varies depending on the question-answer generation strategy, and how the design of the reward function impacts both the learning process and final performance. The current paper primarily relies on an ablation study, which limits the analysis to only partially explaining the effectiveness of the proposed framework and the validity of its design choices.
Questions for authors
- The scores used in equation (1) is unclear. The authors need an explanation of whether the score was empirically determined or etc. This also applies to the scores used in other formulas. To validate the proposed scoring design, it's essential to conduct additional experiments on how the model's performance changes when different scoring methods or weights are applied.
- This study utilizes the AgroCoT dataset for both training and evaluation. Even with only 10% of the total data used in the learning process, evaluating the proposed method solely within the same dataset is insufficient to fully validate its generalization performance. To validate the proposed framework, I recommend further evaluation on external datasets like AgroBench.
- Does the Qwen3.5-27B judge align with human judgement?
- In Table 3, 10% target => 20% target..? But, the authors mention they used only 10% of AgroCoT. What happened?
- As I mentioned above, there's a typo in the sampling count (I think).
2 comments
Thanks for the author's response. However, after review the response, I can't find any evidence to increase the score. Especially, my concern regarding Weakness 2 has not been fully addressed (other concerns also remain). I highly recommend showing more analysis of why the proposed framework is a core contribution. Therefore, I maintain my score.
Teaching signal and probe design. AgriSEED trains the questioner using the effect of a single generated QA pair on the solver. We apply one temporary LoRA step to the solver, compare reference-answer log-likelihood on the same probes before and after the update, and reset the adapter after each candidate. Same-image augmentations measure how the gain carries across views, while cross-image probes measure how it carries to other visual content. At a fixed probe count, the probe-composition analysis can compare each probe type and their combination.
Reward scoring and sensitivity. We will specify the rubric and score ranges in Eq. (1), the reward weights and thresholds, and which settings were fixed or tuned. In the matched comparison on held-out AgroCoT images, a Qwen2.5-VL-7B solver gains +4.2 SS and +3.2 CQ over the base with the original difficulty reward and +5.4 SS and +4.2 CQ with learning gain. This is about 30% more gain relative to the original reward's improvement over the base.
Image-level generalization. We use the 10%-target split for adaptation and reserve the remaining 90% of AgroCoT for held-out testing. We group all questions and augmented views by original image and exclude the held-out images from all adaptation stages, including probe feedback. AgriSEED takes only images from the AgroCoT adaptation split. It generates the training QA pairs and uses feedback from a frozen visual judge. AgriSEED's held-out scores are close to the full-set scores reported in the paper, supporting generalization to unseen images within AgroCoT.
External evaluation on AgroBench. We use the full AgroBench benchmark with five-option exact matching and report macro-averaged accuracy across its seven tasks. The AgriSEED score is 71.3%. The published full-set scores are 70.66% for QwenVLM-72B, 72.24% for Gemini 1.5 Pro, and 73.45% for GPT-4o (Shinoda et al., ICCV 2025, Table 2).
Sampling targets. The 10% and 20% sampling targets came from different experimental versions. All results in the paper use the 10% target. We will correct the sampling entries in Table 3 and Appendix A in the revision to reflect that version.
Human judge agreement. We randomly sampled 10% of the audit pool, which consisted of QA pairs generated during training. Human reviewers and the frozen Qwen3.5-27B judge agreed on 92% of the reviewed pairs.