- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces a benchmark for agricultural visual grounding constructed using existing datasets covering 6 target families (crop, fruit, wheat head, pst, plant disease, tree canopy). The benchmark contains 10k image query pairs. They evaluate bounding box grounding and mask grounding. queries can include single, multi targets or no targets at all. The authores present a pipeline for annotation, quality control, split aware query genration. and evaluation. The paper benchmarks 26 model configs, under zero shot. The multi target F1 is 0.35 and T2 success rate 0.75 and The results show the current models not good at multi target. Primary contribution is a unified grounding benchmark and systematic evaluation of existing models, not a new learning algorithm
Critical strengths
this paper addresses relatively underexplored problem in agricultural VL understanding. existing benchmarks primary emphasize regonition, VQA and reasoning, crop or weed grounding. AgroVG extends the evaluation across several target families and combines bounding box and mask grounding. I consider this a good contribution because agricultural images contain dense, similar, small and irregular targets that is not covered by traditional grounding benchmarks. This standardized eval could support research.
The evaluation covers 26 model configurations and reports results for single target, multi target and target absent as well. The results show model differences and trade off btween positive target localization and abstention. These diagnostis are more detailed than a single accuracy number.
Critical weaknesses
Limited novelty: The generalized set prediction formulation is already established in prior works and agricultural grounding also invesitaged before. Papers novelty therefore lies in dataset integration, annotation processing and expand eval coverage. The benchmark relies on template generated queries. The authors provide human paraphrase eval. But I am not convinced that the benchmark adequately represents the complexity of natural insturctions in agricultural applications.
Questions for authors
-
How do the authors ensure that template generated queries adequately measure natural language grounding rather than recognition of a limited set of category patterns ? Can the authors provide results on a bigger human written query subset, preferably covering both T1 and T2 ?
-
Appendix C.2 indicates that output normalization can change reported results. How much of the observed difference in models is caused by coordinate conventions, parsition failures, or model limitations rather than localization ability? Can the authors report the results using documented recovery procedure and distinguish explicit abstention from invalid or unparseable predictions ?
-
How were the 10 experts selected, trained and assigned to the annotation review ? Was the inter annotator agreement measured?
3 comments
Thank you for the detailed info and additional exp. The results address several of my main concerns and make the evaluation more convincing. While I still have some concerns about novelty and how model outputs are handled, I believe the benchmark can be useful to the research community. I am updating my rating to Weak Accept.
We have clarified that AgroVG’s contribution lies in query–target construction and unified evaluation across agricultural target families, building on established grounding formulations.
For output handling, the main results retain the original protocol with the confirmed InternVL3.5-8B adapter correction. Alternative coordinate policies and output recovery results are reported separately as sensitivity analyses, making their effects on model comparisons explicit.
We sincerely thank you for your time, careful reassessment, and constructive feedback. Your comments have helped us clarify AgroVG’s contribution and strengthen the evaluation.
Response to Reviewer SfZq
We thank the reviewer for the constructive feedback.
W1: Novelty and contribution.
AgroVG differs from existing agricultural detection/segmentation datasets and crop/weed-focused grounding resources in two key respects:
The revision clarifies these distinctions; family-wise results are in Appendix D.1.
Q1: Natural-language grounding beyond templates
We evaluated 500 natural-language queries per task, written and reviewed by domain experts and paired with the original queries. Images, target identities, ground truth, and inference/evaluation settings remain fixed; only wording changes, without additional tuning. The subsets cover six T1 and three T2 target families.
Scores follow the original protocol: T1 positive-query [email protected] and T2 query-level mIoU. Differences are natural − original, computed before rounding; 95% CIs use 5,000 source-stratified paired acquisition-group bootstrap replicates.
The four T1 MLLMs retain their ranking with aggregate changes within 0.015, though GPT-4o remains near the performance floor. OWLv2 and Grounded-SAM show clearer declines, indicating sensitivity to natural phrasing.
These paired evaluations extend beyond fixed templates and quantify model-dependent wording sensitivity with visual targets held constant.
Q2: Output recovery and coordinate sensitivity
We audited cached outputs for 12 auditable T1 configurations on 1,288 original test queries, without rerunning models.
Confirmed correction: the InternVL3.5-8B adapter incorrectly divided two [0,1] outputs by 1,000. The fix raises All-F1 from 0.3963 to 0.3966.
Coordinate-policy sensitivity: Qwen and InternVL3.5-38B inferred units from image size. Their unlabelled outputs leave the intended units ambiguous, so the documented 0–1000 alternative is evaluated as sensitivity rather than a verified correction for every response.
InternVL3.5-8B changes only through the confirmed fix; DeepSeek-VL2-small through syntax recovery; other rows include coordinate sensitivity. Six other audited configurations are unchanged. Under the alternative policy, Qwen3-VL-32B leads in overall F1 and multi-target F1.
Gemini’s 75 submitted correct-empty credits comprise 43 explicit empty JSONs, one verbal absence, 29 normalization fallbacks, and two blank responses. Recovery removes the 29 fallback credits; both blank credits remain. E-Acc measures pipeline-level empty-output correctness, not necessarily intentional abstention. Remaining errors mix recognition, language understanding, target selection, generation limits, and localization.
Main results retain the original protocol with only the confirmed InternVL3.5-8B correction. Alternative coordinate policies and output recovery appear as appendix sensitivity analyses.
Q3: Expert review and inter-annotator agreement
Ten reviewers spanning agriculture and computer science were selected for prior annotation experience and relevant research contributions. Before review, the authors trained them on the annotation workflow and platform.
Each candidate image was reviewed by one expert using normalized box or mask overlays to check image integrity, annotation validity, alignment, and target correspondence. No independent overlapping review or separate adjudication was conducted; inter-annotator agreement was not measured.