- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper studies the robustness of VLMs under extreme visual domains. The authors introduce CrossVLM-Bench covering remote sensing, industrial inspection, agriculture and underwater imagery. The paper also shows several degradation of representative VLMs on these domains. To address the gaps, the authors propose CD-GRPO, a two-stage posting training framework consisting SFT and CD-GRPO for adapting VLMs to extreme visual domains. Experiments across several VLMs and domains show that CD-GRPO consistently outperforms both original models and SFT.
Critical strengths
-
The authors study the generalization ability of VLMs in real-world or extreme visual domains, which is a worthwhile direction. The paper also addresses some meaningful problems and shows detailed and comprehensive experiments.
-
Althgouh the novelty of CD-GRPO is limited as it mainly involves task-specific reward design and an additional structural reward, the experiments are comprehensive and the ablation studies clearly demonstrate the effectiveness of CD-GRPO.
Critical weaknesses
The work constructs the CrossVLM-Bench to evaluate the capabilities of VLMs in extreme domains and proposes CD-GRPO to address the resulting performance gap. Although many models show degradation on CrossVLM-Bench, I would expect to see a deeper analysis of why these models fail and how the benchmark can provide guidance for the development of next-generation models, rather than introducing a new dataset and using its training data to do post-training.
Questions for authors
-
The authors mention that CrossVLM-Bench is supposed to figure out whether domian shift failures share common causes or whether they can be recovered through a unified framework. It's good to see CD-GRPO uses the same reward decomposition to solve the failures across extreme domains, but it does not show that these failures arise from a shared cause. For example, whether the failures in remote sensing and underwater imagery stem from the same type of representation failure, and whether the degradation in detection and segmentation is caused by a shared bottleneck?
-
Table 2 (b) shows the effect of the structural reward in CD-GRPO, but the authors also use samples whose SFT accuracy lies within intermediate range around 0.5. Can authors provide the effect of the sample-filter strategy?
-
If the authors can clarify my concerns above, I will raise my rating.
4 comments
Response to Reviewer KuDm (Part-2)
Q2. Do the domains and tasks share one failure cause?
Response:
No, our results do not establish one representation-level cause across domains. A shared reward decomposition demonstrates a useful adaptation principle, not a shared origin of the errors. We will correct the wording that conflates these claims.
(1) Our additional data analysis shows that the domains present different challenges:
| Domain | Small objects (%) | Occluded objects (%) |
|---|---|---|
| Remote sensing | 68.4 | 14.8 |
| Industrial inspection | 41.2 | 8.6 |
| Agriculture | 28.3 | 31.7 |
| Underwater | 35.8 | 38.9 |
For example, remote sensing contains a much larger proportion of small objects, while underwater imagery has greater occlusion and different imaging degradations. These observations motivate different visual improvements; they do not support treating the domains as interchangeable.
(2) Detection and segmentation share spatial localization in our pipeline, but segmentation additionally depends on mask conversion. Appendix E.3, Table 19 measures these stages separately:
| Method | Box IoU before SAM | Point similarity before SAM | Mask mIoU after SAM |
|---|---|---|---|
| SFT | 36.3 | 56.3 | 27.7 |
| CD-GRPO (Ours) | 38.7 | 88.5 | 48.7 |
Our additional reference-prompt experiment further tests what remains recoverable with the same SAM:
| Prompt source | DIOR | NEU-DET | PlantSeg | UODD |
|---|---|---|---|---|
| SFT predictions | 39.6 | 55.6 | 31.2 | 54.3 |
| CD-GRPO predictions | 48.2 | 60.0 | 40.5 | 75.2 |
| Reference point | 54.0 | 67.7 | 50.7 | 78.5 |
| Reference box | 61.2 | 74.2 | 57.4 | 83.2 |
| Reference box + point | 65.4 | 78.1 | 61.6 | 86.1 |
Each entry averages the four segmentation tasks within a domain. Accurate reference prompts substantially improve masks, identifying upstream localization as one remaining limitation. Performance is still imperfect with reference prompts, so mask generation also remains a limitation. This is more specific than attributing detection and segmentation degradation to one common bottleneck.
(3) We will narrow the conclusion accordingly. The evidence supports recurring prediction failures and a reward framework that helps across domains. It does not establish identical internal causes. The benchmark provides a way to investigate these dependencies; we should not present that research opportunity as an analysis already completed.
Response to Reviewer KuDm (Part-3)
Q3. How much does sample filtering contribute?
Response:
(1) We select examples with SFT scores around 0.5 to prioritize partially solved cases. Near-perfect examples leave limited room for improvement, while near-zero examples may be difficult to learn from within a small rollout budget. The midpoint is a practical heuristic, not a theoretically optimal threshold; the scores can reflect partial geometric correctness rather than success probability.
(2) Inspired by your comment, we compared this strategy with an equal-size random subset. Both groups used the same SFT checkpoint, annotation-quality checks, rewards, hyperparameters, and update budget.
| RL sample selection | DIOR | NEU-DET | PlantSeg | UODD |
|---|---|---|---|---|
| Equal-size random subset | 36.9 | 51.4 | 29.8 | 56.7 |
| Intermediate-difficulty subset | 41.6 | 56.2 | 33.7 | 62.3 |
| Gain from filtering | +4.7 | +4.8 | +3.9 | +5.6 |
Each value is the nine-task mean within a domain. Filtering contributes materially to the final performance. We will report it as a separate part of the training recipe, rather than attribute the full improvement to reward design.
Dear Reviewer, as the discussion deadline approaches, we would appreciate hearing whether you have any remaining questions or concerns about our paper or rebuttal. We would be happy to provide further clarification or additional information. Thank you again for your time and thoughtful feedback!
Response to Reviewer KuDm (Part-1)
Q1. What do the failures teach us?
Response:
We agree that introducing a dataset and improving its scores is not sufficient. Our experiments reveal two limitations that should guide model development: incomplete visual coverage and insufficient supervision of structured predictions.
(1) The errors persist beyond learning a valid response format. Appendix E.4, Table 20 audits the same 900 predictions per model:
After SFT, missed small/dense objects and localization errors remain more frequent than malformed outputs. This suggests that reliable formatting is only one part of the problem: models must also discover the required targets and ground them precisely. These remain substantial errors even after CD-GRPO.
(2) The reward ablation tests whether supervising complete predictions helps. Table 2(b) reports:
The gains indicate that accuracy rewards leave useful constraints unexpressed. Correct individual boxes do not guarantee sufficient coverage, nonredundant instances, or consistent counts. This supports explicitly supervising those relationships alongside improving visual perception.
For future models, the implication is to distinguish target discovery, localization, and output consistency rather than treat them as one score. CrossVLM-Bench supports this analysis through complementary annotations on the same images. We will foreground this diagnostic role and connect the existing analyses more clearly to these development priorities.