- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
CoEvolve separates visual grounding into state construction and coordinate editing. Region-Evolution Reinforcement (RER) uses GRPO and terminal box annotations to generate reasoning steps with progressively refined candidate regions. Bidirectional Denoising Refiner (BDR) then updates coordinate tokens in place while retaining the reasoning text. The stages use separate adapters on a frozen Qwen3.5-9B backbone. Experiments cover RefCOCO-family grounding, controlled box correction, and remote-sensing adaptation.
Critical strengths
-
Explicit, editable spatial states. Associating reasoning steps with candidate regions makes intermediate localization decisions measurable and provides a useful interface for correction.
-
Systematic rewards without intermediate annotations. Equation 6 combines format, spatial-progress, and terminal-localization rewards. The progressive reward derives geometric feedback from final annotations without requiring labeled reasoning chains or region trajectories.
-
A reasonable coordinate-editing mechanism. Bidirectional same-position reconstruction enables synchronous coordinate updates using the complete response context while preserving its text and structure.
-
Strong final localization performance. Across five test splits, CoEvolve improves mIoU from 82.16% to 85.72% and [email protected] from 60.64% to 68.64% over native Qwen3.5-9B, supporting its effectiveness for precise localization.
Critical weaknesses
-
Semantic–spatial consistency remains unclear. RER’s geometric rewards and BDR’s edit preferences refer to the final target annotation rather than each step’s textual meaning. In Figure 8, BDR replaces a scene-level region with a localized region while retaining the scene description. Thus, improved final coordinates do not necessarily imply faithful intermediate states. Appendix D acknowledges this limitation, but its implications for interpreting the trajectory are insufficiently examined.
-
The contribution of RER trajectories is not isolated. In Table 3, RER starts below native output (80.70% versus 82.16% mIoU) but finishes above it after source-matched refinement (85.72% versus 84.57%). This supports the complete pipeline, but the editors differ in editable positions and supervision (Appendix A.6). A controlled comparison under matched training and inference budgets is needed among four settings: (i) refining only a terminal bounding box, without intermediate reasoning or regions; (ii) refining a terminal box accompanied by brief reasoning, without a multi-step region trajectory; (iii) refining the full RER reasoning-and-region trajectory; and (iv) directly predicting coordinates through bidirectional reconstruction or iterative denoising, without an autoregressive state-construction stage. This would clarify the contributions of explicit reasoning, intermediate regions, and the construct-to-edit design.
-
Key design choices lack targeted ablations. Zero-Anchor BDR removes both the localization hypothesis and associated preservation/preference objectives, establishing only their joint value. The individual contributions of the progressive reward, bidirectional attention, quality-gated targets, and auxiliary losses remain unclear.
-
Controlled correction does not isolate editing from re-localization. In Section 4.4, native Qwen3.5-9B reaches 83.85% output mIoU versus BDR’s 86.19%, but only BDR receives task-specific training. A model could also achieve a high repair rate by ignoring the anchor and locating the target again. The experiment therefore does not distinguish the benefits of state-dependent editing from additional localization training.
-
The motivation and positioning could be clearer. The first introduction paragraph moves from prior grounding and reasoning methods to the need for editable intermediate states, without sufficiently explaining why this is an important limitation or how addressing it benefits localization. Meanwhile, although related work is discussed briefly in the introduction and preliminaries, a more structured review would help contextualize the approach and clarify its differences from existing progressive grounding, grounded reasoning, and diffusion-based localization methods.
Questions for authors
-
Are edited historical regions expected to remain consistent with their original textual roles? Please clarify Figure 8 and provide more visualization cases.
-
How much does BDR rely on the reasoning text and input boxes? Text-removal/shuffling and anchor-replacement controls would help distinguish state-dependent editing from re-localization.
-
Does the full RER trajectory outperform a terminal-box-only state under comparable editor supervision and editing budgets? What accounts for RER’s lower initial accuracy than native output?
-
How does Equation 8 compare with always using ground-truth targets? Targeted ablations of the progressive reward, regression, edit preference, and bidirectional attention would clarify the essential components.
4 comments
Thank you for asking how the edits relate to the original reasoning and what the trajectory and input anchors contribute to refinement.
-
Historical boxes (Weakness 1 / Question 1). We added a second example to Appendix C.3 (pp. 28–29). In the first example (Figure 8), the first Step box tightens while its text still describes the full scene. In Figure 9 and Table 16, the first historical box moves toward the person and the Answer stays at IoU 0.936 in both passes. Appendix D (p. 36) explains how edits can change the spatial scope of a Step box. Answer-only decoding leaves earlier text and boxes unchanged.
-
Trajectory context (Weakness 2 / Question 3). We compare three Answer-only editors with the same starting Answer and dedicated Answer losses in Section 4.5 and Table 4 (p. 9). The full trajectory gains 1.29 [email protected] points over the terminal box alone and 1.09 points over a brief explanation with the box. Table 20 (p. 34) gives the gains on all five splits for the full trajectory relative to the terminal box. Table 19 (p. 33) also reports direct prediction from an all-zero box and the full-state editor that supervises and updates earlier boxes.
-
RER adaptation and progressive reward (Weakness 2 / Question 3; Weakness 3 / Question 4). Under the same structured interface, RER raises terminal mIoU from 63.48% to 80.70% (Section 4.5, p. 8). After refinement by their respective BDR editors, the RER and native outputs reach 85.72% and 84.57% mIoU (Tables 3 and 12, pp. 9 and 26). Removing the progressive reward lowers terminal mIoU by 2.05 points and [email protected] by 2.02 points. Table 13 (p. 26) also reports the IoU change from the first Step to the Answer.
-
Targets, losses, and attention (Weakness 3 / Question 4). Section 4.5 (p. 9) reports the causal-mask, preservation, and repair results. Table 21 (p. 34) reports 92.06% [email protected] for Always-GT and gives the results of the loss ablations. For the General and Format CE ablation, we use the same decoding procedure in both conditions. Section 3.3 (p. 5) states the priority of the output branch after Eq. (8).
-
Text and coordinate inputs (Question 2). Appendix C.8 and Table 22 (p. 35) compare removing or shuffling the reasoning text and replacing coordinate boxes. Replacing all boxes lowers [email protected] by 3.81 points; removing the text alone lowers it by 0.44 points. With the boxes replaced, retaining the text recovers 0.44 points in [email protected].
-
Controlled correction with and without the input anchor (Weakness 4). We compare perturbed and zeroed anchors with the same BDR checkpoint, images, expressions, and one-pass budget (Section 4.4, p. 8; Table 23, p. 35). Zeroing the anchors lowers mIoU from 86.19% to 83.26% and [email protected] by 6.80 points.
-
Relation to prior work (Weakness 5). The Introduction (p. 1) places RER alongside work on explicit regions in reasoning and sequential region search. CoEvolve passes those regions to BDR as starting coordinates and as a reference for evaluating edits. Section 2.2 (p. 3) discusses masked diffusion for grounding, including Lavida-O, and describes how BDR edits coordinates in an existing response.
Thanks for the authors' response. I think the authors have addressed my concerns, and I am willing to raise my score to weak accept. However, I am not very familiar with this research area, so I keep my confidence score at 2.
Dear Reviewer eiXH,
Thank you very much for taking the time to review our response and for raising your score. Your constructive questions regarding the matched trajectory controls, input interventions, and historical box behaviors were instrumental in helping us strengthen the experimental analysis and clarity of the revised manuscript. We truly appreciate your thoughtful engagement and feedback throughout the review process.
Best regards, The Authors
Thank you for your feedback. Unless noted, all tests use the 30,969 examples across the 5 test splits (invalid IoU=0, 2 BDR passes).
Critical Weaknesses
58.80%-mIoU anchor in BDR drops mIoU/[email protected] by -2.93/-6.80 pp (below Native Qwen3.5), proving anchor editing:Questions for Authors
Question 1: Historical Roles & Fig. 8
Last Step/Answer use CE+L1, General/Format CE keep text, and Preference pulls Step boxes toward b∗ while keeping accurate answers fixed (Fig. 8 broccoli vs. person in [0,1000]):
For strict text–box alignment, Answer-only editing (Q3.1) keeps Step boxes fixed (85.35 mIoU / 67.27 [email protected]).
Question 2: Text & Box Reliance
Intervening on inputs before Pass 1 (
Replaceuses a valid unrelated box):Replace AnswerbeatsReplace all boxesby +0.92 [email protected], showing BDR reads intact Step boxes (1.1%validity loss from Step K/Answer conflict lowers mIoU).No text:-0.44[email protected]), but under corrupted boxes, text recovers +0.59 mIoU / +0.44 [email protected].Question 3: State Comparison & Initial Accuracy
3.1 Matched Four-Way State Comparison
Settings (i)–(iii) share the backbone, RER inputs, and Answer-only losses (text/Steps read-only); (iv) edits a zero box (
83.79/63.40, matching Fig. 4's83.49/62.16).Read-only trajectories (iii) beat (i)–(ii) (+1.29/+1.09 [email protected], +398 cases at IoU≥0.9 over (i)); all-box editing (iii-b) adds +1.37 [email protected].
3.2 Initial Accuracy: RER vs. Native
Multi-step output is OOD for JSON-pretrained Qwen3.5 (
63.48/24.28unadapted, Table 3). RER reaches80.70/59.81, and its spatial trajectory doubles BDR's gain (+5.02/+8.83 to 85.72/68.64, vs. +2.41/+5.68 to 84.57/66.32 for Native+BDR).Question 4: Targets, Losses, Attention & Rewards
4.1 Target Gating, Attention & Losses
Eq. (8) adds +0.76 mIoU / +0.93 [email protected] / +1.85 [email protected] over Always-GT and lifts IoU≥0.9 retention (18,524 inputs) from 89.76% to 93.61%. Right context, L1, Preference, and Entropy add +1.06 / +0.90 / +1.65 / +0.52 [email protected] (Preference lifts IoU<0.7 [email protected]: 49.25%$\rightarrow$51.50%); no General/Format CE drops validity to 5.39%.
4.2 Progressive Reward (Rprog)
Rprog prevents static boxes and adds +2.05 mIoU / +2.02 [email protected]. We will add all controls and clarify Fig. 8.