acceptodds
Under review as a conference paper at ICLR 2027

CounterGround: Diagnosing and Repairing Counterfactual Failures in Visual Grounding

Abstract

Visual grounding is typically evaluated by box overlap, which conflates association with the intended instance and localization precision. We introduce CounterGround, a paired counterfactual benchmark that fixes the image, object category, and candidate set while a spatial relation edit switches the annotated target instance. Across 5,635 pairs from remote-sensing and natural-image domains, recent multimodal large language models (MLLMs) exhibit a consistent behavioral separation: predictions can respond consistently to the relation edit and remain associated with the intended instances while failing the paired intersection-over-union (IoU) criterion. This separation persists as Qwen3.5 scales from 2B to 9B, despite higher grounding accuracy. Controlled Diagnosis on Qwen3.5-2B and Qwen3-VL-2B-Instruct shows that selecting among explicit ground-truth candidates is substantially easier than free-form box localization, while fixed-center extent prediction and a ground-truth-size oracle leave substantial residual error. Guided by this diagnosis, joint geometric refinement raises Pair Accuracy on CounterGround-DIOR validation from 48.65% to 56.06%, and adding local visual-language features raises it to 61.33%. The repair formulation also improves the reported point estimates on held-out CounterGround-DIOR with both backbones and on CounterGround-COCO, RefCOCO, and VRSBench under their specified training protocols. CounterGround links controlled target-switch evaluation to behavioral diagnosis and targeted repair.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Visual Grounding, Counterfactual Evaluation, Multimodal Large Language Models, Geometric Refinement

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_pbhxOpenReview ↗
Rating
3(+1)
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CounterGround, a paired counterfactual benchmark for visual grounding. Each pair shares the same image, object category, and candidate set, while a minimal spatial relation edit changes the referred target instance. The benchmark contains 5,635 pairs from DIOR and COCO. The authors evaluate several MLLMs and find that models can respond consistently to relation changes and associate their predictions with the intended instances while still failing the bounding-box IoU criterion. They further conduct controlled experiments to examine the differences between target selection, center localization, and box extent prediction. Based on these observations, they propose a repair pipeline that combines joint geometric refinement with local visual-language features. Experiments show improvements in paired grounding accuracy on CounterGround and additional gains on RefCOCO and VRSBench under dataset-specific training protocols.

Critical strengths
  1. An interesting perspective on visual grounding evaluation. I like the idea of evaluating grounding through paired target switches rather than treating each image-query pair independently. The controlled setup makes it possible to examine how models respond when a small change in the referring expression requires a different target instance. The distinction between directional consistency, instance association, and box accuracy is useful for understanding different types of grounding failures. I also appreciate that the authors evaluate multiple MLLMs on both remote-sensing and natural images.

  2. The paper connects its analysis to a practical improvement. Instead of stopping at benchmark construction, the authors investigate possible sources of localization errors and propose a corresponding refinement method. The experiments are reasonably comprehensive, including component ablations, held-out evaluation, two MLLM backbones, and additional grounding benchmarks. The improvement in Pair Accuracy from 48.65% to 61.33% on CounterGround-DIOR validation is encouraging.

Critical weaknesses
  1. The current metrics may overestimate the model's ability to identify the correct instance. My main concern is how the paper distinguishes correct instance association from coarse spatial responses. Direction Accuracy only checks whether the predicted center moves in the expected direction, while Instance Association Accuracy assigns each predicted center to its nearest same-class GT candidate. These metrics are useful, but they may not be strong enough to establish that the model has actually identified the intended object. For example, with a leftmost/rightmost pair, a model could place its predictions toward opposite sides of the image and potentially satisfy both criteria without accurately locating either object. This matters because the separation between instance association and box localization is the central finding of the paper. I would find the conclusion more convincing if the authors also reported stricter measures of instance localization, such as whether the predicted center falls inside the target box or its distance to the GT center normalized by object size. The benchmark is also restricted to two types of extremal spatial relations, which limits how broadly the findings can be interpreted.

  2. The diagnosis and repair experiments do not fully isolate the sources of improvement. The GT-candidate selection experiment is informative, but it provides the model with exact candidate boxes, removing much of the difficulty involved in object detection and box prediction. The authors acknowledge this limitation, but the large gap between GT-candidate selection and free-form grounding does not by itself establish which component of ordinary grounding is responsible for the failures. I also have some reservations about the repair experiments. The proposed pipeline introduces additional trainable geometry and local-feature refinement modules, while the main baseline uses the original MLLM predictions. Although the paper includes center-only and direct-crop controls, a comparison with a conventional learned box refiner trained using the same data and local features would make the contribution clearer. One interesting detail in Table 3 is that joint geometric refinement improves Pair Accuracy from 48.65% to 56.06%, while mean IoU slightly decreases from 55.49 to 55.30. This suggests that the improvement in threshold-based accuracy does not necessarily translate into better overall box overlap. I would like to see further analysis of this behavior.

Questions for authors

Q1. Could the authors evaluate instance association using a stricter criterion than nearest-GT assignment? For example, how often does the predicted center actually fall inside the intended target box, and how does the center localization error vary with object size? I think this would help establish whether the reported association/localization gap reflects correct target identification or simply coarse spatial agreement.

Q2. How much of the observed behavior can be explained by simple spatial heuristics? Since the current benchmark only includes leftmost/rightmost and topmost/bottommost switches, it would be useful to examine performance across different numbers and spatial densities of same-class objects. The GroundingDINO + resolver baseline is helpful, but a more detailed analysis of candidate detection quality and a simple spatial-ranking strategy could further clarify the difficulty of the benchmark.

Q3. Could the authors compare the repair pipeline with a standard learned bounding-box refinement baseline under matched training data and feature inputs? Also, why does joint geometry refinement improve Pair Accuracy while slightly reducing mean IoU? Reporting center errors, box-size errors, and confidence intervals for the held-out repair results would help clarify where the gains come from.

These points, particularly the validity of the instance association measure and the controlled evaluation of the repair module, would influence my overall assessment.

4 comments

Thank you for the detailed and constructive feedback.

To Q1:

We agree that nearest-GT assignment is permissive and should not be interpreted as evidence of semantic instance identification. We therefore added a stricter Center-In-Target (CIT) criterion, requiring the predicted center to fall inside the intended target box, together with target-size-normalized center-error analysis. We will also rename the original metric as nearest-candidate geometric association to emphasize that it measures coarse geometric support rather than semantic recognition.

We report pair-level rates below, requiring each criterion to hold for both members. Headline pair-level metrics use all benchmark pairs, with invalid predictions counted as failures.

Dataset / modelNearest assoc.CITPair [email protected]
DIOR / Qwen3.5-2B73.53%49.71%37.37%
DIOR / Qwen3-VL-2B69.59%49.17%36.27%
COCO / Qwen3.5-2B59.69%52.74%44.52%
COCO / Qwen3-VL-2B62.52%54.01%46.48%

CIT remains above Pair Accuracy by 12.33 pp [11.38, 13.29] and 12.90 pp [11.92, 13.90] on DIOR, and by 8.22 pp [6.56, 9.88] and 7.53 pp [5.97, 9.10] on COCO (10,000 paired bootstrap resamples over complete pairs). Object scale also matters: for DIOR/Qwen3.5-2B, sample-level CIT rises from 46.15% on small targets to 83.12% on large targets, while normalized center error decreases from 5.14 to 0.32.

We will therefore narrow the original interpretation: nearest-candidate association is not evidence of correct instance recognition. However, the stricter CIT analysis still shows that predicted centers fall inside the intended target more often than the model produces an accurate box, so the gap cannot be explained solely by coarse left/right or top/bottom placement.

To Q2:

We further analyzed the GroundingDINO + deterministic extremal spatial resolver baseline. On COCO, it achieves 42.95% Pair Accuracy, versus 44.52% and 46.48% for Qwen3.5-2B and Qwen3-VL-2B; the differences are −1.57 pp [−4.60, +1.47] and −3.52 pp [−6.56, −0.39]. Thus, detection followed by deterministic extremal ranking constitutes a strong control on COCO and reaches performance close to the evaluated MLLMs. On DIOR, however, the same pipeline reaches only 15.65%, versus 37.37% and 36.27%; the gaps are −21.72 pp [−23.09, −20.35] and −20.62 pp [−22.00, −19.23].

Detection quality helps explain this cross-domain difference: at IoU ≥ 0.5, intended-target recall is 68.30% on COCO but 30.29% on DIOR, and detection failures account for 27.64% versus 62.50% of members. We also stratified MLLM performance by same-class candidate count and spatial density. On COCO/Qwen3.5-2B, Pair Accuracy falls from 49.39% with two candidates to 27.55% with four or more, and from 50.44% in sparse scenes to 35.48% in dense scenes; Qwen3-VL shows the same trend. DIOR does not show the same monotonic pattern, so we treat these results as descriptive within-dataset analyses rather than evidence of a causal effect.

We will therefore clarify that CounterGround evaluates controlled extremal target switching, not general spatial reasoning. The detector-plus-extremal-resolver baseline is competitive on COCO when detection is reliable, whereas it remains substantially below the evaluated MLLMs on DIOR, where candidate detection itself is a major bottleneck.

To Q3:

We added a conventional proposal-relative residual MLP under closely matched conditions. It receives the same 578 deployable inputs as our joint geometry + local evidence refiner, uses the same DIOR training split, validation roster, input preprocessing, box-level supervision, optimization budget, three seeds, and nearly identical trainable capacity (90,301 vs. 89,995 parameters). The validation split is not used for training, and all compared refiners use the same fixed-final checkpoint policy.

MethodPair [email protected]mIoU (%)
Free-form grounding49.09%55.45
Joint geometry refinement57.15%56.07
Matched residual refiner59.65%57.03
Joint geometry + local evidence61.17%58.44

The conventional residual refiner therefore explains most of the repair gain. Joint geometry + local evidence provides an additional +1.52 pp Pair Accuracy over this matched baseline, with a paired-bootstrap 95% CI of [+0.75, +2.28] using 10,000 complete-pair resamples. Its mean-IoU improvement over the matched residual baseline is also positive (+0.0140, 95% CI [+0.0122, +0.0159]). We accordingly narrow the claim: repairability is not unique to our architecture; joint geometry + local evidence provides a smaller but statistically supported incremental improvement. We will also disclose that two architecture-specific regularizers in its structured decoder have no direct counterpart in the conventional MLP, so the complete objectives are not literally identical.

The fresh rerun does not reproduce the slight mean-IoU decrease observed in the original table. Joint geometry refinement increases Pair Accuracy from 49.09% to 57.15% (+8.06 pp, 95% CI [+6.05, +10.14]), while mean IoU changes from 0.5545 to 0.5607 (+0.0062, 95% CI [−0.0043, +0.0170]). Thus, the previous slight decrease is not a stable effect. A pair-level transition analysis further shows that, averaged over three seeds, 17.42% of pairs change from failure to success, whereas 9.36% change from success to failure. This confirms that Pair Accuracy primarily reflects net crossings of the IoU = 0.5 threshold and therefore need not change proportionally with mean IoU.

We further analyzed localization geometry as requested. The target-size-normalized center error decreases from 0.962 for free-form prediction to approximately 0.558 with joint geometry refinement, 0.562 with the matched residual refiner, and 0.555 with joint geometry + local evidence. Absolute log-area error, used as a box-size error measure, decreases from 0.663 to 0.451, 0.401, and 0.397, respectively. These results indicate that most center correction is already achieved by geometric refinement, while learned proposal-relative refinement substantially improves box extent; joint geometry + local evidence provides a further, smaller improvement in localization quality.

Overall, conventional proposal-relative regression accounts for most of the repair effect, while joint geometry + local evidence provides a smaller but statistically supported additional gain. We will revise the manuscript to report the matched baseline, uncertainty intervals, center/size errors, and this narrower interpretation.

Note that the experiments in the original submission were conducted on two NVIDIA RTX 5090 GPUs, while the additional rebuttal experiments were conducted on two NVIDIA RTX 4090 GPUs.

Appreciate the authors for the extensive rebuttal, which addresses most of my concerns, I have accordingly raised my overall score to weak accept.

Thank you for your feedback and for supporting our previous rebuttal.

Reviewer_H7VEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces CounterGround, a paired visual grounding benchmark that switches target instances through minimal spatial-relation edits. Across multiple MLLMs and two image domains, the authors observe that direction consistency and nearest-center instance association can remain correct despite inaccurate bounding boxes. They further conduct controlled diagnostic experiments and propose a geometric and local-feature refinement pipeline that improves grounding accuracy. The evaluation perspective is useful, but the diagnostic interpretation and the contribution of the repair design need stronger validation.

Critical strengths
  • The controlled target-switch design provides a useful perspective on the distinction between instance association and localization accuracy.

  • Experiments span multiple MLLMs and two visual domains, with repair gains on held-out and standard grounding benchmarks.

  • The paper clearly acknowledges the limits of GT-assisted diagnosis, paired accuracy, and dataset-specific refitting.

Critical weaknesses
  • The presentation relies heavily on abstract terminology. Phrases such as “relation-consistent responses,” “intended-instance association,” and “structured behavioral separation” make the underlying ideas harder to follow. Although these terms are defined, the paper would benefit from plainer language and concrete examples that explain what each metric measures and how the distinctions support the main findings.
  • The diagnosis does not clearly isolate error sources. GT-candidate selection supplies exact geometry, while the fixed-center experiments use a separately trained predictor. These changes make it difficult to identify what causes the original MLLM’s errors.
  • Stronger matched baselines are needed. Comparing against a direct box-regression head using the same features, supervision, and training budget would help establish the contribution of the proposed repair design.
  • The authors could discuss how interpretability methods, such as [1, 2], could help analyze why incorrect grounding predictions become correct after repair.
  • The distinction from related counterfactual grounding work needs clarification. The paper does not discuss [3], which also introduces controlled counterfactual evaluation and a mitigation approach. The authors should clarify how CounterGround differs in task design, diagnostic findings, and repair strategy, particularly given its focus on switching between existing instances rather than handling absent objects.

[1] Interpreting object-level foundation models via visual precision search. CVPR 2025.

[2] Where mllms attend and what they rely on: Explaining autoregressive token generation. CVPR 2026.

[3] Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination. CVPR 2026.

Questions for authors

Please see the weaknesses.

5 comments

Thank you for the constructive review.

To W1:

We agree that some terminology in the original manuscript is unnecessarily abstract and may obscure the underlying evaluation procedure. Our intention is simply to separate three directly observable properties of paired grounding predictions. Specifically, we measure: (1) whether the predicted center moves in the direction required by the target switch (e.g., from the leftmost instance toward the rightmost instance); (2) whether the predicted center is geometrically associated with the intended same-class candidate; and (3) whether the predicted bounding box accurately overlaps the intended target under the IoU criterion. These measurements correspond to progressively stricter aspects of the grounding output, rather than to latent stages of the model's internal reasoning. We will therefore replace abstract phrases such as "relation-consistent responses," "intended-instance association," and "structured behavioral separation" with these direct descriptions and add concrete examples illustrating what each measurement captures. This revision changes only the presentation and interpretation, not the benchmark construction or evaluation procedure.

To W2:

We appreciate this important distinction. Our diagnostic conditions are not intended to provide a causal decomposition of the original MLLM's internal errors. GT-candidate selection is an oracle-assisted intervention: the same MLLM selects among explicitly provided ground-truth same-class candidates with randomly permuted labels. It examines target selection when candidate geometry is supplied, rather than ordinary free-form localization. Fixed-center extent prediction and the GT-size oracle serve a different purpose. They use an independently trained center predictor and probe the remaining localization performance when the center is fixed and the extent is either predicted or supplied from ground truth.

We agree that these conditions do not isolate the internal error sources of the original MLLM. We will therefore describe them explicitly as controlled behavioral interventions and narrow our interpretation: they characterize performance under controlled geometric assistance rather than establish a causal explanation of the MLLM's internal computation.

To W3:

We added a conventional proposal-relative residual box regressor closely matched to our joint geometry + local evidence refiner, referred to as the structured local-visual refiner in our rebuttal experiments. It receives the same 578-dimensional deployable inputs, including identically preprocessed visual features and the same geometric/crop scalars, and uses the same DIOR training split, box-level supervision, optimization budget, three training seeds, and approximately matched trainable capacity (90,301 vs. 89,995 parameters). The validation split is not used for training, and all compared refiners use the same fixed-final checkpoint policy.

The original free-form prediction obtains 49.09% Pair Accuracy and 55.45 mIoU. The matched residual regressor improves these to 59.65% and 57.03, respectively, while the joint geometry + local evidence refiner reaches 61.17% Pair Accuracy and 58.44 mIoU. Thus, joint geometry + local evidence improves Pair Accuracy over the matched conventional refiner by +1.52 percentage points, with a paired-bootstrap 95% CI of [+0.75, +2.28] using 10,000 complete-pair resamples.

This strong baseline shows that conventional learned refinement explains most of the repair gain. We therefore narrow our claim: repairability is not unique to our architecture; rather, joint geometry + local evidence provides a smaller but statistically supported additional improvement under closely matched information, supervision, training budget, and capacity. We will add this comparison and explicitly distinguish the general benefit of learned proposal-relative refinement from the incremental contribution of joint geometry + local evidence. We will also disclose that two architecture-specific regularizers in its structured decoder have no direct counterpart in the conventional MLP, so the total optimization objectives are not literally identical.

To W4:

We appreciate the suggested references. "Interpreting Object-level Foundation Models via Visual Precision Search" (CVPR 2025) provides fine-grained visual-region attribution for object-level foundation models, while "Where MLLMs Attend and What They Rely On: Explaining Autoregressive Token Generation" (CVPR 2026) analyzes the perceptual evidence underlying autoregressive MLLM outputs. These approaches provide complementary tools for investigating the visual evidence behind grounding predictions. Our current work focuses on observable target-switch behavior, controlled geometric diagnosis, and localization repair; it does not claim to explain the MLLM's internal feature attribution. We will incorporate these works into Related Work and discuss their potential for further analyzing which local visual regions contribute to successful localization repair.

To W5:

Thank you for pointing out this relevant work. We agree that "Counterfactual Segmentation Reasoning: Diagnosing and Mitigating Pixel-Grounding Hallucination" (CVPR 2026 Findings; hereafter referred to as CSR) should be discussed explicitly. Although CSR and CounterGround both adopt controlled counterfactual evaluation, their task formulations are different. CSR studies pixel-grounding hallucination using factual and counterfactual image pairs: when visual evidence supports the presence of the queried object, the model should segment it, whereas when the object is absent in the counterfactual image, the model should abstain from producing a segmentation. CSR further introduces RobustSeg, which uses counterfactual fine-tuning to improve this behavior. In contrast, CounterGround adopts a positive-positive target-switch setting: both expressions in a pair refer to real, existing instances in the same unchanged image. A minimal edit to the spatial relation switches the intended target from one same-class instance to another. Our goal is to distinguish between responding to the target switch, geometrically associating the prediction with the intended candidate, and accurately localizing its bounding box. Thus, CSR primarily addresses visual-presence verification and pixel-grounding hallucination, whereas CounterGround focuses on instance disambiguation and geometric localization under controlled target switches. We will add CSR to Related Work and clarify this complementary relationship.

Note that the experiments in the original submission were conducted on two NVIDIA RTX 5090 GPUs, while the additional rebuttal experiments were conducted on two NVIDIA RTX 4090 GPUs.

Reviewer_H7VEOpenReview ↗

Reply to author rebuttal

I appreciate the author's rebuttal, which has addressed some of my concerns. However, regarding point W3, the authors only mention a strategy of comparing against strong baselines without specifying the actual methods used. It would be better to include comparisons with state-of-the-art methods—such as those recently published in top-tier conferences or journals—or with relevant transfer learning approaches. Consequently, I am inclined to maintain my current rating.

Reviewer_5rMVOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces CounterGround, a benchmark for pairwise counterfactual visual grounding. Each pair shares an image and an object category, but differs in a spatial modifier such as leftmost vs. rightmost. The authors designed a set of metrics and diagnostics to evaluate existing VLMs, demonstrating their inability to perform precise localization on different model scales. Controlled interventions show that the models perform much better when selecting among candidate boxes, and a learned module is designed on top of frozen VLMs to raise their performance on grounding.

Critical strengths
  1. The paper address a valuable question that previous grounding metrics cannot answer by introducing controlled, paired evaluation, i.e., whether a failure comes from choosing the wrong instance or from imprecise localization.
  2. The analyses go beyond accuracy, combining behavioral diagnostics and controlled interventions, based on which the authors designed a refinement approach.
Critical weaknesses
  1. The two extreme spatial relations the the benchmark addresses are too simple to support the conclusions. The leftmost/rightmost and topmost/bottommost relations may be easy to distinguish and undermine the strength of the conclusions.
  2. The construction procedure is under-specified. The authors did not state how they construct the dataset, and whether they put any effort into screening the data or ensuring its quality.
Questions for authors

See weaknesses.

1 comment

Thank you for the careful review.

To W1:

Our goal is not to build a comprehensive benchmark for complex spatial reasoning, but to create a controlled intervention in which the target identity changes while other factors remain fixed. The use of leftmost ↔ rightmost and topmost ↔ bottommost is intentional: these relation edits allow us to keep the image, object category, and same-class candidate set unchanged while changing only the target-defining relation and, consequently, the annotated physical target. This controlled design makes it possible to compare the two predictions under nearly identical visual and linguistic conditions and to determine which aspects of the prediction change correctly when the target switches. We therefore view CounterGround as a diagnostic benchmark for controlled target switching rather than a benchmark intended to cover the full space of spatial relations. We will clarify this scope in the revised manuscript and explicitly state that extending the framework to richer relations, such as relative distance, between-object relations, or compositional spatial descriptions, is an important direction for future work.

To W2:

We agree that the construction criteria and quality-control process should be described more explicitly. CounterGround is constructed from the existing instance annotations of DIOR and COCO 2017. For each image and target category, we organize annotated instances into a shared same-class candidate set containing at least two distinct instances. Each counterfactual pair consists of two referring expressions associated with the same image, category, and candidate set, but with different physical target instances. We consider two controlled relation families: (1) horizontal extremes, where leftmost is replaced with rightmost, and (2) vertical extremes, where topmost is replaced with bottommost. The counterfactual expression is generated by replacing exactly one occurrence of the target-defining relation, while preserving the remaining expression. The two members are associated with distinct target-instance identities, and both target instances are included in the shared candidate set.

To ensure construction integrity, we performed a deterministic audit over the entire frozen benchmark, rather than checking only a subset of examples. The audit verifies eight invariants: pair integrity, image preservation, category preservation, relation transformation, expression-level structural editing, physical target switching, bounding-box validity, and candidate-set validity. All 5,635 pairs (11,270 samples) passed all eight checks, including 4,613 DIOR pairs and 1,022 COCO pairs. We additionally checked 109,670 bounding boxes and found no invalid or out-of-bounds boxes. We will expand the dataset construction subsection to describe the selection criteria, pairing procedure, and quality-control protocol.

Note that the experiments in the original submission were conducted on two NVIDIA RTX 5090 GPUs, while the additional rebuttal experiments were conducted on two NVIDIA RTX 4090 GPUs.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions in the Updated Manuscript

We thank all reviewers for their constructive feedback. We have uploaded a revised manuscript with the following changes:

  1. Clarified benchmark scope and construction. We explicitly define CounterGround as controlled switching between existing same-class instances through extremal relation edits. We expand the construction procedure and report a deterministic audit of all 5,635 test pairs against eight structural invariants.

  2. Simplified terminology and clarified diagnostic interpretation. We describe the metrics through observable prediction geometry and clarify that GT-assisted and fixed-center conditions are behavioral probes, rather than a causal decomposition of the MLLM's internal errors.

  3. Added stricter localization diagnostics. We report Center-In-Target (CIT), center-error analyses by object size, and paired-bootstrap confidence intervals for the CIT–Pair Accuracy gaps.

  4. Expanded heuristic-baseline analysis. We analyze GroundingDINO with deterministic extremal ranking, including intended-target detection recall, failure decomposition, and performance across candidate-count and spatial-density groups.

  5. Added a matched conventional refiner and further repair analyses. In an independent rebuttal rerun under closely matched inputs, supervision, training budget, and capacity, the residual baseline reaches 59.65% Pair Accuracy on CounterGround-DIOR validation, compared with 61.17% for joint geometry + local evidence: +1.52 percentage points, 95% CI [0.75, 2.28]. We accordingly distinguish the general benefit of learned proposal-relative refinement from this incremental gain, and add center/size-error and success–failure transition analyses to clarify the relationship between Pair Accuracy and mean IoU.

  6. Expanded Related Work. We discuss CSR and the suggested visual-attribution methods, clarifying differences in task formulation, diagnostic scope, and mitigation and repair strategies.

The original-submission results remain in the main experimental tables. The additional rebuttal experiments are kept as an independent rerun, with detailed results reported in a separate appendix section and their implications reflected in the revised main text and interpretation.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.