- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces InstanceBench, an instance-centered diagnostic benchmark for referring expression segmentation (RES). It is built around target-centric expression sets (TCESs), each of which pairs multiple expressions referring to the same target instance through different referential logics. The authors evaluate a diverse set of RES model families and MLLMs and report several diagnostic findings, revealing gaps in reliability and target-identity preservation.
Critical strengths
-
The proposed target-centric expression sets provide a structured way to evaluate whether models understand different referential logics.
-
The paper presents a comprehensive evaluation of existing RES models and MLLMs and delivers interesting findings.
Critical weaknesses
-
Closely related benchmarks such as OmniLabel and D³ (Described Object Detection) are not discussed. Both evaluate grounding of free-form descriptions, including negation and relational expressions. The authors should discuss these benchmarks and clarify InstanceBench’s contributions relative to them.
-
The paper proposes new metrics such as All@τ, All-target-ID, and NCI, but it is not clear why they are important or what additional information they provide beyond standard metrics. For example, All@τ is by construction upper-bounded by per-expression accuracy. The authors should demonstrate that these metrics reveal insights that conventional metrics such as mIoU or P@τ cannot.
-
The expressions derive from GPT-5.5 proposals, which may introduce linguistic or distributional bias. The authors should analyze this, for example by comparing performance on human-written versus model-generated expressions.
-
The paper reports several diagnostic findings, but it does not establish that they are new and could not have been observed with existing benchmarks.
Questions for authors
- How do the authors ensure that there is no train/test contamination, given that the COCO-derived images and masks come from existing benchmarks?
- How do the authors ensure that the reported findings are new and not observable in existing benchmarks?
Dear Reviewer rZy6,
As the discussion period is drawing to a close, we wanted to check whether you have any remaining concerns that we could further address.
If our responses have adequately addressed your concerns, we would greatly appreciate your consideration of a score update.
Thank you again for your thoughtful feedback and engagement!