- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper introduces UniRefDet, a unified referring segmentation benchmark for salient and camouflaged objects in unconstrained scenes, and CoRefNet, a SAM 3-based framework tailored to fine-grained referring expressions. Experiments demonstrate that the proposed method achieves strong performance.
Critical strengths
- The proposed benchmark is practical and could stimulate further research in unconstrained scenes.
- The proposed framework is well-motivated and achieves competitive performance.
Critical weaknesses
- The primary contribution of this paper is the construction of the UniRefDet benchmark. However, the images are drawn from an existing benchmark, and the generation of referring annotations relies heavily on LMMs. I therefore consider the contribution to be incremental.
- When constructing the UniRefDet benchmark, how did the authors determine whether an object was camouflaged or salient in scenes where both types of objects coexist? Camouflage and saliency are continuous rather than discrete properties, meaning that an object may exhibit both to varying degrees (e.g., a camouflage score of 0.55 and a saliency score of 0.45). If the dataset simply assigns binary labels, could this lead to confusion for the model?
- The core objective of referring image segmentation is to segment the target object specified by a referring expression, with the output containing only one class. In contrast, camouflaged and salient object detection in unconstrained scenes aims to detect both classes simultaneously, producing outputs that contain two classes. I am therefore quite puzzled by the rationale for unifying these two tasks. In practice, if I want to segment a target using a referring expression, whether the target is salient or camouflaged is either irrelevant to my goal or already reflected in the expression. Alternatively, if I want to detect camouflaged and salient objects simultaneously, I would expect the model to make predictions directly, without requiring me to provide complex referring expressions.
Questions for authors
Please refer to Weaknesses.
2 comments
I appreciate the authors’ responses, which have addressed some of my concerns. However, my main concern regarding the motivation for unifying reasoning segmentation with COD/SOD remains unresolved. I am therefore inclined to maintain my current score. Thank you!
We sincerely thank you for the valuable and constructive feedback. Below, we address your concerns point by point. We would be grateful if you could let us know whether our responses adequately address your questions and concerns.
Q1. Is UniRefDet merely an incremental extension of an existing benchmark?
We acknowledge that UniRefDet reuses images from USC12K. However, our contribution goes beyond simply adding captions to existing images. UniRefDet establishes a fine-grained language-guided benchmark for attribute-aware referring segmentation, with expressions covering eight dimensions, including object appearance and object–environment relationships. LMMs are used only to generate initial descriptions, which may contain factual errors, hallucinations, or ambiguities regarding object categories, shapes, quantities, and visual attributes. To ensure annotation quality, an independent VLM first checks image–mask–expression consistency, followed by two rounds of manual review and refinement conducted by three trained annotators under standardized guidelines. This process involves correcting inaccuracies and removing ambiguous or unsupported descriptions, requiring approximately 2–4 minutes per image. Thus, UniRefDet contributes carefully verified fine-grained referring annotations and a unified evaluation protocol for target localization and visual-attribute assignment, while CoRefNet provides an effective solution to this new task.
Q2. Why are saliency and camouflage represented by binary labels?
We agree that saliency and camouflage are context-dependent perceptual properties that may vary continuously. However, existing SOD/COD benchmarks typically use binary segmentation masks for standardized training and evaluation. UniRefDet follows USC12K's three-class segmentation setting (background, salient, and camouflaged), directly inheriting its annotations rather than introducing new thresholds for attribute classification. These categories serve as operational labels for evaluation rather than quantitative measurements of continuous visual attributes.
In Scenes A and B, salient and camouflaged objects are relatively distinguishable. However, as you correctly point out, Scene C presents a more challenging situation where both attributes coexist, and relying solely on binary labels may lead to attribute confusion. This motivates our fine-grained referring annotations. Beyond object categories, our expressions describe target appearance and object–environment relationships, including how a target visually contrasts with or blends into its surroundings. These descriptions provide complementary contextual cues for distinguishing targets with different visual attributes, rather than relying solely on categorical labels.
We acknowledge that such annotations cannot fully eliminate the inherent ambiguity of continuous perceptual properties, particularly in borderline cases. Nevertheless, they provide richer supervision for studying attribute-aware referring segmentation, especially in coexistence scenes. We will clarify this motivation and the limitations of categorical annotations in the revised manuscript.
Q3. Why is it necessary to unify referring segmentation with SOD/COD?
We appreciate this important question. Our key motivation is that objects of the same semantic category may exhibit different visual attributes within the same scene. For example, two animals of the same category may coexist, with one visually salient and the other blending into its surroundings. A user may wish to locate each animal through a different referring expression. However, applying SOD and COD independently may produce overlapping predictions or ambiguous attribute assignments, particularly when objects exhibit competing visual characteristics. UniRefDet provides a unified framework for investigating how fine-grained language descriptions can help distinguish such targets and alleviate attribute ambiguity.
This setting also has potential practical applications. For instance, in wildlife monitoring, researchers may need to distinguish animals clearly visible in open areas from those concealed by surrounding vegetation, rather than detecting all animals indiscriminately. Similarly, in UAV-assisted search and rescue, operators may use language to specify particular individuals who are either clearly visible or partially concealed by debris or vegetation. The value of UniRefDet lies in providing a complementary benchmark for studying language-guided target disambiguation under competing visual attributes, rather than simply combining existing tasks. We will further clarify this motivation and its distinction from conventional referring segmentation and SOD/COD in the revised manuscript.