- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
This paper introduces Expanded Class-Agnostic Detection (ECAD), which aims to detect both conventional thing instances and semantically meaningful stuff-like regions, such as sky, road, water, and grassland, using category-agnostic bounding boxes. The motivation of revisiting the conventional thing-centric notion of objectness is interesting. However, I have substantial concerns regarding the task definition, annotation protocol, benchmark validity, and experimental evaluation.
Critical strengths
- The benchmark includes real-world and cross-domain images, including artistic styles and adverse conditions.
- ECADet has relatively few trainable parameters due to the frozen DINOv3 encoder.
Critical weaknesses
- The paper does not sufficiently justify why regions should be represented by boxes rather than semantic masks. Saying that detection is sparse while segmentation is dense is not enough.
- Although the task is class-agnostic, annotation still relies on an implicit semantic ontology. It is unclear what should be included as foreground and at what granularity. A clear inclusion/exclusion taxonomy and annotation rules are needed.
Questions for authors
-
How are nested, overlapping, or hierarchical regions handled in dataset?
-
What is the annotation granularity? Are body parts, building components, windows, roads, floors, courts, and other contextual regions all annotated as separate foreground instances? If not, what criteria determine whether a region should be included?
-
What quality-control protocol is used during annotation? Please report annotation consistency and other statistics.
-
Currently, all results are reported on the authors' own BTCO datasets, and SAM3 is the only large foundation-model baseline. To properly validate the method's generalizability and SOTA status, can you provide evaluations on standard public benchmarks and compare against stronger, more recent general-purpose models (e.g., Rex-Omni, DINO-X)?
-
The current ablations are too coarse, as GAER and PGQM are only evaluated as complete modules. Could you provide finer-grained ablations to isolate the specific sources of improvement?
-
The paper claims that GAER models different geometric patterns, but lacks supporting empirical analysis. Can you provide an expert-routing analysis or visualizations to verify that different experts actually learn to specialize in distinct geometric patterns as claimed?
8 comments
Dear Reviewer ARZ4,
As the discussion period draws to a close, we would like to follow up on our response above. We clarified the rationale for bounding boxes, foreground inclusion and annotation granularity, and annotation quality control. We also added baseline and architectural comparisons (Tables 2 and 3) and GAER expert visualizations (Figure 7, Appendix C.2).
We would appreciate your feedback on whether these clarifications and results address your concerns about the task definition, benchmark, and empirical evidence. We would be happy to clarify any remaining questions. Your feedback would be very valuable in helping us further improve the paper.
Thank you for your time.
Thanks for your detailed response. For the Q5 is well resolved. However, my mainly concerns related to the scope of annotations, and the response has not completely resolved.
- The rules of "semantically or functionally meaningful environmental regions", "independently identifiable region" is still subjective. when anaotating a person, as the head, hands, fingers should be annotated?.
- Section 3.2 mentions five annotators, whereas the response refers to two reviewers. Please clarify whether these reviewers are among the five annotators.
- Are there no publicly available datasets that can be used to evaluate ECADet performance?
Thank you for your careful review and constructive suggestions. The revised manuscript adds baseline comparisons, simpler architectural controls, and a visualization of GAER expert specialization.
R1. Why bounding boxes
ECAD generates proposals for downstream use. Boxes provide the position and extent needed for region selection, cropping, and ROI feature extraction through a common interface for objects and environmental regions. Selected boxes can also prompt mask generation when precise contours are needed. Box annotation typically costs less than tracing pixel-level contours, allowing wider image and region coverage within the same budget.
On BTCO-Real and BTCO-XDomain, ECADet-L achieves 41.0 and 65.1 AP, respectively. CUPS mask-to-box conversion yields 3.9 and 5.8 AP, and SAM3 with over 500 prompts yields 32.2 and 39.2 AP after conversion. In some CUPS examples, a region is split into small masks. Converting the fragments yields several local boxes, making the full region harder to select as one proposal.
R2. Foreground definition, annotation granularity, and quality control
ECAD uses one foreground label. BTCO-Real adds independently identifiable objects or components (e.g., windows and signs) and semantically or functionally meaningful environmental regions (e.g., roads, floors, and courts) to the original LVIS instance annotations.
Each annotation covers an independently identifiable region. We annotate continuous roads, floors, and courts as whole regions across local color or texture changes. A part receives its own box only if it has a distinct meaning or function and an independently identifiable extent, as with a clearly visible window. We retain overlapping or nested boxes when both regions qualify, as with overlapping boxes for a vehicle and a road.
All 24,639 BTCO-Bench images were annotated under the same rules. For each batch, two reviewers independently decided whether to retain, adjust, or remove each initial box. Before joint review, they agreed on how to handle 286,731 of 305,021 boxes (94.0% box-level agreement on review decisions). After review, we adjusted the boundaries of 9,607 boxes, removed 273, and added 3,268 missed regions, yielding 308,016 boxes.
R3. Public benchmarks and general-purpose models
ECAD requires consistent foreground annotations for both conventional instances and environmental regions. Standard detection benchmarks primarily annotate conventional instances. Converting panoptic masks to boxes preserves the original region partition and granularity. Valid ECAD predictions may count as false positives when the corresponding regions are unannotated. BTCO-Real (Test) adds ECAD annotations to images from the public LVIS minival split.
Given the API cost and time required to run DINO-X on the full test sets, we evaluated Rex-Omni, an open-vocabulary detector. Under the same class-agnostic box evaluation protocol, Rex-Omni achieves 13.0 and 20.2 AP on BTCO-Real and BTCO-XDomain, respectively, and ECADet-L achieves 41.0 and 65.1 AP. The full comparisons with Rex-Omni, CUPS, and SAM3 with expanded prompts are in Tables 2 and 3 of the revised manuscript.
R4. Module ablations, configuration analysis, and architectural controls
Table 4 reports results for GAER and PGQM individually and jointly. Tables 5–7 examine the number of GAER experts, the number of PGQM prototypes, and the GAER regression residual scale, respectively. Six experts, four prototypes, and a residual scale of 1.1 yield the highest overall AP in their respective experiments.
We evaluated two simpler RF-DETR variants with the same frozen DINOv3-L backbone. The wider-head variant matches ECADet-L in total and trainable parameters. Basic query modulation serves as a simpler control for PGQM. ECADet-L with both modules outperforms both controls on both test sets (Tables 2 and 3 of the revised manuscript).
R5. GAER expert routing and box regression
We visualized routing responses and box regression for all six GAER experts across bins of relative box area and aspect ratio (Figure 7, Appendix C.2 of the revised manuscript). At comparable box areas, the bottom-center expert responds more strongly to horizontally elongated boxes, and the bottom-left expert responds more strongly to vertically elongated boxes (Fig. 7(a)). Across most bins, the bottom-left expert adjusts width more than height, and the bottom-center expert adjusts height more than width (Fig. 7(b)). These observations support geometry-dependent specialization in expert routing and box regression.