acceptodds
Under review as a conference paper at ICLR 2027

ECAD: Expanding Class-Agnostic Detection Beyond Thing-Centric Objectness

Abstract

Object detection is a fundamental task in visual perception, providing structured region representations for recognition, grounding, reasoning, and interaction. However, existing detection paradigms largely inherit a thing-centric notion of objectness, where detectors are mainly trained to localize discrete and countable object instances. Consequently, many semantically meaningful visual elements, such as sky, road, grassland, water, and sports courts, are often absorbed into the background despite their importance for scene understanding and spatial reasoning. In this paper, we formulate Expanded Class-Agnostic Detection (ECAD), a new setting that aims to discover category-agnostic visual candidates beyond conventional thing-centric objects. To support this setting, we construct BTCO-Bench, a Beyond Thing-Centric Objectness benchmark with category-agnostic box annotations covering both real-world and cross-domain scenarios. We further propose ECADet, a lightweight DETR-based detector built upon a frozen DINOv3 encoder, and introduce Geometry-Aware Expert Regression (GAER) and Prototype-Guided Query Modulation (PGQM) to improve localization and objectness estimation for diverse visual elements, respectively. Extensive experiments show that ECADet consistently outperforms representative class-agnostic and proposal-based detectors on BTCO-Bench, demonstrating the effectiveness of expanded objectness discovery. Code and benchmark will be released.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Open-Set, Object Detection, Language-Vision-Complementary

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ARZ4OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces Expanded Class-Agnostic Detection (ECAD), which aims to detect both conventional thing instances and semantically meaningful stuff-like regions, such as sky, road, water, and grassland, using category-agnostic bounding boxes. The motivation of revisiting the conventional thing-centric notion of objectness is interesting. However, I have substantial concerns regarding the task definition, annotation protocol, benchmark validity, and experimental evaluation.

Critical strengths
  1. The benchmark includes real-world and cross-domain images, including artistic styles and adverse conditions.
  2. ECADet has relatively few trainable parameters due to the frozen DINOv3 encoder.
Critical weaknesses
  1. The paper does not sufficiently justify why regions should be represented by boxes rather than semantic masks. Saying that detection is sparse while segmentation is dense is not enough.
  2. Although the task is class-agnostic, annotation still relies on an implicit semantic ontology. It is unclear what should be included as foreground and at what granularity. A clear inclusion/exclusion taxonomy and annotation rules are needed.
Questions for authors
  1. How are nested, overlapping, or hierarchical regions handled in dataset?

  2. What is the annotation granularity? Are body parts, building components, windows, roads, floors, courts, and other contextual regions all annotated as separate foreground instances? If not, what criteria determine whether a region should be included?

  3. What quality-control protocol is used during annotation? Please report annotation consistency and other statistics.

  4. Currently, all results are reported on the authors' own BTCO datasets, and SAM3 is the only large foundation-model baseline. To properly validate the method's generalizability and SOTA status, can you provide evaluations on standard public benchmarks and compare against stronger, more recent general-purpose models (e.g., Rex-Omni, DINO-X)?

  5. The current ablations are too coarse, as GAER and PGQM are only evaluated as complete modules. Could you provide finer-grained ablations to isolate the specific sources of improvement?

  6. The paper claims that GAER models different geometric patterns, but lacks supporting empirical analysis. Can you provide an expert-routing analysis or visualizations to verify that different experts actually learn to specialize in distinct geometric patterns as claimed?

8 comments

Thank you for your careful review and constructive suggestions. The revised manuscript adds baseline comparisons, simpler architectural controls, and a visualization of GAER expert specialization.

R1. Why bounding boxes

ECAD generates proposals for downstream use. Boxes provide the position and extent needed for region selection, cropping, and ROI feature extraction through a common interface for objects and environmental regions. Selected boxes can also prompt mask generation when precise contours are needed. Box annotation typically costs less than tracing pixel-level contours, allowing wider image and region coverage within the same budget.

On BTCO-Real and BTCO-XDomain, ECADet-L achieves 41.0 and 65.1 AP, respectively. CUPS mask-to-box conversion yields 3.9 and 5.8 AP, and SAM3 with over 500 prompts yields 32.2 and 39.2 AP after conversion. In some CUPS examples, a region is split into small masks. Converting the fragments yields several local boxes, making the full region harder to select as one proposal.

R2. Foreground definition, annotation granularity, and quality control

ECAD uses one foreground label. BTCO-Real adds independently identifiable objects or components (e.g., windows and signs) and semantically or functionally meaningful environmental regions (e.g., roads, floors, and courts) to the original LVIS instance annotations.

Each annotation covers an independently identifiable region. We annotate continuous roads, floors, and courts as whole regions across local color or texture changes. A part receives its own box only if it has a distinct meaning or function and an independently identifiable extent, as with a clearly visible window. We retain overlapping or nested boxes when both regions qualify, as with overlapping boxes for a vehicle and a road.

All 24,639 BTCO-Bench images were annotated under the same rules. For each batch, two reviewers independently decided whether to retain, adjust, or remove each initial box. Before joint review, they agreed on how to handle 286,731 of 305,021 boxes (94.0% box-level agreement on review decisions). After review, we adjusted the boundaries of 9,607 boxes, removed 273, and added 3,268 missed regions, yielding 308,016 boxes.

R3. Public benchmarks and general-purpose models

ECAD requires consistent foreground annotations for both conventional instances and environmental regions. Standard detection benchmarks primarily annotate conventional instances. Converting panoptic masks to boxes preserves the original region partition and granularity. Valid ECAD predictions may count as false positives when the corresponding regions are unannotated. BTCO-Real (Test) adds ECAD annotations to images from the public LVIS minival split.

Given the API cost and time required to run DINO-X on the full test sets, we evaluated Rex-Omni, an open-vocabulary detector. Under the same class-agnostic box evaluation protocol, Rex-Omni achieves 13.0 and 20.2 AP on BTCO-Real and BTCO-XDomain, respectively, and ECADet-L achieves 41.0 and 65.1 AP. The full comparisons with Rex-Omni, CUPS, and SAM3 with expanded prompts are in Tables 2 and 3 of the revised manuscript.

R4. Module ablations, configuration analysis, and architectural controls

Table 4 reports results for GAER and PGQM individually and jointly. Tables 5–7 examine the number of GAER experts, the number of PGQM prototypes, and the GAER regression residual scale, respectively. Six experts, four prototypes, and a residual scale of 1.1 yield the highest overall AP in their respective experiments.

We evaluated two simpler RF-DETR variants with the same frozen DINOv3-L backbone. The wider-head variant matches ECADet-L in total and trainable parameters. Basic query modulation serves as a simpler control for PGQM. ECADet-L with both modules outperforms both controls on both test sets (Tables 2 and 3 of the revised manuscript).

Model configurationBTCO-Real APBTCO-XDomain APParameters (M, total / trainable)
RF-DETR24.526.0315.1 / 11.9
RF-DETR + Wider Regression Head26.737.1316.1 / 12.9
RF-DETR + Basic Query Modulation32.945.0315.5 / 12.4
ECADet-L41.065.1316.1 / 12.9

R5. GAER expert routing and box regression

We visualized routing responses and box regression for all six GAER experts across bins of relative box area and aspect ratio (Figure 7, Appendix C.2 of the revised manuscript). At comparable box areas, the bottom-center expert responds more strongly to horizontally elongated boxes, and the bottom-left expert responds more strongly to vertically elongated boxes (Fig. 7(a)). Across most bins, the bottom-left expert adjusts width more than height, and the bottom-center expert adjusts height more than width (Fig. 7(b)). These observations support geometry-dependent specialization in expert routing and box regression.

Dear Reviewer ARZ4,

As the discussion period draws to a close, we would like to follow up on our response above. We clarified the rationale for bounding boxes, foreground inclusion and annotation granularity, and annotation quality control. We also added baseline and architectural comparisons (Tables 2 and 3) and GAER expert visualizations (Figure 7, Appendix C.2).

We would appreciate your feedback on whether these clarifications and results address your concerns about the task definition, benchmark, and empirical evidence. We would be happy to clarify any remaining questions. Your feedback would be very valuable in helping us further improve the paper.

Thank you for your time.

Thanks for your detailed response. For the Q5 is well resolved. However, my mainly concerns related to the scope of annotations, and the response has not completely resolved.

  1. The rules of "semantically or functionally meaningful environmental regions", "independently identifiable region" is still subjective. when anaotating a person, as the head, hands, fingers should be annotated?.
  2. Section 3.2 mentions five annotators, whereas the response refers to two reviewers. Please clarify whether these reviewers are among the five annotators.
  3. Are there no publicly available datasets that can be used to evaluate ECADet performance?
Reviewer_4UoBOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper revisits the scope of objectness in class-agnostic detection and introduces Expanded Class-Agnostic Detection (ECAD): discovering not only conventional thing instances such as humans, vehicles but also semantically or functionally meaningful visual elements often treated as background, e.g., sky, sea, grassland, roads, and courts, without assigning class names. The authors construct BTCO-Bench, with BTCO-Real and BTCO-XDomain splits providing category-agnostic box annotations. They propose ECADet, a lightweight RF-DETR-based detector with a frozen DINOv3 encoder, Geometry-Aware Expert Regression (GAER) for diverse shapes/boundary ambiguity, and Prototype-Guided Query Modulation (PGQM) for objectness estimation.

Critical strengths

Novel problem: extends class-agnostic objectness beyond thing-centric instances to semantically meaningful non-thing elements, distinct from open-vocabulary detection and segmentation.

Benchmark contribution: BTCO-Bench covers real-world and cross-domain scenarios, with category-agnostic box annotations and explicit design principles.

Lightweight method: frozen DINOv3 with lightweight trainable modules; GAER handles geometric diversity, PGQM injects prototype-based objectness priors; ablations show complementary gains.

Critical weaknesses

Small training set: BTCO-Real training uses only 1,000 LVIS images; scaling is unknown.

Annotation subjectivity and boundary ambiguity: stuff-like regions such as sky, grassland, and roads have less clear boundaries; inter-annotator agreement is not sufficiently reported.

Unified foreground label limits evaluation: only objectness/localization is evaluated, not semantic correctness; overlapping or hierarchical candidates are not systematically assessed.

Overlap/hierarchy handling is under-analyzed: the paper motivates overlapping/hierarchical candidates, but how sparse ECADet boxes handle person-parts or stuff-thing overlaps is not thoroughly evaluated.

Questions for authors

How does ECADet handle overlapping and hierarchical candidates, e.g., person-parts or thing-stuff overlaps, to avoid duplication or omission?

BTCO-Real training uses only 1,000 images. How does expanded objectness scale with training data?

Can ECADet be used in other continual learning tasks other than BTCO? Does it still outperforms other methods?

Is freezing DINOv3 essential, or mainly an efficiency trade-off? How does it compare with fine-tuning DINOv3 or using other visual foundation models?

5 comments

Thank you for your careful review and constructive suggestions. We have revised the manuscript to include an analysis of training set size, separate Thing and Stuff evaluations, and comparisons of frozen, fine-tuned, and alternative visual encoders.

R1. Training set size

We added BTCO-Real experiments with 300, 500, 800, and 1,000 training images (Appendix B, Table 8 of the revised manuscript). Expanding the training set requires manually annotating regions outside the scope of the original LVIS annotations and reviewing the added boxes. We therefore used subsets of the existing annotated training set to keep the foreground definition and annotation protocol consistent across sizes.

Training images3005008001,000
RF-DETR AP12.218.824.624.5
ECADet-L AP17.322.334.141.0

With the same frozen DINOv3-L backbone, ECADet-L's AP rises steadily with training set size and exceeds RF-DETR at all four sizes. With 800 images, ECADet-L achieves 34.1 AP, 9.6 points above RF-DETR trained on all 1,000 images.

R2. Boundary annotation and quality control

For regions with ambiguous boundaries, such as sky, grassland, and roads, annotators box the discernible extent of each region using visible contours, boundaries with adjacent regions, and image edges as cues. After initial annotation, two additional reviewers independently decided whether to retain, adjust, or remove each box. They agreed on 286,731 of the 305,021 initial boxes in BTCO-Bench, yielding 94.0% box-level agreement on review decisions. We cross-checked and reconciled disagreements over region extent or boundaries.

R3. Unified foreground label and overlapping proposals

ECAD provides class-agnostic proposals for conventional object instances and meaningful environmental regions. Box locations and extents can be used directly for region interaction, cropping, and ROI feature extraction. Downstream systems can predict category labels as needed. We therefore use a single foreground label to evaluate region discovery and localization directly.

ECADet treats overlapping or nested regions as independent targets when they are annotated separately under our protocol, such as a person and a separately annotated part, or an object and an environmental region. During training, Hungarian matching assigns distinct predictions to distinct targets. At evaluation, overlapping predictions count as separate true positives when they match different annotations. Each annotation can be matched once. Unmatched duplicate predictions count as false positives, and missed targets reduce recall. The unified box evaluation thus accounts for detections, misses, and duplicate predictions for these regions.

R4. ECADet and downstream continual learning

ECADet generates class-agnostic boxes covering conventional objects and environmental regions, which downstream systems such as continual learning systems can select as needed. We use BTCO-Real and BTCO-XDomain to evaluate proposal discovery, localization, objectness ranking, and cross-domain generalization.

Strong performance on BTCO-Bench demonstrates that ECADet meets the proposal-generation goal of providing high-quality proposals with broad coverage for downstream use. Our method design and performance comparisons therefore focus on region proposal generation.

R5. Encoder freezing and backbone comparisons

Freezing DINOv3 is a design choice. We retain its pretrained dense visual representations and train the detection modules on limited labeled data to discover and localize expanded foreground regions while updating fewer parameters.

Freezing also facilitates sharing the visual encoder with downstream systems. When a downstream system uses the same frozen weights, input preprocessing, and feature extraction settings, ECADet can reuse its visual features. A single encoding of the image can serve both the downstream task and region proposal generation.

ECADet backboneBTCO-Real APBTCO-XDomain APTrainable parameters
DINOv3-L (frozen)41.065.112.9M
DINOv3-L (fine-tuned)45.854.9316.1M
SigLIP2-L (frozen)38.252.312.9M

Fine-tuning DINOv3-L raises in-domain AP by 4.8 points. The frozen variant achieves 10.2 points higher AP on BTCO-XDomain while updating only 12.9M parameters. The SigLIP2-L results show that ECADet can also generate expanded region proposals with a different visual foundation model. We use frozen DINOv3-L by default based on cross-domain performance, trainable parameter count, and feature reuse. The full results are reported in Tables 2 and 3 of the revised manuscript.

Dear Reviewer 4UoB,

As the discussion period draws to a close, we would like to follow up on our response above. We added training set size experiments (Appendix B, Table 8) and comparisons of frozen and fine-tuned DINOv3-L and frozen SigLIP2-L (Tables 2 and 3). Our response also reports annotation-review agreement and explains how overlapping regions are supervised and evaluated.

We would appreciate your feedback on whether these experiments and clarifications address your concerns. We would be happy to clarify any remaining questions. Your feedback would be very valuable in helping us further improve the paper.

Thank you for your time.

Reviewer_4UoBOpenReview ↗

Other benchmarks besides BTCO?

It would be better if the authors could provide experimental results for ECAD on different continual learning benchmarks.

Reviewer_6mhLOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ECAD, which detects both thing and stuff regions as category-agnostic boxes. It also presents BTCO-Bench and ECADet, a detector built on frozen DINOv3 features with two additional modules.

Critical strengths

1, Discovering regions beyond conventional objects is useful, and the benchmark construction is a valuable effort.

2, The reported gains over the RF-DETR baseline are substantial, with limited additional parameters and runtime overhead.

Critical weaknesses

1, The task novelty is not clear enough. Panoptic segmentation already covers both things and stuff. Predicting boxes without category labels is a different output setting, but I would like stronger evidence that this creates a distinct research problem with practical advantages.

2, Some important comparisons are missing. Standard panoptic segmentation models converted to boxes and relevant open-vocabulary detectors would provide useful baselines. The SAM3 comparison uses only the generic prompt “objects,” which seems insufficient to establish the limitations of segmentation-based approaches.

3, The architecture changes appear incremental. GAER and PGQM combine expert routing, prototype matching, and residual modulation. The ablations show that they help, but comparisons with simpler alternatives, such as a larger regression head or basic query modulation, would make their contribution clearer.

4,The results do not isolate the expanded detection capability. Separate results for things, stuff, and newly annotated regions would help show whether the improvements actually address the paper’s main motivation.

Questions for authors

see above

3 comments

Thank you for your careful review and constructive suggestions. We have revised the manuscript to include additional baselines, comparisons with simpler architectural variants, and separate evaluations for things and stuff.

R1. Task novelty and practical value

We agree that panoptic segmentation already covers things and stuff by partitioning pixels according to semantic labels and instance membership. ECAD expands the definition of foreground in detection by treating conventional object instances and semantically or functionally meaningful environmental regions as independent targets for training and evaluation. We ask whether a detector can learn a unified notion of objectness to discover and localize both types of regions without category prompts.

This setting gives downstream systems a single set of proposals for both objects and their surroundings. For example, to determine which court a person is on, a system can use separate proposals for the person and the court, even when their boxes overlap. See R2 for mask-to-box comparisons and R4 for results on environmental regions.

R2. Additional baselines and SAM3 prompts

We added CUPS (CVPR 2025), an unsupervised panoptic segmentation method, and Rex-Omni, an open-vocabulary detector. We converted CUPS and SAM3 masks to bounding boxes and evaluated all methods using class-agnostic box AP. For SAM3, we expanded the single objects prompt used in the original submission to over 500 text prompts covering things and stuff. The full results are reported in Tables 2 and 3 of the revised manuscript.

MethodBTCO-Real APBTCO-Real Thing APBTCO-Real Stuff APBTCO-XDomain APBTCO-XDomain Thing APBTCO-XDomain Stuff AP
CUPS3.93.012.35.84.015.0
Rex-Omni13.013.312.120.220.619.2
SAM3 (500+ prompts)32.231.934.539.239.637.1
PF-RPN26.430.611.242.948.813.8
RF-DETR (DINOv3-L)24.526.926.426.038.037.7
RF-DETR + Wider Regression Head26.727.023.937.137.834.8
RF-DETR + Basic Query Modulation32.932.635.245.045.145.2
ECADet-L41.043.347.365.160.261.8

Expanding the prompts raises SAM3's box AP from 9.6 to 32.2 on BTCO-Real and from 12.4 to 39.2 on BTCO-XDomain. ECADet-L uses no category prompts and leads by 8.8 and 25.9 AP points, respectively.

R3. Comparison with simpler architectural changes

Using the same frozen DINOv3-L backbone, we evaluated two RF-DETR variants, one with a wider regression head and one with basic query modulation. The wider-head variant and ECADet-L both have 12.9M trainable parameters. Both modifications improve overall AP over the original RF-DETR baseline. The full ECADet-L model with GAER and PGQM outperforms the stronger control, basic query modulation, by 8.1 AP points on BTCO-Real and 20.1 AP points on BTCO-XDomain. The full comparison is reported in Tables 2 and 3 of the revised manuscript.

R4. Evaluation by region type

We report Thing and Stuff AP separately by region type in the table in R2. With the same frozen DINOv3-L backbone, ECADet-L improves Stuff AP over RF-DETR by 20.9 points on BTCO-Real and 24.1 points on BTCO-XDomain. Thing AP also increases by 16.4 and 22.2 points, respectively. The gains extend to both conventional object instances and the environmental regions that ECAD includes as foreground.

Dear Reviewer 6mhL,

As the discussion period draws to a close, we would like to follow up on our response above. Following your suggestions, we added CUPS and Rex-Omni, expanded SAM3’s prompts, tested a wider regression head and basic query modulation, and reported separate Thing/Stuff results (Tables 2 and 3).

We would appreciate your feedback on whether these comparisons, together with the task clarification in R1, address your concerns about the task setting and the sources of improvement. We would be happy to clarify any remaining questions. Your feedback would be very valuable in helping us further improve the paper and clarify its contribution.

Thank you for your time.

Dear Reviewer 6mhL,

As the discussion period draws to a close, we would like to briefly follow up again on our response above. Following your suggestions, we added CUPS and Rex-Omni baselines, SAM3 with expanded prompts, simpler architectural controls, and separate Thing/Stuff evaluations (Tables 2 and 3 of the revised manuscript). We also further clarified the task setting and practical value in R1.

We would be grateful if you could review our point-by-point responses and revised manuscript, and let us know whether these additions have alleviated your main concerns. We also hope you could reconsider your overall assessment of the paper in light of the additional experiments and clarifications. Your feedback and consideration would mean a great deal to us. If any key questions remain, we would be happy to clarify them in the remaining time. Thank you for your time and careful review.

Comments on the paper

3 comments
AuthorsOpenReview ↗

ECAD Revision and Responses to the Reviews

Dear Reviewers and Area Chair,

Thank you for your careful reviews and constructive feedback. We have responded point by point to all three reviews and revised the manuscript to include additional baselines, separate evaluations for things and stuff, comparisons with simpler architectures, and a visualization of GAER expert specialization.

ECAD asks whether a detector can discover and localize both conventional object instances and semantically or functionally meaningful environmental regions without category prompts. For example, a downstream system can use separate proposals for a player and a court to reason about which court the player is on. This use case motivates expanding the definition of foreground in detection. In the revised experiments, with the same frozen backbone, ECADet-L improves Stuff AP over RF-DETR by 20.9 points on BTCO-Real and 24.1 points on BTCO-XDomain, directly measuring gains on the environmental regions that motivate ECAD.

As you continue the discussion, we hope you will consider our point-by-point responses and the new evidence in the revised manuscript when reassessing the value of the task setting and its empirical support. Your feedback is important to us in clarifying ECAD’s contribution, and we would be glad to address any remaining questions.

Dear Reviewers and Area Chair,

As the discussion period draws to a close, we would appreciate any feedback on our point-by-point responses and revised manuscript. We would be happy to clarify any remaining questions. Thank you for your time.

Dear Area Chair and Reviewers,

Thank you for your careful review and discussion of our work. We would like to summarize the research goal, benchmark contribution, and additional evidence presented in our responses, and kindly ask you to consider them together when forming your final assessment.

Our research was motivated by the need for more comprehensive region proposals in a real-world project. Downstream systems need to locate not only conventional objects such as people and vehicles, but also environmental regions such as roads and sports courts, to support region selection, feature extraction, and spatial reasoning. ECAD therefore asks whether a model can discover and localize both types of regions through a unified notion of objectness, without category prompts.

This expanded foreground scope also introduces corresponding evaluation requirements. Based on our investigation of public datasets, including LVIS and V3Det, existing annotations do not fully cover the object instances and environmental regions targeted by ECAD. Predictions that correctly discover unannotated regions may be counted as false positives. We therefore constructed BTCO-Bench to provide consistent annotations and evaluation for this setting.

Following the reviewers’ suggestions, we added CUPS, Rex-Omni, SAM3 with expanded prompts, and controls using a wider regression head and basic query modulation. ECADet-L achieves 41.0 and 65.1 AP on BTCO-Real and BTCO-XDomain, respectively, outperforming these baselines and controls on both benchmarks. With the same frozen DINOv3-L backbone, ECADet improves Stuff AP over RF-DETR by 20.9 and 24.1 points, respectively, directly demonstrating improved detection of environmental regions.

The discussion has helped us further clarify the task scope, annotation granularity, and applicability of existing benchmarks. We have responded to the concerns point by point and understand your expectations for more comprehensive validation. We respectfully ask you to reconsider your overall assessment of the paper in light of the revised manuscript and our responses, taking into account the task’s value, the benchmark contribution, and the empirical evidence. Your feedback and consideration would mean a great deal to us. Thank you for your time and effort.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.