- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
UniOSE addresses open-ended object detection and segmentation without a user-supplied category vocabulary. It combines detector queries and regional detector features with features from a pretrained vision–language encoder to construct semantics-enriched object queries. These queries condition category-name generation, provide semantic costs for training-time detection matching, and condition a SAM-family mask decoder. Validation includes LVIS minival detection, five cross-dataset semantic segmentation settings, component ablations, and diagnostic analyses. The method reports improvements of 5.0 and 3.6 detection AP over the corresponding Open-Det configurations, although several comparisons and diagnostics do not fully isolate the proposed mechanisms.
Critical strengths
-
A coherent integration of naming and spatial prediction. Reusing a pre-generation object representation for naming, supervision assignment, and mask conditioning provides a concrete extension of detector-based open-ended recognition. The design preserves object correspondence and avoids requiring generated names to be re-encoded before segmentation.
-
Promising empirical detection performance. The reported gains over Open-Det are substantial, and the VLM-branch ablation shows a large improvement within the tested configuration. Strong results on several segmentation settings further suggest that the resulting object representations transfer beyond detection. These findings make the approach worth considering despite the validation concerns.
Critical weaknesses
-
The principal comparisons do not sufficiently isolate the contribution. The Open-Det comparison changes the backbone and introduces a separate VLM image encoder, while confidence scoring has a substantial effect on reported AP. Appendix B.2 shows that changing the fusion weight from 0.5 to 0.2 improves UniOSE by 4.23/5.39 AP, comparable to or greater than the headline gains. The weight is selected on the same LVIS minival set used for reporting, and equivalent scoring treatment of the baselines is not established. Consequently, the results demonstrate promising system performance but do not clearly attribute the gains to the proposed mechanisms.
-
The central diagnostic evidence does not cleanly substantiate the claimed semantic benefits. Figure 4(a) compares absolute similarities in different native embedding spaces and measures positive-category alignment without assessing separation from incorrect categories. Figure 4(b) changes both the training matcher and the offline evaluation matcher, conflating prediction improvements with assignment effects. Meanwhile, semantic matching contributes only 0.8 AP with the VLM branch, and mask conditioning improves mIoU by 0.3/0.4/1.1 points without reported uncertainty. These limitations leave the central claim—that enriched naming semantics reliably improve spatial prediction—only partially supported.
Questions for authors
The following are clarification requests and requests for evidence relevant to the current claims, rather than suggestions for future work.
-
Was the confidence-fusion weight selected using an independent validation set? Were the Open-Det baselines evaluated with the same category mapper, suppression settings, and equivalent fusion-weight tuning? Please provide comparisons under a common scoring protocol.
-
What performance does an Open-Det-style baseline achieve with the same SAM2 backbone, EVA02 text encoder, training schedule, and evaluation settings? Does the gain persist when these factors are held constant?
-
Does the Joint@τ improvement persist when both checkpoints are evaluated using the same semantic-independent assignment rule? Can the alignment analysis report category retrieval accuracy or positive-versus-negative margins rather than comparing absolute similarities across embedding spaces?
-
Were bbox-only, random-query, and empty-query variants independently trained with equivalent adaptation and LoRA settings, or were queries replaced only at inference? Are the small improvements over bbox-only consistent across repeated runs?