- Rating
- 3
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces sub‑semantic image segmentation, a task to discover appearance‑driven visual regions and output disjoint masks paired with free‑form textual descriptions without pre‑defined semantic labels.
The authors propose Detecture, a frozen two‑backbone model based on Qwen3‑VL‑8B and SAM3. Key designs include structured autonomous generation with [SEG] markers, block‑diagonal attention for isolated grounding, a lightweight Bridge projection, competitive pixel‑wise assignment with a residual channel, and a Shifted‑Zero loss for training stability.
Three new benchmarks plus an external texture dataset are built for evaluation. Results show competitive performance against existing language‑mask models, and controlled interventions validate that text guides mask assignment. Nonetheless, concerns remain regarding dataset annotation quality, missing ablations for critical hyper‑parameters, and limitations of the teacher‑forced intervention setup.
Critical strengths
- The task is well motivated and unusually carefully specified: this work defines the novel sub-semantic image segmentation task, focusing on appearance-based region partitioning within the same semantic category, and establishes six clear task design principles.
- The authors build three complementary benchmarks. TextureADE (annotation-mined natural scenes, variable cardinality), RWTD-COCO (independently constructed natural two-surface transitions), CSTD (controlled synthetic compositions), plus external RWTD.Controlled intervention experiments are carefully designed to verify that text descriptions actively guide mask generation rather than being post-hoc explanations.
- The proposed Detecture introduces block-diagonal attention for isolated grounding, lightweight Bridge projection, competitive pixel assignment and Shifted-Zero language loss, enabling frozen two-backbone (VLM + segmentation foundation model) to output disjoint masks with free-form descriptions.
Critical weaknesses
- The ground-truth masks of the proposed benchmarks are largely inherited from legacy semantic segmentation datasets instead of re-annotated specifically for sub-semantic appearance boundaries. There lacks human evaluation to verify the correctness of text-mask pairing.
- The method called "Qwen3-VL + SAM 3 semantic head" in Table 22 is significantly better on RWTD and CSTD, statistically unresolved on RWTD-COCO, and worse only on TextureADE. Since this baseline is both directly relevant and strong, it should appear in the main comparison.
- The source-data exposure of the compared systems is asymmetric and not fully characterized. The paper correctly notes that source identifiers cannot establish absence from all pretraining or development data. This issue should be discussed alongside the corresponding table rather than only in the appendix(p.40).
Questions for authors
- Since TextureADE and RWTD-COCO largely inherit masks from legacy semantic annotations, could the authors provide a dedicated human evaluation.
- Could the authors include Qwen3-VL + SAM 3 semantic head in the main comparison table? If the authors believe it is not directly comparable, please clearly explain the task or protocol distinction.
- Could the authors more explicitly characterize and discuss source-data exposure for all compared systems alongside the main results?