- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes E3-RIS, a referring image segmentation framework that separates target identification, object-extent recovery, and the decision of whether a boundary correction should be accepted. PARE estimates referential evidence, ECBP uses proposals from a frozen SAM model to make constrained local mask edits, and BTRA learns to judge the reliability of these edits. Experiments on referring segmentation benchmarks show improvements in mask quality, particularly at high IoU thresholds, together with a series of component and controlled comparisons.
Critical strengths
The paper addresses a meaningful problem in referring segmentation. Evidence that identifies the correct object does not necessarily describe its complete spatial extent, and separating these two roles provides a reasonable motivation for the proposed framework.
The method is carefully evaluated. In addition to standard ablations, the authors compare constrained editing with unrestricted replacement, include matched-capacity and matched-coverage controls, and study failures caused by unreliable mask changes. These experiments provide useful support for the boundary-editing design rather than attributing all improvements to additional model capacity.
The improvement in high-overlap segmentation is encouraging. On RefCOCO val, the B-scale model improves [email protected] from 40.1% to about 44.2%, while mIoU increases from 77.8% to about 78.5%. BTRA also outperforms the matched-capacity generic refiner in [email protected] (44.25% versus 42.94%; Table 4), suggesting that learning when to accept an edit can be useful.
Critical weaknesses
-
The practical accuracy-efficiency trade-off is not sufficiently established.
The reported gains, especially at [email protected], are valuable, but the complete system also introduces considerable inference overhead. Under the reported B-scale setting, latency increases from 49.56 ms for TALENT to 154.60 ms for E3-RIS, and peak memory increases from 0.80 GiB to 3.36 GiB (Table 13). This does not mean that the additional computation is unjustified, since accurate boundaries can be more important than mean IoU in some applications. However, the current evaluation does not clearly show which operating points offer the best balance between segmentation quality and computational cost. In addition, oIoU decreases on six of the nine reported splits for the B-scale comparison against TALENT (Table 1), indicating that the gains depend in part on the evaluation metric. A comparison of accuracy, high-overlap success rate, latency, and memory across reduced-cost configurations would make the practical contribution clearer.
-
The additional benefit of the full BTRA ensemble needs more direct validation.
The existing ablations do support the proposed editing and arbitration mechanisms, so the concern is not a lack of component studies. However, relative to ECBP, adding BTRA increases mIoU by only 0.22 points on RefCOCO val (78.23% to 78.45%) while increasing [email protected] by 1.62 points (42.63% to 44.25%). At the same time, latency rises from 101.70 ms to 154.60 ms (Table 13). These results suggest that BTRA has a particular advantage for highly accurate masks, but they do not show whether all three arbiters are needed to obtain that benefit. It would be useful to compare the current ensemble with a single arbiter or a lighter arbitration design under the same evaluation protocol.
Overall, the paper presents a well-motivated framework with convincing evidence for safer boundary editing and better high-overlap masks. My main reservation is whether the complete inference pipeline is necessary to achieve these benefits at a practical computational cost. A stronger efficiency-quality comparison, particularly with a reduced-cost BTRA configuration, could change my assessment.
Questions for authors
- Could the authors report an accuracy-latency-memory comparison across lighter E3-RIS variants, including the effect on [email protected]?
- How much of BTRA's high-overlap improvement remains when using one arbiter instead of the three-member ensemble, and what is the corresponding latency?
- Could the authors explain the oIoU decreases in the B-scale TALENT comparison and clarify which types of examples benefit most from the boundary-editing process?