- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes Boundary Adaptation (BA), a sparse mask-refinement policy that iteratively reclassifies boundary pixels. Boundary-SAM implements it through native-resolution pointwise decoding using frozen HQ-SAM features. Topology-aware updates, heuristic artifact removal, and multiscale processing are used to stabilize refinement. Experiments report competitive results on natural-image benchmarks, with benefits for fine structures and small objects, plus remote-sensing and 2D CT-slice results after domain-specific retraining.
Critical strengths
- BA provides a simple, well-motivated refinement policy that targets boundary errors in high-quality coarse masks. The oracle analysis and iterative-update/multiscale ablations provide useful empirical support for the refinement procedure, making its practical design a substantive strength.
- The method demonstrates useful improvements for fine structures and small objects. On ThinObjects-5K, it improves over the input SAM masks by 3.78 mIoU and 6.72 bIoU points. The size-stratified experiment in Figure 6 and native-resolution ablation in supplementary Table 7 further support its value for small objects in high-resolution images.
Critical weaknesses
- The theoretical reduction to independent pixel classification is not justified as stated. In Proposition 1, Eq. (6) omits interactions between boundary pixels, although the candidate set permits both labels to vary; these terms generally prevent the factorization in Eq. (7). Moreover, minimizing energy requires maximizing the defined ratio, whereas Appendix A minimizes it. The four-pass update schedule in Section 4.1 may support a restricted formulation under suitable interaction assumptions, but these assumptions are not stated in the proposition. Consequently, the claimed MAP interpretation requires correction.
- The experiments do not fully isolate the pixel-selection policy, which is the paper’s stated core contribution. The BA-versus-entropy comparison uses different training-mask distributions. Supplementary Table 8 provides useful boundary-band evidence, but does not report matched query budgets or clearly specify training adaptations for band selection. The CRM-based patch comparison in Table 9 also changes the decoding formulation, and its failure is attributed to missing SAM context-token conditioning. These limitations leave uncertainty about BA’s advantage independently of training and decoder effects.
Questions for authors
- Clarification of the theory: Is Proposition 1 intended to apply to single-pixel or noninteracting-subset updates, such as the four-pass schedule in Section 4.1, rather than simultaneous optimization over the full boundary? Please state the required interaction assumptions, account for boundary–boundary terms in Eqs. (5)–(7), and clarify whether the defined energy ratio should be maximized.
- Assessment-critical empirical comparison: Does BA retain its advantage over entropy and boundary-band selection using the same B-SAM decoder, matched training-mask distributions, and comparable total pixel-query budgets, including repeated queries? Please clarify which controls already hold in Section 5.3 and supplementary Table 8, and provide a controlled comparison if these conditions are not currently matched.
W1 & Q1. We thank the reviewer for raising this important point. Indeed, Proposition 1 applies only to noninteracting-subset updates, such as the four-pass schedule in Section 4.1. We have revised Section A so boundary updates are executed on a subset of pixels according to their color class. This will remove the coupling between boundary terms in Eq 6, which makes the canceling of terms valid.
W2 & Q2. To more directly compare the selection policy, we include the Entropy-based Oracle method and the Entropy-based B-SAM method in Tables 1, 2, 5, and 6. The Entropy Oracle method queries the same number of points as the BA Oracle method, and the Entropy B-SAM method queries the same number of points as the BA B-SAM method. Note too that Entropy B-SAM uses the same B-SAM architecture as BA B-SAM.
However, the training procedure for BA B-SAM and Entropy B-SAM is necessarily different. BA B-SAM is trained on synthetically deformed ground-truth masks, whereas Entropy B-SAM is trained directly on coarse SAM masks and logits because Entropy-based selection requires access to the model's predicted probabilities. Importantly, both methods are evaluated on coarse SAM masks in Tables 1 and 2. Thus, the training distributions are not matched, and this difference potentially favors Entropy B-SAM, as its training data more closely resembles the coarse-mask inputs encountered during evaluation. Our training distribution of BA B-SAM is selected to match methods designed for refinement like CPSP and CRM.
In Table 10, the banded update experiment uses the same number of queries as our standard network (including repeated queries), the same B-SAM decoder, and the same training-mask distribution. These controls are already present in the originally submitted experiments.