- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SPHERETRUST, an unsupervised segmentation method that uses SAM/SAM3 and an MLLM(QWEN-VL) as teachers and distills their capability into a student model built on a frozen DINOv2 backbone. Although SAM can output multiple candidate masks, its ability to score these masks is poor, which is a widely acknowledged limitation. The authors’ core idea is to treat the normalized DINOv2 patch features as directions on a unit hypersphere and, using hand-crafted priors, to attribute several types of mask errors to characteristic patterns of the induced feature partition (Fig. 1). Three label-free measures (angular contrast, spherical coverage, frame contact) are combined to rank the candidates, and the leading candidates are then used to train the student through candidate-set training and a cross-fitted second round. The method is validated on candidate selection over eight SAM/SAM3 pools and on downstream camouflaged, salient, dichotomous, and low-light segmentation benchmarks.
Critical strengths
- Interpreting the problems of masks through the hypersphere is a novel perspective. Mapping each common failure type of a candidate mask to a property of the feature partition (Fig. 1) offers a label-free and interpretable way to rank SAM’s candidates.
- The paper reports the data and conclusions of each experiment in detail.
Critical weaknesses
- The benefit of the claimed core innovation is not sufficiently demonstrated. According to the authors’ narrative, the core innovation is to interpret mask quality through hyperspherical geometry and to select among multiple candidate masks, thereby obtaining high-quality training data (SPHERETRUST ranking). However, the paper does not sufficiently demonstrate the benefit of this aspect, and the main source of gain appears to be the teacher–student training framework rather than this selection scheme. The authors themselves state (line 460, Sec. 4.2): “the total gain does not isolate the contribution of spherical geometry.” In the ablation study (Tab. 9), removing the sphere term from both rounds (i.e., the vMF prototype term Γ; the SPHERETRUST ranking R is still used) changes the weighted F by only +0.01 on SOD and −0.21 on DIS. Moreover, under the same complete training procedure, replacing the SPHERETRUST ranking with the size control or the adapted DSS rule leaves differences of only 0.2–1.0 points (Tab. 16).
- The narrative around the spherical scoring is obscure and does not match the title and abstract. The narrative around this innovation is rather hard to follow; in particular, the concept of “spherical scoring” is heavily wrapped in spherical terminology, which is very confusing. Judging from the title, the paper should be about a method that scores masks with the sphere. However, in the mask scoring section (Sec. 3.1), I could find almost nothing related to the “sphere” beyond L2-normalizing the patch features and computing inner products (cosine similarities). This is inconsistent with the impression, given by the title and abstract, that the sphere plays a key role in mask selection. I hope the authors can clarify this point.
Questions for authors
(Clarification) In the mask scoring of Sec. 3.1, which component relies on spherical geometry beyond L2 normalization and cosine similarity? (Clarification) Can the authors provide evidence that isolates the contribution of the spherical selection from that of the training framework, e.g., a variant in which the scoring does not use normalized directions, evaluated on both selection quality and downstream performance?
6 comments
Official Comment by Authors (Cont.)
3. Isolating the ranking from the training framework (W1). We agree that the rule comparison as published did not settle this, and Tab. 17 of the revision adds the two rows that do. The full-procedure column cannot separate the two stages, because candidate-set training and the cross-fitted second round are designed to recover from a poor ranking. Under fixed labels the ranking acts alone.
| Ranking rule used for the labels | COD | SOD | DIS |
|---|---|---|---|
| generator confidence | .7130 | .7598 | .4847 |
| size control | .7287 | .8009 | .5842 |
| no normalization | .7597 | .7905 | .5853 |
| adapted DSS | .7624 | .7918 | .5464 |
| SPHERETRUST | .7526 | .8144 | .6047 |
Fixed-label student quality under five ranking rules, weighted , three seeds. Tab. 17 of the revision.
The gap to the generator's own mask score, which is what a practitioner faces when no external selector is used, is 4.0, 5.5 and 12.0 points. The gap to the size control is 1.4 to 2.4, and to the adapted DSS rule 2.3 on SOD and 5.8 on DIS. Across the fifteen rule and regime pairs, with each regime's mean subtracted from both variables, selected Dice and the fixed-label student's weighted correlate at 0.94, against 0.64 under the complete procedure. These are observed correlations on fifteen points, not a fitted model. We would add that the training stage is the third contribution of the paper rather than a confounder, so the selector is measured where it acts alone, on fixed pools in Tabs. 1 and 7 and under fixed-label training here.
4. A correction the new runs forced. On camouflage our ranking is not ahead. The adapted DSS rule trains the better fixed-label student by 1.0 point, and once the prototype fit and the pseudo labels are reseeded along with the student the five rules fall within 0.3 points in no stable order. Appendix §F.3 and the new Tab. 19 of the revision report this. Tab. 18 covers the size control and the adapted DSS rule only, and its caption now says so, since under final-student-only reseeding the unnormalized rule is ahead of us on COD on the shared seed.
Thank you for the thorough response and for candidly reporting the new experimental results, which have resolved most of my concerns. Before finalizing my evaluation, I hope the authors could provide a brief analysis explaining: (1) why removing feature normalization consistently degrades selection quality across the pools, and (2) why SPHERETRUST trails two simpler rules (no normalization and adapted DSS) on downstream COD. Even brief hypotheses or empirical observations would be valuable here, especially for readers less familiar with the dataset characteristics or domain nuances. If the authors can include this discussion in their reply and incorporate it into the main text (e.g., Experiments or Limitations) or the Appendix, I would be happy to raise my rating.
Official Comment by Authors
We thank the reviewer for a careful and constructive reading. Both requested experiments have been run, both are now rows of the paper, and they also led us to correct a claim of our own. One note on numbering, since the revision inserts a new Tab. 4 in the main text and every appendix table moves up by one. Tab. 9 of the submission is Tab. 10, and Tab. 16 is Tab. 17. Numbers below are those of the revision.
1. What is spherical in the selection rule. The sphere is a modelling choice, which is to read direction and discard magnitude. Patch features are normalized to unit directions, μ(S) is the renormalized resultant of a set of them and therefore the maximum likelihood von Mises-Fisher mean direction, angular contrast is one minus the cosine similarity between the two mean directions a candidate induces, the appearance modes are k-means centroids on unit directions projected back to the sphere, and coverage assigns patches to modes by cosine. Frame contact is not of this kind. It measures contact with the outermost pixel ring of the image frame, so it is a spatial prior rather than a statistic of the directions, and the revision now says so in Sec. 3.1 and in the abstract, which describes two properties of the split together with a spatial prior. The claim behind the other two is that direction alone carries enough signal to rank mask quality, and point 2 is the experiment that tests it directly. We accept that Sec. 3.1 did not make the construction explicit, and the revision now opens with it, including ∥f^p−f^q∥2=2−2⟨f^p,f^q⟩ for unit vectors, so that no stronger geometric claim is implied than the one we test. What uses the sphere distributionally rather than directionally is the vMF prototype term Γ at training time, and the reviewer reads the training ablation, Tab. 10, correctly there. Removing Γ from both rounds changes weighted F by −0.65, +0.01 and −0.21 points on COD, SOD and DIS. We would only ask that Γ and R be kept separate, since the evidence under W1 concerns R.
2. Scoring without normalized directions (Q2). Following the reviewer's suggestion, we rebuilt F and C in raw Euclidean geometry, with arithmetic means replacing renormalized mean directions, squared Euclidean distance replacing the cosine distance in F, and k-means in raw space. The frame-contact term E uses no features and is unchanged. The within-image standardization is kept so that scale is not confounded, and both arms score the same masks against the same ground truth under the same eligibility filter and the same native-resolution Dice evaluation. Removing feature normalization lowers selected Dice and Top-1 and raises the catastrophic-error rate on all eight pools.
Change from removing the normalization, in points. Appendix §H.1 of the revision.
Downstream the variant trains students at .7597, .7905 and .5853 of fixed-label weighted F against our .7526, .8144 and .6047, so the normalization is worth 2.4 points on salient objects and 1.9 on dichotomous segmentation under fixed labels, and 1.2 and 0.4 under the complete procedure, leading on every seed in both columns. On camouflage it loses 0.7, which is the same regime exception as in point 4. The variant is now a row of Tab. 17 and a section of the new Appendix §H.1, which states its construction in full.