- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies the adversarial robustness of SAM3 for text-prompted concept segmentation. The authors point out that SAM3 multiplies every candidate's query score by a single prompt-level presence probability, so the presence token acts as a shared gate over all masks, and they argue this is a new attack surface. They propose Universal Concept Disruption (UCD), a single universal perturbation trained on (image, noun phrase) pairs. The objective has four terms: a top- hinge loss on the final presence-gated scores, a mask area loss, a Dice loss against the clean masks, and a feature divergence term on the last FPN feature map of the image backbone. With , UCD lowers average mask AP from 59.43 to 18.73 and cgF1 from 50.32 to 20.49 across five benchmarks (SA-Co Gold, LVIS, RefCOCO, PhraseCut, OpenImages), while baselines adapted from SAM/SAM2/classifier attacks barely change performance. The paper also reports transfer to SAM3.1 and to SAM3 video inference, and evaluates prompt ensembling, head fine-tuning, and temporal filtering as defenses.
Critical strengths
- As far as I know, this is the first systematic study of adversarial robustness for SAM3. Since SAM3 is likely to be used as a backbone or a labeling tool in many pipelines, the topic is timely.
- Looking at the presence token as a possible single point of failure is an interesting angle. It is specific to SAM3's design and not something you get by simply rerunning SAM/SAM2 attacks.
- The attack is strong in absolute terms and works across quite different prompt types (categories, referring expressions, phrases) even though it is trained on only 100 images from LVIS and SA-Co Silver.
- The image-to-video result, where perturbing only early frames still degrades later clean frames through memory, is a nice extra observation, and the authors did make an effort to include defenses rather than only reporting the attack.
- The paper is generally easy to follow, and the loss terms are clearly defined.
Critical weaknesses
1. The main claim about the presence gate is not really supported by the experiments. The title and the introduction frame the presence gate as the key attack surface, but in Table 2a, attacking only the presence score gives 57.34 mask AP, which is almost the same as clean (59.43). Only the product of query and presence works (18.73). The paper also never shows what actually happens to under the full attack. The qualitative figures report top score drops like , but I can't tell whether this comes from the presence probability collapsing or from the query logits. To support the claim, I think the authors should (a) report how and change between clean and adversarial inputs, and (b) evaluate the same perturbation on SAM3 with the presence gate removed at inference ( only). If the attack is still about as effective without the gate, then the presence gate is not the main reason the attack works.
2. Most of the attack strength seems to come from the generic feature divergence term, and the novelty of the method is limited. Removing the feature term gives 57.45 mask AP (Table 2d), which is basically at the level of the baselines. So the score, area, and Dice losses together, which are the parts described as SAM3-specific, do almost nothing on their own. Each of these components also has clear prior counterparts: maximizing feature distance is a standard way to improve transferability, the top- score hinge is close to object-vanishing attacks on detectors, and shrinking mask area is similar to the goal of DarkSAM. The only part I see as truly SAM3-specific is using the product score in . Because of this, the method reads more like a well-tuned combination of existing losses than a new attack design. A feature-only result is needed to see how much the SAM3-specific losses really add on top.
3. The ablations are all leave-one-out from the full UCD, which makes it hard to interpret the contribution of each component. The interactions between terms seem very strong. Using the numbers in Table 2d, removing each loss recovers 18.12 (score), 27.55 (area), 12.28 (Dice), and 38.72 (feature) mask AP points relative to full UCD (18.73). These add up to about 96.7 points, which is more than twice the total drop from clean to full UCD (). If the terms contributed independently, these two numbers should be roughly similar. So the leave-one-out numbers only tell us that each term is necessary when all the others are present, not how much each term contributes by itself. There is also an inconsistency between panels. If, as the caption says, all other settings are kept as in full UCD, then "presence only" (57.34) and "w/o score" (36.85) differ only by adding a presence-based score loss, and adding it makes the attack 20 points weaker. Similarly, "query only" (37.45) is about the same as having no score loss at all. I would like to see ablations that build up from the bottom: clean SAM3, a random-noise perturbation of the same budget (), each loss applied alone ( only, only, only, only), and then terms added one at a time (e.g., ). Crossing this with the prompt type (point grid vs. text) would also help separate the effect of text-prompt training from the loss design. The authors should also clarify the exact loss configuration used for each variant in Table 2a and 2b.
4. I have concerns about the fairness of the baseline comparison, and some baseline results are hard to understand. All baselines use the same schedule as UCD (about 200 Adam steps with lr ), which was presumably chosen for UCD, and the paper does not explain how the baselines, most of which were designed for point/box prompts, were adapted to the text-prompt setting. More importantly, several baselines actually increase AP over the clean model. For example, DarkSAM goes from 34.34 to 35.72 on PhraseCut and from 75.85 to 76.59 on OpenImages, UAPGD goes from 57.30 to 58.09 on RefCOCO and to 77.09 on OpenImages, and S-RA is also above clean on RefCOCO, PhraseCut, and OpenImages. DarkSAM's average box AP (60.51) is also higher than clean (60.35). An optimized white-box universal perturbation at making the model better is strange, and it suggests either an optimization failure or a problem in how the baselines were adapted. I'd like the authors to explain this result, describe the adaptation of each baseline in detail (which prompts, which outputs and losses are used), and check whether the conclusions still hold when each baseline is run with its own tuned schedule (longer optimization, original hyperparameters).
5. All evaluations use the text-prompt path only. SAM3 also supports point/box interactive segmentation and exemplar prompts, and none of these are evaluated. This narrows what can be claimed about SAM3's robustness in general. It also connects to the previous point: the baselines were designed for geometric prompts, but they are only compared on the text path, which is not where they were meant to work. Results under point/box prompts for both UCD and the baselines would make the comparison fairer and the scope of the claims clearer.
Questions for authors
- Under the full UCD attack, how much do and the query probabilities each drop? Does the perturbation still work if the presence gate is removed at inference ()?
- What is the performance of alone, and of a random perturbation with the same budget?
- Could you explain why "presence only" (57.34) is much weaker than "w/o score" (36.85) in Table 2? Were the other loss terms kept on in Table 2a?
- How were DarkSAM, S-RA, UAPGD, and UAP-SAM2 adapted to SAM3's text-prompt setting, and why do some of them increase AP over the clean model?
- How does UCD compare to the baselines when SAM3 is prompted with points or boxes?