acceptodds
Under review as a conference paper at ICLR 2027

SAM3 for Cross-Domain Few-Shot Segmentation via Dual-Branch Candidate Calibration

11 views

Abstract

Cross-Domain Few-Shot Segmentation (CD-FSS) aims to segment novel categories in unseen domains from a few annotated support examples. Existing methods often rely on source-domain episodic training and remain sensitive to domain shifts. Motivated by advances in vision foundation models, we explore a fully frozen Segment Anything Model 3 (SAM3) for training-free CD-FSS. We observe that class-name text improves visual prompting in some domains but impairs it in others. To address this inconsistency, we propose a dual-branch candidate calibration framework. Support and query images are arranged on a shared canvas, where visual-only and visual-text branches generate complementary candidate masks. We assess candidate reliability using SAM3 confidence, support-query prototype consistency, and cross-branch agreement, then select and merge reliable masks into the final prediction. The framework requires no source-domain training, parameter updates, or additional learnable modules. Extensive experiments on diverse CD-FSS benchmarks demonstrate its effectiveness and superior average performance over the compared training-required and training-free methods.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Few-Shot Segmentation, Cross-Domain Few-Shot Segmentation, Segment Anything Model 3

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_uXZUOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies cross-domain few-shot segmentation (CD-FSS), where source and target domains differ in both data distribution and label space, and no source-domain training data are available; no model parameters are updated. The authors propose a training-free framework that directly reuses a fully frozen SAM3, a foundation segmentation model supporting "promptable concept segmentation" with text or visual-exemplar prompts. The method has three components:

  1. Shared spatial canvas: the support and query images are composed into one canvas, so SAM3 can model cross-image correspondence within a single forward pass via its native attention. The support mask is converted to a tight box and mapped to canvas coordinates as a visual prompt.
  2. Dual-branch candidate generation: a visual-only branch uses only the support box; a visual-text branch additionally uses the class name as a text prompt. Candidates from both branches are merged into a unified candidate pool.
  3. Candidate reliability calibration: each candidate is scored by a weighted fusion of (i) SAM3's own confidence, (ii) support–query prototype consistency (cosine similarity of the candidate's regional feature to support foreground vs. background prototypes in the frozen encoder's feature space), and (iii) cross-branch agreement (max IoU with candidates from the other branch). Candidates above threshold δ are retained and merged by pixel-wise union. Validation: on DeepGlobe, ISIC, Chest X-Ray, and FSS-1000, the method reports 71.56% (1-shot) and 74.60% (5-shot) average mIoU, outperforming the listed training-required CD-FSS methods and several SAM-based training-free methods. Ablations cover canvas layout, calibration terms, threshold, fusion weights, negative prompting, and domain-aware text prompts, plus qualitative comparisons, failure cases, inference speed (7.5–9.8 FPS), and pseudocode.

the novelty is primarily a system-level combination—"frozen SAM3 + dual-branch candidate calibration"—rather than a new learning mechanism; this matters for assessing the contribution.

Critical strengths
  1. The problem formulation and the method are cleanly matched, with practical value. Rather than training a cross-domain adapter, the authors exploit SAM3's concept segmentation and text/visual-exemplar prompting to recast CD-FSS as spatial reasoning on a shared canvas, achieving the best average 1-shot/5-shot results in Table 1 (71.56/74.60) with zero source-domain data and zero parameter updates. This is meaningful for settings where source data are unavailable or compute is limited, and it challenges the default assumption that CD-FSS must be trained.
  2. The dual-branch + three-term calibration design is clearly motivated and supported by ablations. The authors first use a preliminary analysis (Fig. 1(a)) to show the domain-dependence of text prompting and a "prompt conflict" effect, then design complementary branches accordingly. Table 3 shows prototype consistency and cross-branch agreement each help (70.16→71.15 / 71.02) and their combination is best (71.56); Fig. 4 shows the dual branch is never worse than either single branch across domains; Figs. 5–6 provide qualitative support.
Critical weaknesses
  1. The attribution of the core gain is not clean, and the necessity of the dual branch is not isolated. Fig. 4 only shows bar plots of average/per-domain mIoU, with no per-domain numeric table; and the Table 3 calibration ablation appears to be run on the dual-branch pool. A reader cannot tell how much performance would drop if (a) candidates from the visual-text branch were fed into the same calibration and threshold (a single-branch-generation + dual-branch-calibration mismatch), or (b) the dual branch were used without prototype/agreement calibration. If the dual-branch gain mainly comes from "a larger candidate pool + calibration picking the better one," the contribution is closer to candidate re-ranking than to complementary prompting, which would materially affect the novelty judgment.
  2. The negative-prompt conclusion may be confounded by the experimental construction and is in tension with the paper's own motivation. Table 5 shows performance degrading monotonically when negative prompts are added (71.56→66.63→63.14→60.90), attributed to "SAM3 being sensitive to negative guidance" and supported by the t-SNE in Fig. 6. But per Sec. 5.4, negative prompts are built by running SAM3 on the support image with a positive box and converting low-confidence candidates into negative boxes—those low-confidence candidates are themselves likely noise or background fragments, so using them as negative prompts injects erroneous spatial priors. The experiment therefore more plausibly shows "this automatic way of constructing negatives is unreliable" than "SAM3 should not use negative prompts." The claim is currently too strong; please weaken it.
Questions for authors
  1. Fig. 4 reports visual-only 64.95, visual-text 62.24, and dual-branch 71.56 average mIoU. Table 3(a) reports a no-calibration baseline of 70.16. Please clarify: (a) were the single-branch numbers in Fig. 4 obtained with the same reliability calibration (prototype consistency + cross-branch agreement + δ) as the dual-branch number, or without it? (b) Does the 70.16 baseline correspond to the dual-branch candidate pool scored by SAM3 confidence alone? This determines whether the dual-branch gain comes from complementary candidates or from calibration operating over a larger pool.
  2. Sec. 5.4 states that negative prompts are obtained by converting low-confidence candidates from the support image into negative boxes. Please specify the criterion for "low-confidence" (threshold or count) and whether such candidates may themselves be background/noise. If negatives were instead drawn from confident background regions outside the support mask, would the monotonic degradation in Table 5 persist? This affects whether the conclusion is "SAM3 is sensitive to negative guidance" or "this particular construction is unreliable."
  3. Please clarify the irreplaceable component of this work relative to the training-free baselines already in Table 1 (Matcher, CPS, GF-SAM), given that the shared-canvas idea is explicitly credited to Tsai et al. (2026).
  4. Please report the fraction of episodes in which no candidate passes δ and an empty mask is returned on each of the four datasets. Table 4 indicates that the 5-shot setting uses the same α=1.0, β=0.2, γ=0.1; please confirm that δ=0.5 is also retained for 5-shot and briefly justify why no shot-specific adjustment is needed.
2 comments

We thank the reviewer for the detailed assessment and for highlighting the clear motivation and supporting ablations of our dual-branch design. Your comments help us clarify the sources of performance gains, refine the interpretation of the negative-prompt experiments, and better define our contribution. We respond to each point below.

Response:

Q1:(a) The single-branch results in Fig. 4 use SAM3 confidence and prototype consistency with the same fixed selection threshold, δ = 0.5. Cross-branch agreement is omitted because it is undefined for a single branch. The dual-branch result additionally includes cross-branch agreement. Thus, Fig. 4 does not isolate candidate-generation effects under identical scoring terms. (b) The 70.16% result in Table 3(a) corresponds to the combined visual-only and visual–text candidate pool filtered using SAM3 confidence alone, with δ = 0.5. Applying full calibration to this same pool increases average mIoU to 71.56%. This comparison isolates the contribution of calibration within the dual-branch pool, whereas Fig. 4 compares the overall branch configurations.

Q2: In our implementation, “low-confidence” candidates are defined as support-image predictions with scores in the interval 0.1 < s ≤ 0.5. Candidates whose intersection with any positive candidate box exceeds 5% of their own area are removed. The remaining boxes are sorted by score, filtered using non-maximum suppression with an IoU threshold of 0.5, and capped at the specified maximum number of negative prompts. These candidates may contain background, noise, or unrecognized foreground regions. We tested negative boxes sampled from annotated support-background regions. As shown in Table 8, increasing their maximum number from one to three reduces average mIoU from 63.60% to 57.12%, with declines across all four domains. Thus, the degradation is not limited to low-confidence negatives and supports sensitivity to negative guidance under our evaluated setting.

Table 8. Comparison of negative prompts derived from low-confidence candidates and annotated support-background regions. Results are reported as mIoU (%).

Negative sourceMax negativesDeepGlobeISICChest X-rayFSS-1000Average
Low-confidence candidates≤ 141.1562.8379.3183.2266.63
Low-confidence candidates≤ 336.3954.8272.4179.9660.90
Annotated background≤ 139.1457.2175.7882.2663.60
Annotated background≤ 334.4949.2367.1777.5757.12

Q3: Our contribution couples complementary visual-only and visual–text candidate generation with support-conditioned candidate selection. The two branches expand the candidate pool, while selection combines SAM3 confidence, foreground–background prototype consistency, and cross-branch agreement to retain candidates matching the support-defined target. Matcher uses feature matching and instance-level selection, CPS adapts matched-point density, and GF-SAM employs graph clustering and gating. Our distinction lies in jointly exploiting complementary prompting routes and candidate reliability, with their contributions supported by Tables 5 and 6.

Table 5. Threshold analysis of candidate pools and calibration variants under the 1-shot setting. Results are reported as mIoU (%).

Candidate poolPrototypeCross-branchδ = 0.3δ = 0.4δ = 0.5δ = 0.6
Visual-only + visual-text––68.8569.4570.1669.91
Visual-only + visual-text✓–69.9570.6171.1570.96
Visual-only + visual-text–✓69.7270.3871.0270.67
Visual-only + visual-text✓✓70.3170.9871.5671.24
V-Top + V-Bottom✓✓66.7167.2567.8667.51

Table 6. Decomposition of 1-shot mIoU (%) across three matched episode seeds. Results are reported as mean ± standard deviation.

ConfigurationDeepGlobeISICChest X-rayFSS-1000Average
Visual-only, confidence-only26.18 ± 0.2562.97 ± 0.1980.43 ± 0.2584.62 ± 0.1863.55 ± 0.21
Dual-branch, confidence-only44.59 ± 0.2464.13 ± 0.2383.25 ± 0.1988.67 ± 0.1670.16 ± 0.21
Dual-branch, full calibration46.05 ± 0.2165.55 ± 0.1884.60 ± 0.2290.04 ± 0.1571.56 ± 0.19

Table 5 demonstrates the benefits of complementary prompting and individual calibration components, while Table 6 separates the gains from dual-branch candidate generation and full calibration, reporting means and standard deviations across three matched episode seeds on four datasets.

Q4: The fractions of successfully evaluated episodes in which no candidate passes δ and an empty mask is returned are 10.21% for DeepGlobe, 7.17% for ISIC, 4.00% for Chest X-ray, and 1.96% for FSS-1000. For 5-shot evaluation, we retain δ = 0.5 together with α = 1.0, β = 0.2, and γ = 0.1. The support prototypes are averaged and re-normalized, and the candidate-scoring formula remains unchanged, so increasing the support count does not change the nominal score range. We therefore use the same threshold as a consistent evaluation setting, rather than introducing shot-specific tuning.

Reviewer_qF9WOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies training-free cross-domain few-shot segmentation using a fully frozen SAM3. Support and query images are composed into a shared canvas, with bounding boxes derived from the support annotations serving as visual prompts. Two prompting configurations—visual-only and visual-text—produce complementary candidate masks. The method scores these candidates using a weighted combination of SAM3 confidence, support foreground/background prototype consistency, and cross-branch mask agreement, then takes the union of candidates exceeding a threshold.

Critical strengths
  1. The framework requires no task-specific parameter updates or additional trainable modules. Table 1 reports strong average performance across natural images, remote sensing, and medical imaging, with particularly substantial results on ISIC. Although the method is not the strongest on every dataset, its overall performance suggests practical value for applying a pretrained segmentation model under limited target-domain supervision.
  2. The proposed use of complementary prompting routes has relevant supporting evidence. Figure 4 supports the observation that the two prompting configurations have different domain preferences: visual-text prompting performs better on DeepGlobe and FSS-1000, whereas visual-only prompting performs better on ISIC and Chest X-Ray. The full method exceeds either individual branch on all four datasets in this comparison. Table 3(a) also reports improvements from each auxiliary reliability term and from their combination.
Critical weaknesses
  1. Tsai et al., Few-Shot Semantic Segmentation Meets SAM3, already introduced a fully frozen SAM3 framework with a shared support-query canvas and class-name textual priors. The current paper appropriately cites this work, but the main comparison does not explicitly evaluate it. The principal methodological addition therefore lies in pooling candidates from the two prompting configurations and applying the proposed reliability score, together with its validation on cross-domain benchmarks. A more precise comparison with this closest predecessor is necessary to establish the significance of that addition.

  2. I acknowledge that Figure 4 provides same-backbone single-branch comparisons and that Table 3(a) includes a confidence-only candidate-selection baseline. However, the latter already achieves 70.16 average mIoU, compared with 71.56 for the full method. Thus, the two auxiliary calibration signals contribute 1.40 percentage points in the reported ablation. This improvement is potentially useful, but its per-domain consistency and uncertainty are not reported, and the relationship between the Figure 4 baselines and the closest prior method is insufficiently specified.

  3. With the reported weights and threshold, Eq. (11) retains a candidate when \bar{s}_i > 0.65 - 0.2r_i^{\mathrm{proto}} - 0.1r_i^{\mathrm{agree}}, whereas the confidence-only configuration retains candidates when (\bar{s}_i>0.5). This does not invalidate the ablation, but it makes an independently tuned confidence-only threshold an important control. Without that comparison, it is difficult to assess how much of the additional gain comes from informative candidate-specific reliability signals versus a different acceptance operating point.

  4. Section 5.4 adopts the 1-shot canvas layout, selection threshold, and fusion weights according to their reported performance across the target datasets. Appendix A.4 similarly states that the 5-shot inverse-L layout was chosen because it achieved the highest average mIoU. However, the paper does not identify an independent validation set or explain how the data used for these choices were separated from the final evaluation. Consequently, it is unclear whether the reported results evaluate a configuration fixed independently of the target test labels.

  5. Although the appendix specifies the 5-shot canvas arrangement and the use of five support boxes, the prototype-calibration equations and Algorithm 1 describe the 1-shot case. The aggregation of foreground and background prototypes across five support examples is not defined. Different choices, such as averaging per-image prototypes or pooling all support pixels, can produce different candidate scores.

Questions for authors
  1. What is the exact relationship between the single-branch baselines and the closest prior SAM3 method? Do the visual-only and visual-text results in Figure 4 reproduce the corresponding configurations of Tsai et al. under matched checkpoints, canvas construction, prompt information, and candidate postprocessing? Which components differ? Please clarify whether the branches receive the same applicable prototype-based filtering and comparable threshold selection.

  2. Which data were used for model selection, and how were they separated from final evaluation?Please identify the samples, classes, or episodes used to choose the canvas layouts, (\alpha,\beta,\gamma,\delta), and prompt variants. Were these choices fixed using independent validation data before evaluation on the four target test sets? Please also specify the target splits, episode counts, sampling procedure, and provenance of the baseline results.

  3. Does the calibration improvement remain after independently tuning the confidence-only baseline? Using the same independent validation protocol, how does the full method compare with the confidence-only dual-branch baseline when each is allowed its own selection threshold? Please provide per-dataset results and uncertainty over repeated, matched episode samples.

  4. How is prototype calibration extended to five support examples? How are the five foreground prototypes and five background prototypes combined when calculating candidate consistency? Are prototypes averaged equally across support images, weighted by foreground/background area, or used to score candidates separately before aggregation?

5 comments

We appreciate the reviewer’s careful examination of our method and its relationship to prior SAM3 work. Your questions about the comparison protocol and candidate calibration are valuable, and we address them point by point below.

Response:

Q1:Our single-branch baselines build on Tsai et al.’s frozen-SAM3 framework and shared support–query canvas, but add prototype-based candidate calibration. Accordingly, the results in Fig. 4 are within-framework controls on our CD-FSS episodes, rather than numerical reproductions of Tsai et al.’s results; the evaluation datasets and candidate postprocessing differ. Within our comparison, the visual-only and visual-text branches use the same canvas and support-derived box, while only the visual-text branch additionally receives the class name. Both branches apply the same SAM3-confidence and prototype-based candidate filtering with a fixed threshold of (q>0.5), without branch-specific tuning. Cross-branch agreement is not used because it is inapplicable to a single branch.

Q2: No independent validation set was used: the canvas layouts, (\alpha,\beta,\gamma,\delta), and prompt variants were compared on the same target data used for reporting performance. We therefore regard these results as exploratory benchmark comparisons. Evaluation was conducted using random seeds 0, 123, and 210, with 600 episodes per seed for each of DeepGlobe, ISIC, and Chest X-ray. Classes were selected cyclically for DeepGlobe and ISIC, while Chest X-ray contains a single class. Query and support images were sampled randomly within each class, excluding identical support–query images. FSS-1000 used the 240 classes, with each available image serving once as a query per seed and support images sampled randomly from the same class. Results for the compared methods are taken directly from their respective cited publications.

Q3: Table 5 compares both methods over the same threshold grid, allowing each its own best threshold within the tested range. Both achieve their best aggregate performance at δ = 0.5, where the full method outperforms the confidence-only dual-branch baseline. Table 6 reports per-dataset means and standard deviations across three matched episode seeds, showing consistent improvements on all four datasets. These results support the benefit of the full method under the evaluated settings. For convenience, we reproduce both tables below.

Table 5. Threshold analysis of candidate pools and calibration variants under the 1-shot setting. Results are reported as mIoU (%).

Candidate poolPrototypeCross-branchδ = 0.3δ = 0.4δ = 0.5δ = 0.6
Visual-only + visual-text––68.8569.4570.1669.91
Visual-only + visual-text✓–69.9570.6171.1570.96
Visual-only + visual-text–✓69.7270.3871.0270.67
Visual-only + visual-text✓✓70.3170.9871.5671.24
V-Top + V-Bottom✓✓66.7167.2567.8667.51

Table 6. Decomposition of 1-shot mIoU (%) across three matched episode seeds. Results are reported as mean ± standard deviation.

ConfigurationDeepGlobeISICChest X-rayFSS-1000Average
Visual-only, confidence-only26.18 ± 0.2562.97 ± 0.1980.43 ± 0.2584.62 ± 0.1863.55 ± 0.21
Dual-branch, confidence-only44.59 ± 0.2464.13 ± 0.2383.25 ± 0.1988.67 ± 0.1670.16 ± 0.21
Dual-branch, full calibration46.05 ± 0.2165.55 ± 0.1884.60 ± 0.2290.04 ± 0.1571.56 ± 0.19

Q4: In the 5-shot setting, we compute foreground and background prototypes separately for each support image using masked average pooling, followed by L2 normalization. The five foreground prototypes are then averaged with equal weights and L2-normalized again to form a single foreground prototype; the background prototypes are aggregated in the same way. Each candidate is scored using the difference between its cosine similarities to these aggregated foreground and background prototypes, following the same scoring rule as in the 1-shot setting. Thus, aggregation is performed at the prototype level, without foreground/background area weighting or separate per-support candidate scoring. If a support region is empty at the feature-map resolution, its prototype is excluded from the corresponding average.

Reviewer_qF9WOpenReview ↗

Feedback

Thank you for clarifying the 5-shot prototype aggregation. This answers my implementation question. The threshold sweep, two-pass visual-only control, and matched-seed results are also helpful. My main concerns, however, remain the contribution beyond the closest prior method and the evaluation protocol used to support it.

  1. Tsai et al. already introduced the frozen-SAM3 framework, shared support–query canvas, and class-name textual priors. What this paper adds is dual-branch candidate pooling and reliability-based selection. A direct comparison is needed to judge the contribution of these additions. The authors' response makes clear that the single-branch baselines in Figure 4 include your prototype calibration. They therefore do not evaluate the original method. If a configuration in Table 6 is already equivalent, please specify the correspondence and include the corresponding visual-text result.

  2. The authors' response confirms that the canvas layouts, fusion weights, thresholds, and prompt variants were chosen using the same target data on which performance is reported. These choices were therefore informed by the evaluation labels. The reported scores are not an independent test of a configuration fixed before evaluation. Results across matched seeds describe sampling variability under the selected configuration. They do not address the bias introduced by selecting that configuration on the evaluation data. Since the competing methods’ scores are quoted from their papers, please also clarify whether their configuration-selection protocols are comparable.

  3. The controlled decomposition in Table 6 is 63.55 → 70.16 → 71.56. Adding the complementary prompting branch contributes 6.61 points, while the auxiliary calibration terms contribute 1.40 points. The 64.95 visual-only result already includes prototype calibration, so it cannot serve as the starting point for isolating the effect of candidate pooling alone. The four-point threshold sweep and matched-seed results support a calibration benefit under the selected settings. To assess calibration as a central technical contribution, I need to see whether that benefit holds after independent threshold selection.

Overall, the rebuttal clarifies the implementation and strengthens the internal ablations. My assessment still depends on a direct comparison with the closest prior method and a properly selected confidence-only baseline. Until these comparisons are available under an independent evaluation protocol, I remain unconvinced that the paper has sufficiently established its central contribution.

Thank you for taking the time to review our response and for clarifying your remaining concerns. We appreciate your feedback on the comparison with Tsai et al. and the evaluation protocol. We address your points below.

Concern1: The visual-only, confidence-only configuration in Table 6 closely follows Tsai et al.’s visual-only setup in how masks are generated and selected. Both use frozen SAM3 on a shared support–query canvas with a box prompt derived from the support image, and select masks using SAM3’s own confidence scores. We evaluated the corresponding visual–text configuration on the same 1-shot episodes by adding a class-name prompt and using the same SAM3 confidence-based mask selection rule. As shown in Table 12 of the appendix, the visual-only and visual–text configurations achieve average mIoUs of 63.55% and 60.74%, respectively. Pooling candidates from both branches while retaining confidence-only selection raises average mIoU to 70.16%, a gain of 9.42 points over the visual–text configuration. Applying full calibration to the same dual-branch candidate pool further improves average mIoU to 71.56%.

Table 12. Comparison of single- and dual-branch configurations on matched 1-shot episodes. Results are reported as mIoU (%).

ConfigurationDeepGlobeISICChest X-rayFSS-1000Average
Visual-only, SAM3 confidence26.1862.9780.4384.6263.55
Visual–text, SAM3 confidence43.4039.0271.7588.7860.74
Dual-branch, SAM3 confidence44.5964.1383.2588.6770.16
Dual-branch, full calibration46.0565.5584.6090.0471.56

Concern2: Our goal was to identify a shared inference configuration that performs well across the four CD-FSS benchmarks while keeping SAM3 frozen. We used average mIoU across the four target datasets to select among the configurations we evaluated. These benchmarks cover satellite imagery, dermoscopic images, chest radiographs, and everyday-object images, allowing us to assess the same configuration across diverse domains. The matched seeds quantify episode-sampling variability for the selected configuration. As configuration selection and reporting used the same benchmarks, we interpret the scores in that context. Regarding comparability, Table 1 includes both source-trained and training-free baselines. Several baselines, such as PATNet and DR-Adapter, use PASCAL VOC for source-domain training before evaluation on the four target datasets. PerSAM, Matcher, GF-SAM, and CPS are closer comparisons to our method because they use pretrained models and annotated support examples without task-specific source-domain training. Our method likewise keeps SAM3 frozen and uses the support examples at inference time. The studies report mIoU on the same four target datasets under the 1-shot and 5-shot settings. This provides a common basis for comparing reported segmentation performance at the benchmark level.

Concern3: We agree that 63.55% is the appropriate starting point for isolating the effect of adding the visual–text branch, since the 64.95% visual-only result already includes prototype calibration. As shown in Appendix Table 12, pooling candidates from both branches under confidence-only selection raises average 1-shot mIoU from 63.55% to 70.16%. We also previously used PASCAL VOC alone to select the inference hyperparameters and then evaluated both dual-branch variants on all four target datasets with those settings fixed. The selected hyperparameters included the fusion weights for full calibration and the candidate selection threshold for each variant. These settings were largely consistent with those selected on the four target datasets. Thus, full calibration still improves average mIoU when these hyperparameters are selected without using the four target datasets. At the same time, these four widely used CD-FSS benchmarks span a broad range of image domains, providing a diverse basis for evaluating cross-domain generalization.

We appreciate the reviewer’s careful consideration of our work and hope these clarifications address the remaining concerns and help put our contributions in context.

Reviewer_LhEPOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies training-free cross-domain few-shot segmentation (CD-FSS) with a fully frozen SAM3. Support and query images are placed on a shared canvas, and a box derived from the support mask serves as a visual exemplar prompt. The authors observe that adding the class name as a text prompt helps on some target domains (DeepGlobe, FSS-1000) but hurts on others (ISIC, Chest X-ray). They therefore run SAM3 with both visual-only and visual+text prompts, pool the candidate masks, rescore each candidate with a weighted sum of SAM3 confidence, support–query prototype similarity and cross-branch IoU, and output the union of candidates above a threshold. On the standard four-dataset CD-FSS benchmark the method reports 71.56 / 74.60 average mIoU (1-shot / 5-shot), above both training-required and SAM-based training-free baselines. Ablations cover canvas layout, the calibration terms, fusion weights, negative prompts and domain-aware text prompts.

Critical strengths
  1. A strong, simple, fully training-free baseline for CD-FSS. With no source-domain training and no parameter updates, the method improves the average mIoU by +7.33 / +5.83 (1-shot / 5-shot) over the strongest compared method (HSL), with especially large gains on ISIC (65.55 / 72.54) and FSS-1000 (90.04 / 91.14). This matters for my assessment because the pipeline is easy to reproduce and gives the community a high, practical reference point for CD-FSS.

  2. A useful empirical finding about text guidance under domain shift. Fig. 4 shows that adding the class name improves DeepGlobe by +17.4 mIoU but degrades ISIC by -23.9, and the dual-branch design is the best configuration on all four datasets. This matters because it both motivates the design and is informative for anyone applying concept-segmentation models such as SAM3 outside natural images.

Critical weaknesses
  1. The gain is attributed to "candidate calibration", but the paper's own ablations suggest it mostly comes from pooling the two branches. In the 1-shot averages, the visual-only branch gives 64.95, pooling both branches with SAM3 confidence alone gives 70.16 (+5.21), and adding the prototype and agreement terms gives 71.56 (+1.40). Picking the better single branch per dataset already gives 70.36. With the reported weights (alpha=1, beta=0.2, gamma=0.1, delta=0.5), Eq. 11-12 is algebraically equivalent to keeping a candidate iff its SAM3 confidence exceeds 0.55 - 0.05Δcos - 0.1r_agree, i.e., a confidence cutoff near 0.5 shifted by at most ±0.15. This matters because the paper's named contribution adds relatively little, and it is unclear whether the improvement reflects visual/text complementarity or simply ensembling two forward passes.

  2. The evaluation protocol does not establish the robustness of the smaller effects. The canvas layout, delta, alpha/beta/gamma and the 5-shot layout are all selected by test mIoU on the same four datasets reported in Table 1, and the paper reports no episode count, seeds or variance. Within the sweeps, mIoU moves by 0.6-1.3 points (e.g., alpha=0.5: 70.31; delta=0.4: 70.98), which is comparable to the +1.40 calibration gain. This matters because the calibration and layout claims cannot currently be separated from selection effects and noise. I consider this fixable, and it does not call the large gains over prior methods into question.

Questions for authors

Clarifications and evidence that could change my assessment:

Q1. Please report the decomposition in Fig. 4 / Table 3a (visual-only -> pooling with SAM3 confidence only -> full calibration) with the standard deviation over at least three episode seeds and with hyperparameters selected on held-out data (e.g., leave-one-dataset-out). Please also re-tune delta for each Table 3a variant and add a same-cost ensemble control that pools two visual-only runs (e.g., V-Top and V-Bottom canvases) under the same selection rule.

Q2. How is the SAM3 confidence s̄ normalized (L244)? If it is min-max normalized within an episode, the top candidate always passes the threshold. What are the empirical ranges of r_proto and r_agree? Eq. 9 (Norm[0,1]) and Alg. 1 ((Δcos+2)/4 with clipping) also appear to differ.

Q3. Could you add SAM3 rows to Table 1 for both shot settings: visual-only, visual+text, and text-only (class name on the query, no support)? Please also mark which methods use class names, since the compared baselines do not. The text-only row is particularly informative for DeepGlobe, where visual-only gives 27.24 but visual+text gives 44.65.

Q4. What are the exact text prompts for each dataset and class? Why do the single-branch numbers in Fig. 1a (e.g., ISIC 62.97) differ from those in Fig. 4 (64.45)? How are the prototypes and cross-branch agreement computed in the 5-shot setting, and why can 5-shot underperform 1-shot on Chest X-ray (83.99 vs 84.60)?

Suggestions that do not affect my score:

  • The negative-prompt study and the 1-shot/5-shot layout comparisons closely follow Tsai et al. (2026, arXiv:2604.05433), which is already cited for the canvas; please cite it in those sections as well.
  • L083 states that prompt conflict "explains" the negative effect of text, while L080 presents it as a guess; please either support it with analysis or phrase it as a hypothesis.
  • Please proofread: "CROOSS" (L105), "They can generate" (L050), "method rely" (L067), the support-set definition (L138), and \citet vs \citep.
3 comments

We thank the reviewer for the constructive feedback and for recognizing the practical value of our training-free framework and the domain-dependent effect of text prompts. We address the concerns point by point below.

Response:

Q1:We first clarify the gain decomposition: visual-only prompting achieves 64.95 average mIoU; pooling both branches with SAM3 confidence alone increases it to 70.16; and full calibration further improves it to 71.56. We also report the corresponding per-dataset results across three matched episode seeds(0,123, and 210) as mean ± standard deviation in Table 6 of the revised main text. To further examine these gains, we evaluated four thresholds for each Table 3(a) variant and added a two-pass visual-only control using V-Top and V-Bottom under the same selection rule. The results are reported in Table 5 of the revised main text.

Table 6. Decomposition of 1-shot mIoU (%) across three matched episode seeds. Results are reported as mean ± standard deviation.

ConfigurationDeepGlobeISICChest X-rayFSS-1000Average
Visual-only, confidence-only26.18 ± 0.2562.97 ± 0.1980.43 ± 0.2584.62 ± 0.1863.55 ± 0.21
Dual-branch, confidence-only44.59 ± 0.2464.13 ± 0.2383.25 ± 0.1988.67 ± 0.1670.16 ± 0.21
Dual-branch, full calibration46.05 ± 0.2165.55 ± 0.1884.60 ± 0.2290.04 ± 0.1571.56 ± 0.19

Table 5. Threshold analysis of candidate pools and calibration variants under the 1-shot setting. Results are reported in mIoU (%).

Candidate poolPrototypeCross-branchδ = 0.3δ = 0.4δ = 0.5δ = 0.6
Visual-only + visual-text––68.8569.4570.1669.91
Visual-only + visual-text✓–69.9570.6171.1570.96
Visual-only + visual-text–✓69.7270.3871.0270.67
Visual-only + visual-text✓✓70.3170.9871.5671.24
V-Top + V-Bottom✓✓66.7167.2567.8667.51

Full calibration performs best across all four tested thresholds, improving average mIoU from 70.16% to 71.56% at δ = 0.5. It also outperforms the two-pass visual-only control (67.86%), supporting the benefit of complementary visual and textual prompting beyond simply running two inference passes.

Q2: We use SAM3’s output confidence, clipped to [0, 1], without episode-wise min–max normalization. Thus, the top candidate is not automatically assigned a score of 1. SAM3 confidence filtering and the subsequent fused-score threshold are separate steps. The implementation computes r_proto = clip((Δcos + 2)/4, 0, 1), using fixed theoretical bounds rather than episode-specific extrema. We will replace the ambiguous “Norm” notation in Eq. (9) with this explicit formula to match Algorithm 1:riproto=clip⁡(cos⁡(zi,pfg)−cos⁡(zi,pbg)+24,0,1).r_i^{\mathrm{proto}} = \operatorname{clip}\left( \frac{\cos(z_i,p_{\mathrm{fg}})-\cos(z_i,p_{\mathrm{bg}})+2}{4}, 0,1 \right). r_agree is the maximum cross-branch mask IoU, or zero if the other branch has no candidates, with no additional normalization. Both scores are bounded by [0, 1]; their observed ranges are [0.35,0.86] and [0.19,0.96], respectively.

Q3: We have added results for frozen SAM3 with visual-only, visual-text, and text-only prompting to Table 1 under both shot settings. The text-only configuration receives only the query image and class-name prompt, without any support image or annotation. The results are reproduced below for convenience. All values are mIoU (%).

MethodDeepGlobe 1-shotDeepGlobe 5-shotISIC 1-shotISIC 5-shotChest X-ray 1-shotChest X-ray 5-shotFSS-1000 1-shotFSS-1000 5-shotAverage 1-shotAverage 5-shot
Training-required CD-FSS methods
HSNet29.6535.0831.2035.1051.8854.3677.5380.9947.5751.38
DMTNet34.5247.6234.3440.8171.1472.6970.8172.8952.7058.50
PATNet37.8942.9741.1653.5866.6170.2078.5981.2356.0661.99
DR-Adapter41.2950.1240.7748.8782.3582.3179.0580.4060.8665.42
ABCDFSS42.6049.0045.7053.3079.8081.4074.6076.2060.6764.97
APM40.8644.9241.7151.1678.2582.8179.2981.8360.0365.18
LoEC44.1049.6738.2147.0481.0282.7378.5180.6060.4665.01
HSL46.1353.8048.0155.5684.5785.3478.2280.3664.2368.77
SAM-based training-free methods
PerSAM33.3937.9021.2725.8731.1236.3479.6580.8641.3545.24
Matcher48.1250.9238.6035.0366.8566.3787.0189.6560.1560.49
GF-SAM49.5057.7648.7955.2447.6547.4088.0088.9858.4962.35
CPS46.0249.8551.3356.6070.0670.2781.7682.8962.3064.90
Ours (visual-only)27.2429.4664.4569.6682.3682.0485.7587.0664.9567.10
Ours (text-only)37.59–33.79–52.24–82.59–51.55–
Ours (visual-text)44.6548.3140.5647.5473.7872.8289.9690.8162.2464.87
Ours (dual-branch)46.0550.7265.5572.5484.6083.9990.0491.1471.5674.60

Q4: For DeepGlobe and FSS-1000, we use the dataset category names as text prompts, replacing underscores with spaces. For ISIC, we use “skin lesion”; for Chest, we use “lung”. The same prompts are used in both 1-shot and 5-shot settings. Figure 1(a) presents a preliminary analysis of the original frozen SAM3 outputs without candidate calibration, whereas Figure 4 reports single-branch results after candidate calibration using the SAM3 confidence and prototype scores. In the 5-shot setting, we obtain foreground and background prototypes by masked average pooling for each support image, L2-normalize them, average across the five supports, and normalize again. Candidate consistency is the foreground–background cosine-similarity difference mapped to [0, 1]. Both branches use all five supports; cross-branch agreement is the maximum query-mask IoU with candidates from the other branch, or zero if none exist. Finally, although 5-shot improves performance on the other three datasets, it is 0.61 mIoU points lower on Chest X-Ray. Changes in canvas layout and prototype aggregation may offset the benefit of additional supports in this domain.

Thank you for these additional suggestions. We have made corresponding revisions, such as citing Tsai et al. in the relevant sections and correcting the noted writing and citation-format errors.

Reviewer_LhEPOpenReview ↗

Maintaining my original score

Thank you for your detailed and thoughtful response. Your clarifications have resolved several of my concerns, and I appreciate the additional experiments, which provide useful supporting evidence. However, my main remaining concern is that the contribution still seems comewhat scattered and lacks a sufficiently clear and unified technical narrative. Taking all factors into consideration, I will maintain my original rating.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.