acceptodds
Withdrawn from ICLR 2027

Training-Free Cross-Domain Few-Shot Segmentation via Preserve-Relative Counterfactual Refinement

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cross-domain few-shot segmentation aims to segment unseen classes in new domains using only a few annotated examples. In training-free settings, frozen semantic representations guide initial predictions, while promptable segmentation models generate structurally coherent candidate masks. However, under domain shift, structurally coherent masks can still misidentify target regions, undermining the semantic reliability of refinement. When the initial prediction also guides candidate selection, refinement can preserve or reinforce its semantic errors. We therefore propose a training-free framework that uses an audited semantic reference to improve the semantic reliability of mask refinement under domain shift. To this end, Cycle-Consistent Semantic Cover (CSC) uses semantic matching to select reliable prompts covering complementary target regions. Building on these prompts, Preserve-Relative Counterfactual Mask Selection (PRCMS) compares mask responses across prompt subsets and combines this evidence with support semantics to audit the reference and select refinements. Across eight datasets, our framework achieves mean mIoU of 69.33% in one-shot and 75.20% in five-shot segmentation, exceeding the evaluated state-of-the-art baselines. Ablation studies further show that complementary prompt coverage and reference auditing both improve cross-domain segmentation. Code will be released upon publication.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Cross-domain Few-shot Segmentation, Training-free Segmentation, Vision Foundation Models

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_aW5POpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free framework for CD-FSS. CSC selects reliable and complementary point prompts, while PRCMS uses SAM 3 responses across prompt subsets and support semantics to construct an audited reference and evaluate foreground additions and removals. Experiments across eight datasets demonstrate the effectiveness of the method.

Critical strengths
  1. The paper distinguishes mask quality from revision value, and audits the reference before accepting changes. This helps preserve correct regions and avoid reinforcing errors in the initial mask.
  2. The method requires no additional model training, which is useful when only a few annotated support examples are available. It avoids updating model parameters for each target domain.
  3. The experimental validation includes matched episodes across three seeds, component ablations, and controls with fixed features, prompts, and response banks. These controls help separate the effects of reference auditing and selection from candidate generation.
Critical weaknesses
  1. The complete decision process involves substantial manual design. PRCMS uses numerous thresholds, scoring coefficients, geometric checks, branch priorities, and fallback rules. The terminal procedures also differ between one-shot and five-shot inference, making the overall method complex to implement and adapt.
  2. The accuracy–cost trade-off of the complete response bank needs further evaluation. The method evaluates up to 32 prompt subsets, while the reported 13.1% latency increase is measured against a control that already generates the same response bank. This does not show the additional cost relative to RSRM or a simpler refinement pipeline. Comparing smaller response banks would help clarify whether similar gains can be achieved with fewer SAM 3 evaluations.
Questions for authors

see weakness

Reviewer_e6mgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a training-free refinement layer on top of a frozen RSRM prediction for cross-domain few-shot segmentation. CSC picks complementary prompt points by submodular coverage over DINOv3 matches. PRCMS runs SAM 3 on all 2^k prompt subsets (k≤5), computes Shapley attributions, "audits" the initial mask into a preserve state B, and then accepts directional add/remove edits relative to B. It reports 69.33 (1-shot) and 75.20 (5-shot) mean mIoU on 8 datasets, +4.47 and +3.18 over RSRM.

Critical strengths
  1. The framing is sensible: a candidate's value depends on the current prediction, not just its quality.
  2. The evaluation protocol is careful. Episodes and seeds are matched, gains are paired, and the configuration is chosen on ADE20K rather than the test sets.
  3. The ablations are informative, especially B=B0 vs. audited B (−3.25) and the fixed-pool selector controls (Tables 6 and 8).
  4. The exponential cost, the single-seed CG-ICS result, and the failure cases are all mentioned.
Critical weaknesses
  1. The SOTA claim is not apples-to-apples.

    a. The method uses SAM 3 plus DINOv3, while PerSAM, Matcher, GF-SAM and similar baselines use older SAM backbones. The paper itself notes backbone and budget differences.

    b. The strongest control, SAM 3 concatenation (66.00), is only 3.3 points behind and beats the method on ISIC (68.14 vs 59.05), VOC and PASCAL.

    c. GF-SAM beats it on COCO and INSID3-L on iSAID.

    d. The claim that the method exceeds the evaluated state of the art therefore depends on which baselines are chosen. A fair test would put SAM 3 under the strongest existing matchers/refiners (e.g., INSID3 or GF-SAM prompts into SAM 3).

  2. The method depends on RSRM.

    a. It is a refinement module, and the only host-generality evidence is CG-ICS at a single seed (Table 10).

    b. It is unclear how much of the gain survives with a weaker or different B0 (e.g., INSID3 or Matcher output).

  3. Selective or inconsistent ablation reporting.

    a. Table 4 uses 6 of 8 datasets and drops VOC and iSAID, where the method's margins are smallest.

    b. Table 5(a) is seed 0 only, and Table 5(b) is a different subset again.

    c. Several deltas (0.41, 0.84, 0.7) sit below the per-dataset seed std (~1–4 points on SUIM and ISIC). Paired significance is not reported for the ablations.

  4. Five-shot is weakly supported.

    a. SAM 3 only uses the first support, and SUIM drops (−0.49).

    b. Gains shrink relative to one-shot, and the baseline set is small (no SAM 3 concatenation, no INSID3-B or Matcher).

    c. Large pieces of five-shot logic (coalitions, interaction mass) are hand-designed.

  5. Complexity versus principle.

    a. The core is a hand-built decision procedure with many thresholds, score coefficients, branch priorities and fallbacks, all specified in appendix tables.

    b. The sensitivity study perturbs only ±20% around defaults and tunes on ADE20K, so robustness to a genuinely different regime is untested.

    c. "Counterfactual," "Shapley" and "audit" are used loosely. Exact Shapley over prompt subsets measures response attribution, not semantic correctness. There is no analysis of when the audit is wrong, and no oracle upper bound for the selection step in the main text.

  6. Cost reporting is narrow.

    a. Latency (387 ms vs 343 ms) is compared only against uniform averaging with the same 2^k bank, on 12 ISIC episodes.

    b. Missing: cost versus RSRM alone or versus the baselines in Table 1. The 2^k scaling limits k, which also limits the coverage idea.

  7. Presentation.

    a. The main text is dense and notation-heavy, and key definitions (admissibility conditions, branch priority) are deferred to a 46-page appendix.

    b. The "state transition ambiguity" concept is intuitive but is mostly a relabeling of the known issue of confirmation bias in self-referential refinement. Its measurement is indirect.

Questions for authors
  1. How does the method perform with SAM 3 as the common backbone for all baselines, and with a non-RSRM B0?

  2. Why were VOC and iSAID excluded from the main ablations?

  3. Give the end-to-end cost versus RSRM and GF-SAM, and the gain when k is reduced to 3.

  4. What fraction of audit decisions (restore versus retain) is correct, measured against ground truth?

Reviewer_MNsaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free framework for cross-domain few-shot segmentation that refines an initial RSRM prediction using frozen DINOv3 features and SAM3 responses. Its central idea is to audit the reference prediction before deciding which candidate revisions to accept.

Cycle-Consistent Semantic Cover (CSC) selects prompts using semantic reliability and complementary coverage, while reciprocal matching determines eligibility for subsequent attribution evidence. Preserve-Relative Counterfactual Mask Selection (PRCMS) evaluates all subsets of at most five prompts from a fixed SAM3 state, computes prompt-response attributions, and combines these with support semantics to construct an audited preserve state. Candidate foreground additions and removals are then assessed relative to this state. One-shot inference selects a complete mask, whereas five-shot inference composes admissible regional edits.

Critical strengths
  1. The central reference-auditing idea is supported by controls that meaningfully isolate its contribution. The paper goes beyond evaluating the composition of a semantic matcher with a stronger segmentation model.
  2. The reported improvements are broad, repeated, and accompanied by unusually detailed reporting of their limitations. The evaluation uses matched episode lists across methods, three support-sampling seeds, and class-macro foreground IoU. The analysis also examines class-level and episode-level behavior, rather than relying exclusively on aggregate means. It explicitly reports harmful episodes and the five-shot SUIM regression, and distinguishes repeated episode records from independent observations.
Critical weaknesses
  1. PRCMS combines numerous thresholds, evidence statistics, and branch priorities, with substantially different procedures for one-shot and five-shot inference. The necessity of these separate designs and the ADE20K configuration-selection process are not explained sufficiently clearly.
  2. Replacing exact Shapley attribution with leave-one-out effects reduces the six-dataset mean by only 0.41 points (Table 4). This leaves uncertainty about how much the exact attribution mechanism contributes beyond simpler prompt-response evidence.
  3. The reported 13.1% overhead is measured against a control that also constructs the full exponential response bank, rather than against RSRM or reduced-response alternatives. Timing covers only 12 one-shot ISIC episodes, leaving broader deployment costs and five-shot efficiency insufficiently characterized.
  4. Despite positive average gains, refinement degrades 31.1% of SUIM episodes and 24.7% of COCO episodes in the one-shot analysis, while five-shot SUIM decreases by 0.49 mIoU points. These results expose a practical limitation of the audit and admissibility rules in preventing detrimental revisions.
  5. Appendix A.2.1 excludes the query patch from its support set but does not enforce separation by original aerial scene. Consequently, support and query patches may share scene-specific context, leaving performance under a stricter scene-disjoint protocol unresolved.
  6. The CG-ICS experiment uses a host-specific selection adaptation and only one seed (Table 10). It provides limited evidence that the same auditing and revision rules transfer unchanged across different initializers or segmentation pipelines.
Questions for authors
  1. Appendix A.1.1 identifies ADE20K as the configuration-selection dataset. Which split and episode protocol were used, what objective was optimized, and what search space and budget were considered? Were the one-shot and five-shot branch structures fixed beforehand, or selected together with their numerical thresholds and coefficients? Please also clarify how configuration selection was applied to the calibrated simpler selectors.
  2. What motivates selecting an entire mask only in the one-shot setting and composing regional edits only in the five-shot setting? With the response bank and available support evidence held fixed, how does a common decision procedure across both shot settings compare with the proposed shot-specific procedures?
  3. Under the same hardware and input-processing conditions, what are the end-to-end latency and memory requirements of RSRM, direct SAM3 alternatives, and the full method for both shot settings? How much accuracy is retained when evaluating fewer prompt subsets or using an approximate response-bank construction?
  4. Which parts of PRCMS are retained, modified, or retuned in the CG-ICS adaptation reported in Table 10? Does it use the original decision configuration, or a host-specific selection rule and configuration? Please clarify which aspect of the proposed method this experiment demonstrates to be transferable, and qualify the overall performance claims accordingly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.