acceptodds
Withdrawn from ICLR 2027

DualBridge: What to Transmit Between Heterogeneous Foundation Models for Domain-Generalized Medical Image Segmentation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Single-source domain generalization requires medical image segmentation models to generalize well using supervision from a single source domain. Recently, vision foundation models (VFMs) have attracted growing interest for their transferable knowledge acquired through large-scale pretraining. Self-supervised VFMs such as DINOv3 provide broadly transferable representations, whereas promptable VFMs such as SAM 3 offer segmentation-specific capabilities. However, bridging these complementary strengths is nontrivial: effective integration requires deciding what information to transmit and where to inject it. Prompt-only interfaces can discard dense semantics, while unrestricted feature fusion can increase the adaptation burden under limited medical supervision. To balance these competing demands, we propose DualBridge, a scale-selective framework that connects DINOv3 and SAM 3 through two complementary information-transfer pathways. The spatial pathway converts DINOv3-derived predictions into explicit spatial and shape prompts for SAM 3. The feature pathway complements this compact interface by selectively transferring dense semantic features. Specifically, lightweight adapters project intermediate DINOv3 features into SAM 3’s high-level representations through gated residual connections that regulate their contribution. Together, these pathways combine explicit spatial guidance with selective feature transfer, without access to target-domain data during training. Experiments on cross-domain retinal fundus and polyp segmentation benchmarks demonstrate that DualBridge outperforms state-of-the-art methods under the single-source protocol. These results support selective information transfer as an effective strategy for harnessing complementary foundation models under unseen domain shifts.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Domain Generalization, Medical Image Segmentation, Vision Foundation Model

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_7kvoOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes DualBridge, a framework that connects DINOv3 and SAM 3 for single-source domain-generalized medical image segmentation. It uses two paths: a spatial path that converts DINOv3 predictions into mask prompts, and a feature path that injects selected DINOv3 features into higher-level SAM 3 representations through gated adapters. The model is trained only with source-domain labels and does not use target-domain adaptation at test time. Experiments on retinal fundus and polyp segmentation show strong cross-domain performance, with ablations examining the roles of the two paths and the transferred feature content.

Critical strengths
  1. DualBridge provides a clear way to combine DINOv3 and SAM 3 through spatial prompting and selective feature transfer.
  2. The cross-domain gains are consistent across different training sources.
Critical weaknesses
  1. The novelty of the method is limited. DINO-based prompting and feature transfer into SAM both have clear precedents.
  2. Table 2 shows only a small gain from the feature pathway over the spatial-path model, which weakens the case for the dual-path design.
Questions for authors
  1. Was the choice of injecting features into the two deepest SAM 3 levels compared with other injection locations? If so, what were the results?
  2. Table 2 shows only a small gain from the feature pathway. Is this improvement consistent across individual source-target pairs, or mainly driven by a few domains?
  3. Is there a matched baseline that uses the same DINOv3 and SAM 3 backbones with the same trainable SAM 3 modules, but replaces DualBridge with a simpler fusion strategy?
Reviewer_XDu6OpenReview ↗
Rating
1
Confidence
2
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes DualBridge, a framework for single-source domain-generalized medical image segmentation that transfers knowledge across two frozen vision foundation models, DINOv3 and SAM 3. The method has two pathways: a spatial pathway that converts multi-depth DINOv3 predictions into a binary mask prompt for SAM 3, and a feature pathway that injects deep DINOv3 tokens into SAM 3's high-level feature pyramid via gated residual adapters. The model is trained in two stages using only source-domain data, with both backbones frozen. On five retinal fundus datasets and four polyp datasets, DualBridge achieves source-averaged Dice scores of 87.62 (polyp) and 86.00 (fundus), outperforming TTA, DG, and VFM-based baselines. Ablations show that the spatial pathway provides most of the gain (8.33 points), while the feature pathway adds a smaller improvement (0.49 points).

Critical strengths
  1. DualBridge achieves the highest source-averaged Dice on both tasks and, notably, its per-source scores span only 3.10 points on polyp and 2.27 on fundus. This robustness to the choice of source domain is a practically meaningful result.
  2. The design keeps both DINOv3 and SAM 3 frozen and trains only a small interface, while still outperforming heavier baselines. The two-stage, source-only training procedure is simple and avoids any target-domain adaptation at test time.
  3. Table 3 fixes the adapter architecture and parameter count while swapping the injected signal (real residual vs. magnitude-matched noise vs. spatially shuffled tokens). The real residual leads both controls, which is a clean way to show that the gain comes from feature content and spatial alignment rather than from adding parameters or perturbation.
Critical weaknesses
  1. Section 3.2 reduces to two elementary bounds from the data-processing inequality: a binary prompt carries at most NN N bits, and the feature channel carries at most bd_R bits beyond the prompt. These bounds are compatible with almost any interface of finite capacity. They do not predict why a mask prompt is preferable to a learned token, why gated residuals are preferable to cross-attention, or why levels {2, 3} are the right injection points. Removing Section 3.2 and Appendix A would leave the method and experiments unchanged, which suggests the analysis is decoration rather than derivation.
  2. Table 2 shows the spatial path contributes 8.33 points and the feature path only 0.49. The title and abstract present “selective cross-model transfer” as a co-equal contribution, yet the feature path is a rounding error. Table 3 shows the real residual leads magnitude-matched noise by 0.70 and shuffled tokens by 1.19, but no variance or significance is reported. The paper does not confront the possibility that the adapter is helping simply because it is a trainable module, not because of the specific DINOv3 content.
  3. The injection levels {2, 3} are justified by an architectural argument (level 3 reaches both encoder and head, level 1 carries high-resolution detail), but there is no experiment injecting at {1, 2, 3}, {3} only, or {1, 3}. The gated residual form is not compared against concatenation, ungated addition, or cross-attention. The “what to transfer and where to inject” framing promises a systematic answer, but the paper delivers one configuration and no comparison.
  4. The two-stage training freezes the DINOv3 segmentation head before Stage 2, so the mask prompt is optimized for standalone mask accuracy rather than for what SAM 3 actually needs under domain shift, and the paper never tests whether joint or end-to-end prompt training would perform better. The fixed threshold \tau = 0.5 is also adopted without ablation or sensitivity analysis, leaving the prompt quality–segmentation accuracy relationship unverified.
Questions for authors

see weaknesses

Reviewer_PmoYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents DualBridge for single-source domain-generalized medical image segmentation. It couples DINOv3 with SAM 3 through two routes: a DINOv3 segmentation head supplies a binary spatial prompt, while gated adapters inject dense features into selected levels of SAM 3's feature pyramid. The authors report mean Dice scores of 87.62 on polyp and 86.00 on fundus segmentation, with favorable comparisons to several existing methods. I find the general direction useful, but the evidence for the particular bridging design is less convincing than the overall performance numbers suggest.

Critical strengths
  1. The central problem is well chosen. DINOv3 provides transferable dense features, whereas SAM 3 offers a strong prompt-based segmentation interface; using both without target-domain adaptation is a reasonable direction for medical domain generalization.

  2. The architecture is fairly disciplined. The spatial prompt has a clear role, and injecting gated residuals only at higher feature levels avoids extensively modifying SAM 3's high-resolution path. Keeping the backbone weights frozen also limits the trainable components.

  3. The experimental section contains useful controls, not just a comparison table. In particular, the magnitude-matched noise and spatially shuffled-token experiments in Table 3 help test whether the transferred feature content matters.

Critical weaknesses
  1. The gain from the feature pathway is modest. In Table 2, adding feature transfer to the spatial-prompt configuration raises Dice by only 0.66 points on polyp and 0.33 on fundus. The feature-content controls in Table 3 are helpful, but they do not establish that these relatively small increments are stable across training runs. I would like to see repeated-seed results and per-source gains, rather than only the two task averages.
  2. The 'scale-selective' choice is not sufficiently isolated. Section 3.5 injects DINOv3 information into the two deepest SAM 3 pyramid levels while leaving the finest level unchanged. This is plausible, but the paper does not show why this particular pair is preferable to one level alone or to other injection locations. A focused ablation of feature levels and gated versus ungated residuals would make the design claim stronger.
  3. The fundus preprocessing protocol raises an important question. Appendix D states that an 800 × 800 region is extracted around the optic disc before resizing. In a target-domain test image, however, the optic-disc center is not necessarily available in advance. Please explain how the crop center is obtained without using ground-truth information, and report performance with an automatic localization step or full-image input if applicable.
  4. Not all fundus 'domains' represent independent acquisition sources. Section 4.1 treats the REFUGE training split as domain B and its validation/test split as domain D. These partitions are useful for benchmarking, but transfer between them is different from transfer between institutions or independent datasets. I would like to see fundus results recalculated after excluding B-to-D and D-to-B evaluations, together with a clearer account of what domain shift each split represents.
  5. Stronger matched comparisons are needed to establish the value of the bridge. Table 1 covers a good range of methods, but comparisons across different foundation models and adaptation recipes do not isolate the contribution of DualBridge itself. Appendix G provides some DINOv3 replacement controls, which is useful. The key missing test is a simpler prompt-plus-feature fusion baseline using the same DINOv3, SAM 3, trainable budget, and source supervision, preferably without the proposed gated design.
  6. The averaged Dice scores hide clinically relevant failures. Each source-domain score in Table 1 combines several targets, and the fundus score further averages optic-disc and optic-cup segmentation. These averages can conceal poor transfer to a particular target or a difficult optic-cup boundary. Please provide a source-to-target matrix and separate disc/cup Dice scores, ideally with boundary or failure-case analysis for the weakest transfers.
  7. Efficiency and dependence on prompt quality remain unclear. Table 5 compares trainable parameter counts, but inference still requires both DINOv3 and SAM 3, along with the prompt-generation head. Trainable parameters are therefore not a good proxy for deployment cost. Report end-to-end latency and peak memory against prompt-only SAM 3, and test whether the method remains effective when the Stage-1 mask prompt is inaccurate or its fixed threshold is varied.
Questions for authors
  1. For unseen fundus images, what locates the optic disc used for the 800 × 800 crop? Is this preprocessing fully automatic and identical for the baselines?
  2. How much of the feature-path improvement remains after averaging over different training seeds and reporting each source–target pair separately?
  3. What exactly is passed to SAM 3 in the 'No guidance' condition of Table 2, and are the geometry-encoder and LoRA training settings identical to the prompted variants?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.