acceptodds
Withdrawn from ICLR 2027

CHRF-SAM3: Constrained Heterogeneous Representation Fusion for Cross-Domain Few-Shot Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Cross-domain few-shot visual learning aims to adapt models to unseen visual domains with only a small number of labeled samples. Benefiting from large-scale pretraining, SAM3, as a new-generation vision foundation model, exhibits strong zero-shot generalization, yet it struggles to effectively adapt to novel cross-domain visual distributions under extremely limited supervision. To address this issue, we introduce DINOv3, which is heterogeneous from SAM3 and yields multi-scale visual representations learned via a distinct pretraining paradigm. Specifically, we adopt its convolutional variant. Building upon this, we develop Convolutional Residual Fusion (CRF) and Layer-Shared Conditional LoRA (LSC-LoRA) to inject the heterogeneous multi-scale representations into the visual encoder and enable conditional low-rank adaptation. On the decoder side, we employ SoMA to constrain low-rank parameter updates. Extensive experiments demonstrate that our method achieves state-of-the-art performance across multiple benchmarks.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Cross-domain few-shot learning, Vision foundation models, SAM3, Few-shot object detection, Few-shot semantic segmentation, Low-rank adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_8DCxOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

The paper adapts SAM3 to cross-domain few-shot detection and segmentation by fusing frozen DINOv3 features into the SAM3 encoder through convolutional residual fusion and a conditionally modulated low-rank adapter, while applying singular-value-based low-rank adaptation to the decoder. It evaluates the method on CD-FSOD, ODinW-13, PASCAL, and eight cross-domain segmentation datasets and reports gains over several recent baselines. The main claim is that constrained heterogeneous fusion lets SAM3 learn novel-domain knowledge while preserving pretrained generalization.

Critical strengths

The paper is readable and professionally written. The modular pipeline is easy to follow.

Critical weaknesses

1.The CD-FSOD comparison omits recent vision-foundation-model detectors such as FSOD-VFM and the NTIRE 2026 CD-FSOD challenge entries, as well as recent target-domain refinement work by Jiang et al. The paper also does not discuss multi-modal prototype methods beyond reporting their numbers, which weakens the claimed state-of-the-art positioning. 2. Recent studies already show that SAM-family models are edge-strong while DINOv3 is structure-strong, and they propose fusion schemes for dense prediction, including Guo et al. and SD-SAM. The paper does not cite or distinguish itself from these works, so the core premise of complementary SAM and DINOv3 fusion is not clearly new. 3.The residual injection is a standard multi-scale fusion pattern, the conditional adapter resembles shared-convolution adaptation tuning, and the decoder adaptation follows SoMA and MiLoRA logic. The cumulative ablation does not isolate whether the specific combination is necessary beyond simpler conditional or residual designs. 4. The conditional mapping defines two condition maps but the adaptation uses only one for both feed-forward layers. The residual fusion and conditional mapping are computed from the same DINOv3 features without clarifying whether their projections are independent. The segmentation results show an unexplained reversal for PAXRay4 under point prompting, and the DINOv3 ConvNeXt label is ambiguous because the primary DINOv3 releases are ViT-based. The term constrained fusion is also not operationalized beyond freezing and low-rank adaptation.

FSOD-VFM: Few-Shot Object Detection with Vision Foundation Models and Graph Diffusion

Metric-Guided Feature Fusion of Visual Foundation Models for Segmentation Tasks

Remedying Target-Domain Astigmatism for Cross-Domain Few-Shot Object Detection

The Second Challenge on Cross-Domain Few-Shot Object Detection at NTIRE 2026: Methods and Results

Learning Multi-Modal Prototypes for Cross-Domain Few-Shot Object Detection

MiLoRA: Harnessing Minor Singular Components for Parameter-Efficient LLM Finetuning

SD-SAM: Synergizing SAM and DINOv3 for Parameter-Efficient Polyp Segmentation and a Challenging Benchmark Dataset

SoMA: Singular Value Decomposed Minor Components Adaptation for Domain Generalizable Representation Learning

Cross-Domain Few-Shot Infrared Ship Segmentation with Class-Specific Adapters and SAM Refinement

Shared-Convolution Adaptatio

Questions for authors

See Weaknesses

Reviewer_d9X1OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes CHRF-SAM3, a framework for cross-domain few-shot object detection and semantic segmentation based on SAM3. The main idea is to enhance the cross-domain adaptation capability of SAM3 by incorporating heterogeneous visual representations extracted from a frozen DINOv3 ConvNeXt-L backbone.

The proposed framework consists of (1) Convolutional Residual Fusion (CRF), which injects multi-scale convolutional features from DINOv3 into the final three blocks of the SAM3 visual encoder through learnable residual projections, (2) Layer-Shared Conditional LoRA (LSC-LoRA), which introduces image-dependent features derived from DINOv3 into the low-rank adaptation branches of the SAM3 visual encoder, and (3) SoMA-based decoder adaptation, which adapts the tail singular components of pretrained decoder weights to preserve pretrained knowledge while learning target-domain representations.

The authors evaluate the proposed framework on multiple few-shot detection and segmentation benchmarks, including CD-FSOD, ODinW-13, PASCAL-5i, and eight cross-domain semantic segmentation datasets. The experimental results demonstrate improvements over existing methods and SAM3 adaptation baselines across various few-shot settings. Additional ablation studies investigate the contributions of the proposed components, auxiliary backbone choices, and preservation of pretrained knowledge.

Critical strengths
  • The paper explores an interesting direction for cross-domain few-shot adaptation by combining the complementary representations of SAM3 and DINOv3. Integrating convolutional features from a self-supervised foundation model into the SAM3 visual encoder is a reasonable approach to addressing domain-specific representation limitations under scarce supervision.
  • The proposed method demonstrates competitive results on a broad range of benchmarks covering both object detection and semantic segmentation. In particular, it achieves improvements over the original SAM3 on ODinW-13 and strong performance on PASCAL-5i. The evaluation across diverse target domains, including underwater, medical, industrial, and remote sensing imagery, provides useful evidence of the framework's applicability.
Critical weaknesses
  • The proposed method primarily combines existing pretrained foundation models and established parameter-efficient adaptation techniques. In particular, SoMA is directly adopted from prior work, while CRF and LSC-LoRA extend relatively familiar residual feature fusion and conditional low-rank adaptation mechanisms. Although their integration into SAM3 is practically meaningful, the paper does not sufficiently establish which aspects represent fundamentally new methodological contributions beyond existing feature fusion and parameter-efficient fine-tuning approaches.
  • A central claim of the paper is that heterogeneous representations from DINOv3 improve cross-domain adaptation. However, the current experiments do not fully distinguish the benefits of the additional pretrained visual encoder from those of the proposed fusion mechanisms. For example, it remains unclear whether comparable improvements could be obtained using simpler feature concatenation, conventional feature adapters, or standard LoRA combined with DINOv3 features. Furthermore, the auxiliary backbone analysis indicates that DINOv3 provides substantially larger improvements than other convolutional backbones, suggesting that the quality of the additional pretrained representations may be a major factor.
  • The main ablation study in Table 5 incrementally adds CRF, LSC-LoRA, and SoMA, but does not evaluate all combinations of these components independently. In particular, LSC-LoRA without CRF is not examined. Moreover, adding LSC-LoRA slightly decreases performance on CD-FSOD in the 5-shot and 10-shot settings, while SoMA does not consistently improve performance across all settings. These observations raise questions about the individual effectiveness and complementarity of the proposed components.
  • The proposed framework introduces an additional DINOv3 ConvNeXt-L backbone, increasing the total parameter count from 838.2M to 1066.5M and the computational cost from 5.7T to 7.4T FLOPs, corresponding to increases of approximately 27% and 30%, respectively. Although the method employs parameter-efficient adaptation, its inference cost is substantially higher than that of the original SAM3. The paper does not provide sufficient information about the number of trainable parameters, training memory requirements, adaptation time, or actual inference latency.
Questions for authors

Please see the weaknesses section.

Reviewer_mgiqOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes CHRF-SAM3 to improve cross-domain few-shot object detection and segmentation. The motivation is that SAM3 has strong zero-shot capabilities but struggles to adapt to new domains with limited training data, so this paper introduces DINOv3 to provide additional visual features, Convolutional Residual Fusion (CRF) and Layer-Shared Conditional LoRA (LSC-LoRA) to fuse these features into SAM3, and SoMA to adapt the decoder. Improvements were shown across many object detection and semantic segmentation benchmarks.

Critical strengths
  1. The motivation for improving cross-domain few-shot adaptation by combining features from SAM3 and DINOv3 is intuitive and convincing. Those two are among the best models for tackling this problem.
  2. I appreciate the exhaustive experiments on both object detection and semantic segmentation across many datasets.
  3. The ablation studies are helpful. E.g., Table 5 shows the effectiveness of CRF and LSC-LoRA, while Table 8 shows the benefit of using DINOv3 compared to other backbones.
  4. I also appreciate the experiments under different few-shot settings (1-shot, 5-shot, 10-shot) and prompting strategies to show the effectiveness of the method.
Critical weaknesses
  1. The method requires an additional DINOv3 backbone, making it slower and larger. Table 10 shows +29% FLOPs and +27% parameters, but the improvements are sometimes small, e.g., +1.1% on CD-FSOD 10-shot.
  2. Some comparisons use different backbones, which might be unfair. E.g., Table 1 compares FT-FSOD (Swin-B) with the proposed method (PE-L+). I understand that it is very difficult to re-train SAM3 with Swin-B backbone though, so this might be an unfair ask from me.
  3. The experiments only use SAM3 with the PE-L+ backbone. Would it work on other models, or different SAM backbones?
Questions for authors
  1. Would the method still outperform existing works under the same model size and speed budget (weakness 1)?
  2. Would it work on other SAM backbones or other models (weakness 3)?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.