acceptodds
Withdrawn from ICLR 2027

Conditional Role Allocation for Heterogeneous Supervision in Source-Free Domain Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Source-Free Domain Adaptation (SFDA) seeks to adapt a pre-trained source model to an unlabeled target domain without access to source data. Foundation models (FMs) offer new opportunities for SFDA through their rich and heterogeneous pre-trained knowledge, with their predictions serving as pseudo-supervision. Such approaches often treat additional supervision as uniformly beneficial across target samples. We find that this assumption does not always hold: supervision from a newly introduced model, such as a multimodal large language model (MLLM), can disrupt already reliable pseudo-labels, while providing valuable complementary corrections for samples not yet reliably covered. Motivated by this finding, we propose Conditional Role Allocation (CRA), a framework that coordinates FMs by assigning supervision roles according to existing reliable coverage. Specifically, CRA proceeds in three stages: (1) building reliable anchors by combining CLIP’s semantic confidence with DINOv2’s visual structural support; (2) expanding the supervision coverage through confidence ranking within predicted classes and class-balanced selection; and (3) applying MLLM adjudication constrained by CLIP-derived candidate classes to samples not covered by preceding stages. In this way, CRA preserves reliable supervision while directing complementary model knowledge toward samples where it is most needed. Extensive experiments demonstrate the effectiveness and versatility of CRA in closed-set, partial-set, and open-set SFDA, achieving state-of-the-art results on multiple benchmarks—all without accessing source data or fine-tuning foundation models.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
source-free domain adaptation, foundation models, pseudo-labeling, multi-model collaboration

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_MJEiOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates additional supervision in SFDA and observes that its incremental value depends on whether samples are already covered by reliable supervision. Based on this observation, the authors propose CRA. The framework first establishes reliable anchors using CLIP semantic confidence and DINOv2 structural support, then expands supervision through class-wise budgets and confidence-based selection, and finally applies candidate-constrained MLLM adjudication to residual samples. Experiments on three benchmarks cover closed-set, partial-set, and open-set settings, with ablations and supervision-allocation controls examining the framework’s effectiveness.

Critical strengths

The paper identifies a meaningful phenomenon: the incremental value of additional supervision depends on where it is applied. It can be redundant or even harmful for samples already covered by reliable supervision. This observation clearly motivates the proposed conditional role allocation framework. The evaluation covers closed-set, partial-set, and open-set SFDA and includes supervision-allocation controls, component ablations, and candidate-set analyses, providing substantial empirical support for the approach.

Critical weaknesses
  1. The comparison with the closely related method RCL is relativey limited. Both methods use frozen foundation models to provide supervision and train the student in stages based on reliability. However, the paper lacks a discussion of their differences in reliability assessment and supervision allocation.
  2. The rationale for selecting expansion samples in Stage II is insufficiently explained. According to Equation (4), the reference set selected through structural support is used only to determine the per-class expansion budgets. The actual expansion samples are selected from all non-anchor samples assigned to each class, ranked by the CLIP prediction margin. The paper does not adequately explain why this design is preferable to directly using the reference set.
  3. The evaluation of computational overhead is incomplete. The paper reports only the number of logical MLLM requests, without providing a time-complexity analysis or measured runtime.
  4. The writing and visual presentation need improve. For example, “domain shift Dataset statistics…” contains a sentence-boundary and punctuation error. In Figure 2, similar graphical elements represent different functions without sufficiently clear visual distinctions, making the modules and their relationships harder to follow.
Questions for authors

See Weaknesses.

Reviewer_KuXeOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Conditional Role Allocation (CRA), a method to address source-free domain adaptation through coordinating complementary foundational models to provide pseudo-labels based on CLIP and DINOv2 expertise. The method introduces new foundational teacher model knowledge in stages, starting with CLIP and DINOv2, and finally moving onto candidate-constrained MLLM adjudication for residual samples judged beyond reliable coverage. Experiments on three SFDA benchmarks (OfficeHome, DomainNet-126, and VisDA-C) in open-set, closed-set and partial-set adaptation settings show CRA exhibit comparable performance to RCL in closed-set SFDA while being meaningfully better than other listed competing methods in the open-set and partial-set SFDA settings.

Critical strengths

Strengths - The paper's motivation and methods are clearly written and easy to follow. Additionally, the authors present good controlled ablation studies and supplementary experimental results that are informative. Finally, the setting is not only constrained to the closed-set adaptation, but also include partial and open-set comparisons, though the significance of the later two comparisons are questionable.

Critical weaknesses

Weaknesses - Despite these strengths, this paper's main weaknesses are that all three of the paper's claimed contributions, the supervision-position effect, the proposed CRA, and the experimental results are problematic.
First, the supervision-position effect in foundation-model collaboration, where additional supervision from different foundational models can hurt learning for samples already well supported by other teacher models but can help for samples out of the existing model expertise is a well known and established knowledge in deep learning. It is obvious that models with different expertise will help on data within their expert domain while potentially being harmful for data outside of their expert domain. This effect is foundational to various well established areas of research like multi-model ensembling, mixture-of-experts gating, co-training, etc. I have a very hard time buying this as a new contribution to the field.
Next, for CRA, this idea of just a repackaging of traditional ideas like model cascades or deferrals, where a cheap model is used for initial prediction then escalating to a more powerful model for difficult cases. Even RCL, the biggest competitor for this paper, partitions samples by reliability and schedules MLLM supervision in a multi-stage curriculum. The proposed CRA contribution is really just an incremental engineering framework on which models are used for each step and how to choose the samples for escalation. The source free aspect of the method are also pretty well established in terms of the class anchors and support set expansion for SFDA (e.g. RCL). Additionally, open-set handling with two-cluster entropy rejection and information maximization closely follows SHOT.
Finally, the experimental results are also kind of weak. For closed-set SFDA setting, the proposed method is only on par with the closest competitor RCL (+1.1 OH, +0.2 DN, -1.3 VDA-C). despite using CLIP as well as a newer MLLM. For open-set and partial-set SFDA, CRA does beat the the nearest shown competitor by a substantial amount, but the comparison fairness is questionable as it's unclear how much of this lead is from superior methodology vs just stronger foundational teacher models compared to DIFO++ and ProDe.

Questions for authors

See the weaknesses section above.

Reviewer_R1zHOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses Source-Free Domain Adaptation (SFDA), focusing on the observation that additional supervision from foundation models is not uniformly beneficial across target samples. In particular, the authors identify the supervision-position effect, where additional supervision may degrade performance on samples with reliable pseudo-labels but provides substantial error correction for samples lacking reliable supervision. To address this issue, the authors propose Conditional Role Allocation (CRA), a framework that selectively assigns supervisory roles to CLIP, DINOv2, and MLLM based on sample reliability and existing supervision coverage. Specifically, CRA employs a three-stage training strategy that establishes reliable pseudo-labels using CLIP semantic confidence and DINOv2 structural information, expands supervision through class-wise selection, and applies complementary MLLM supervision to the remaining samples. Finally, the authors evaluate the proposed framework across closed-set, partial-set, and open-set SFDA settings to demonstrate its effectiveness.

Critical strengths
  1. The authors identify an important issue in existing SFDA research: additional supervision from foundation models is not uniformly beneficial across target samples. Through an empirical analysis of the Supervision-Position Effect, they demonstrate that not only the quality of supervision but also where it is applied matters.

  2. The proposed CRA progressively leverages semantic information from CLIP, structural information from DINOv2, and complementary knowledge from MLLM. The framework is well aligned with the identified problem, as it preserves reliable pseudo-labels while providing additional supervision for uncertain samples.

  3. The authors conduct experiments across diverse SFDA settings and evaluate the effectiveness of the proposed method through ablation studies and comparisons of supervision allocation across different sample regions.

Critical weaknesses
  1. The novelty of the supervision-position effect and the necessity of selective MLLM supervision are insufficiently justified. The observed difference in label correction may largely result from applying a more accurate MLLM to samples with less reliable CLIP pseudo-labels. Thus, whether this effect constitutes a distinct contribution beyond existing confidence-based sample selection strategies remains insufficiently established.

  2. The necessity of incorporating DINOv2 as an additional foundation model is not sufficiently justified. Its additional benefit over an otherwise identical variant without DINOv2 remains unclear, and the performance gains relative to the increased computational overhead are not adequately evaluated.

  3. The design of class-balanced expansion in Stage 2 is not sufficiently justified. The rationale for determining class-wise expansion budgets using DINOv2 structural information remains unclear. Furthermore, the benefit of constructing a separate expansion set (S2S_2) based on CLIP confidence margins, rather than directly using the structurally verified reference set (VV), is not adequately demonstrated.

Questions for authors
  1. Could the authors provide additional CRA results using different MLLMs across multiple domain transfers, beyond the Office-Home A→C setting in Table 7? This would help clarify whether the performance improvements primarily stem from the proposed supervision allocation strategy or the strength of the selected MLLM.

  2. Could the authors clarify why RCL, which also uses MLLM-based supervision and serves as a key baseline in the closed-set experiments, is excluded from the partial-set and open-set evaluations? If possible, could the authors provide RCL results under these settings to enable a more comprehensive comparison?

  3. The analysis of computational efficiency is insufficient. CRA employs multiple foundation models and multi-stage student training, but lacks a comprehensive comparison of training cost, and GPU usage with existing methods.

  4. Could the authors clarify the benefit of assigning different expansion budgets to each class in Stage 2? A comparison with uniform per-class expansion under the same total budget would help justify this design choice.

  5. Could the authors clarify why the CRA accuracy on Office-Home A→C is reported as 84.3% in Table 1 but 84.4% in Tables 5 and 6?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.