acceptodds
Withdrawn from ICLR 2027

SARA: Semi-Supervised Multi-Domain Learning of Vision-Language Models without Test-Domain Labels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Vision-language models (VLMs) such as CLIP provide a promising foundation for multi-domain learning (MDL) by introducing rich multimodal semantics and transferable pretrained knowledge. However, existing MDL methods that preserve domain-specific differences often rely on test-domain labels to select specialized parameters or computation paths. Without such labels, domain-specific information can provide useful cues for some samples but induce misleading prediction variation for others. We refer to changes in such information for a fixed image as domain perturbations: some predictions remain stable, whereas others become unstable and may even undergo prediction flips. Moreover, most MDL approaches rely heavily on labeled data, while existing semi-supervised learning methods do not explicitly account for such domain-specific prediction variation, which can aggravate confirmation bias during self-training. To address these challenges, we propose Stability-Aware Routing and Adaptation (SARA), a CLIP-based semi-supervised MDL framework without test-domain labels. SARA assesses sample stability under domain perturbations through predictive consistency and decision margin, and consists of two coordinated components: (i) Stability-Guided Self-Training applies a conservative stability assessment to filter unreliable pseudo-labels; and (ii) Stability-Based Prediction Routing exploits domain-specific information for stable samples while reducing its influence on unstable samples through sample-specific prediction mechanisms. Extensive experiments on four multi-domain benchmarks demonstrate consistent improvements over MDL and SSL baselines, including a 1.39% average gain over the state of the art.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
multi-domain learning, semi-supervised learning

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_X2y8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

To address semi-supervised multi-domain image classification, where domain labels are available during training but not at inference, this paper proposes stability-aware routing and adaptation (SARA) that introduces shared and domain-specific prompts into CLIP and assesses sample stability using predictive consistency across domain-specific streams. This guides pseudo-label selection during training and routing between weighted feature aggregation and a shared classifier at inference. The experimental results demonstrate the effectiveness of the proposed method.

Critical strengths
  1. Clear motivation -- The observation that some images receive consistent class predictions across domain-specific streams while others receive conflicting predictions is interesting. This clearly motivates the use of cross-stream stability to assess the pseudo-label reliability.

  2. Technical Novelty -- Using cross-domain predictive stability to guide pseudo-label selection and inference-time routing is an interesting and novel idea. This method is also consistent with the motivation.

Critical weaknesses
  1. Additional inference cost -- The proposed method needs to process test images through domain-specific streams, which can increase the computational cost significantly. This could limit the practicality of the method.

  2. Dependence on predefined domain partitions -- The proposed method relies on predefined discrete domain labels during training and uses them to construct domain-specific streams. However, in real-world scenario, domain can be more ambiguous, and merging or splitting similar domains requires changing the leaned streams. It remains unclear how sensitive the method is to these changes.

Questions for authors
  1. How sensitive is the proposed method to the predefined domain partition? For example, how would merging similar domains (e.g., painting and sketch from DomainNet) affect the stability assessment and classification performance?

  2. Could parallel evaluation of the domain-specific streams mitigate the inference overhead?

Reviewer_8Z3UOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SARA, a CLIP-based framework for semi-supervised multi-domain learning without requiring test-domain labels at inference. It assesses sample stability through predictive consistency and decision margin, selectively leveraging domain-specific information for stable samples while reducing its influence on unstable samples. Experiments on four benchmarks show consistent improvements over existing MDL and SSL baselines, with an average gain of 1.39 percentage points over state-of-the-art baselines.

Critical strengths

The paper addresses a meaningful limitation of existing multi-domain learning methods: domain-specific information can provide useful discriminative cues, but selecting a domain-specific model or computation path is challenging when test-domain labels are unavailable. By focusing on sample-dependent reliability rather than relying exclusively on a fixed domain assignment, SARA offers a practical direction for adapting vision-language models to heterogeneous data distributions.

Critical weaknesses

The methodological innovation of the SARA framework proposed in this paper remains somewhat unclear. First, the concepts of predictive consistency and decision margin overlap to some extent with existing methods, such as confidence-based pseudo-label filtering and disagreement-based uncertainty estimation. The authors have not sufficiently clarified the unique advantages of their proposed stability assessment over these existing methods. Second, stability-based prediction routing assigns samples to different prediction branches according to their estimated stability. Could this sample-dependent prediction mechanism be regarded as a variant of the Mixture-of-Experts paradigm or an adaptive expert routing strategy? The authors are encouraged to clarify the fundamental differences between SARA and existing pseudo-label selection and adaptive expert-routing methods, while explicitly detailing the unique methodological contributions of the proposed stability assessment.

Questions for authors

1.What are the fundamental theoretical and methodological differences between the proposed predictive consistency, decision margin, and stability-based prediction routing mechanisms and existing methods, such as confidence-based pseudo-label selection, disagreement-based uncertainty estimation, and adaptive expert routing? The authors are encouraged to clarify the unique contributions of the proposed method and explain its advantages over existing methods.

2.The ablation studies compare the proposed pseudo-label selection strategy with Random, FixMatch, and Consistency-only variants. How does the proposed strategy perform in comparison to some representative dynamic-threshold pseudo-label selection methods?

3.The paper emphasizes that SARA does not require test-domain labels during inference. Some existing domain adaptation methods also operate without requiring domain labels at test time. Have the authors considered comparing SARA with state-of-the-art domain adaptation methods?

4.The experiments primarily adopt a fixed two-shot setting, in which only two labeled training samples per class are used within each domain. Does SARA maintain its performance advantage under other few-shot settings?

Reviewer_mfM5OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This work address the multi-target domain adaptation problem without domain labels during inference. It propose Stability-Guided Self-Training (ST), evaluating unlabeled data using predictive consistency and decision margin; Stability-Based Prediction Routing (PR), predicting stable samples via a weighting network while unstable samples are predicted by a shared classifier only with the non-sensitive channel at test time.

Critical strengths
  • the paper is easy to follow
Critical weaknesses
  • SARA's setup is not functionally different from domain-agnostic multi-target DA at test time.
  • the author omit the obvious baseline: what happens if you simply take existing SFDA experts for each target domains and ensemble at inference? overall, SARA does not solve the fundamental problem; it feels like an ensemble of domain-specific adapters with a gating and filtering mechanism.
  • using variance ratios of visual features to decide "domain-sensitive" and "complementary" dimensions is not theoretically grounded; visual representations are highly non-linear, and dependent across multiple entangled dimensions; high variance channel may also encode critical class-discriminative signals due to cross-channel feature interaction.
  • during inference, the author deploy weighted average of visual feature based on learned weight; however, the feature representations are not necessarily convex, a point halfway between two visual feature may not be a valid, meaningful visual representation; I wonder why not deploy the weighted average over the output probability.
  • too many hyper-parameters which can be sensitive across different tasks.
  • pure heuristic design; lack of theoretical analysis
Questions for authors

see weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.