acceptodds
Under review as a conference paper at ICLR 2027

Joint Contrastive-Adversarial Representation Learning for Unsupervised Domain Adaptation

Abstract

Effective unsupervised domain adaptation (UDA) requires a representation that captures transferable structure from both the labeled source domain and the unlabeled target domain. We introduce Joint Contrastive-Adversarial Representation Learning (JCAR), a two-stage framework that learns a common representation from the union of unlabeled source and target samples by combining contrastive learning with adversarial domain alignment, and then freezes the representation to fit a source-supervised predictor. We establish a non-asymptotic excess-risk bound for this procedure that couples representation and domain-discriminator estimation while separating downstream prediction, with explicit neural-network approximation errors. Under the stated assumptions and common H\"older smoothness , the bound identifies a sufficiently large unlabeled-sample regime in which the downstream rate is , where counts labeled source samples and logarithmic factors are suppressed. The labeled-sample exponent is governed by the latent dimension rather than the ambient dimension , which enters the pretraining cost. Experiments on Office-31, Office-Home, and DomainNet demonstrate clear transfer gains across diverse domain shifts, with particularly strong performance when representations are learned from scratch.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Domain adaptation, Representation learning, Contrastive learning, Non-asymptotic error bound

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_Y4tnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes JCAR, a two-stage UDA method that jointly learns a shared representation from unlabeled source and target data via contrastive learning and adversarial alignment, then freezes it for source-supervised prediction. However, the experiments do not support the abstract’s claimed transfer gains.

Critical strengths

None

Critical weaknesses

Weaknesses 1: The main-paper results do not match the gains claimed in the paper: despite comparing only with traditional baselines such as ERM[1] (1991), DANN[2] (2015) and CDAN[3] (2018), JCAR remains substantially below all baselines on Office-Home (table 2).

Weaknesses 2: The appendix exceeds 20 pages while the main text is only 8 pages, with most experiments relegated to the appendix, leaving the main-paper evaluation highly insufficient.

[1]Vapnik V. Principles of risk minimization for learning theory[J]. Advances in neural information processing systems, 1991, 4.

[2]Ganin Y, Lempitsky V. Unsupervised domain adaptation by backpropagation[C]//International conference on machine learning. PMLR, 2015: 1180-1189.

[3] Long M, Cao Z, Wang J, et al. Conditional adversarial domain adaptation[J]. Advances in neural information processing systems, 2018, 31.

Questions for authors

Why are the from-scratch results so low? Could the authors clarify whether this is due to an issue with the experimental setup or training bugs?

Reviewer_yKbaOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes joint contrastive-adversarial represention learning. It is a two-stage UDA procedure that first learns a shared feature representation with unlabeled source and target samples, and then conducts source-supervised learning with frozen feature extractor. A non-asymptotic excess-risk bound is established. Experiments are conducted on a few UDA benchmarks, demonstrating transfer gains.

Critical strengths
  • The paper is a theoretical work. It attempts to establish a non-asymptotic error bound that explictly considers feature extractor and adversarial domain discriminator. The analysis is useful to understand the proposed framework.
Critical weaknesses
  • On the method side, the novelty is weak as it mainly combines contrastive learning with adversarial domain alignment. In UDA, it is common to learn feature representation jointly from labeld source data and unlabeled target data. The paper does not show why learning from unlabeled data only is superior. In terms of the empirical results, the comparison is also limited to some a few very old UDA methods. The practical application of this framework is questionable.
Questions for authors
  1. Could authors explain why phase one uses unlabeled data only?
  2. How robust is this strategy in terms of backbone capacity and domain gap? Tables 1-2 do not include comparison methods for ViT.
Reviewer_qxM2OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper implements JCAR, a two-stage DA approach to unsupervised domain adaptation. In the first stage it combines contrastive learning on unlabeled source and target samples with adversarial domain alignment to train a base network. The resulting network is frozen, and a classifier is trained using labeled source examples.

Critical strengths

a. Potential Theoretical contributions - I could not thoroughly verify the theoretical contributions. My decision is primarily based on the model and experimental results. My decision will not be affected even if the theory is sound and accurate.

Critical weaknesses

a. Limited Novelty - The proposed method combines established contrastive learning with DANN-based adversarial domain alignment. Similar combinations have already been explored, for e.g., Adversarial Domain Adaptation Using Contrastive Learning (2023) (https://dl.acm.org/doi/10.1016/j.engappai.2023.106394) and CDA: Contrastive-adversarial Domain Adaptation (2023) (https://arxiv.org/abs/2301.03826). Shen et al. (already cited in the paper https://proceedings.mlr.press/v162/shen22d.html) shows contrastive representations can support successful domain adaptation without the need for domain invariance. Given this prior work, the methodological novelty appears incremental.

Questions for authors

a. How does JCAR compare against the same joint-domain contrastive pretraining procedure without DANN adversarial alignment? Is there a direct comparison with Shen et al. 2022, using a similar backbone network or training budget or evaluation protocol?

Comments on the paper

1 comment

Dear reviewers and authors,

With limited time left for discussion, please keep an eye on the newest replies and comments and deal with any outstanding concerns promptly. Thank you!

AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.