Testing the domain invariance hypothesis in low target data Domain Adaptation

Out-of-Distribution (OOD) generalization aims to ensure robust model performance when a target distribution differs from the source distribution observed during training. Within this broader problem, Domain Generalization (DG) considers the challenging setting where no target-domain data are available during training, whereas Domain Adaptation (DA) leverages target-domain information to facilitate generalization to the target domain. In practice, the transition between DG and DA depends on the amount of target data and is particularly relevant when only a few target samples are available. We therefore revisit this dichotomy by introducing a unified formulation that treats DG and DA as continuum governed by the amount of available target data. To this end, we extend the experimental protocol DomainBed designed for DG experiments to Unsupervised Domain Adaptation (UDA) and Supervised Domain Adaptation (SDA), and empirically investigate how adaptation occurs through domain discrepancy and decision boundaries evolutions. We investigate the role of the domain-invariant hypothesis across the transition from DG to DA. If domain invariance contributes to generalization, greater domain invariance should be systematically associated with improved target-domain performance. We therefore compare statistical (MMD) and adversarial (DANN) alignment with alignment-free ERM under varying target-data availability, allowing us to examine whether alignment systematically translates into improved target performance along the proposed critical dimension. To assess the generality of our findings across modalities, we conduct experiments on the CWWS speech dataset, which includes various types of corruptions, and the popular VLCS image dataset. Across both data modalities, we observe comparable overall trends, while the supervised and unsupervised adaptation continuums exhibit distinct behaviors as target-data availability increases. Explicit domain-alignment does not outperform classical ERM in DG$\to$SDA while they reach early performance plateaus in DG$\to$UDA despite target error reduction in all scenarios. Quantitative analysis identifies the factors that account for these differences and assesses the empirical validity of the domain-invariant hypothesis in our experimental setting. The code will be publicly released upon publication of the paper.