acceptodds
Under review as a conference paper at ICLR 2027

VarMatch: Variational Consistency and Test-Time Adaptation for Semi-Supervised Domain Generalization

Abstract

Semi-supervised domain generalization (SSDG) aims to generalize to unseen domains using only a few labeled examples per class and additional unlabeled data from multiple source domains. In this setting, limited supervision makes it difficult to learn reliable class structure, while domain shift can cause the learned structure to become misaligned with target data. We introduce VarMatch, a variational latent-space framework that combines stochastic consistency training with source-aware and frozen test-time adaptation. During training, pseudo-labels obtained from weakly augmented examples supervise two complementary views: an image-space augmentation and a variance-scaled stochastic perturbation of the learned latent representation. This encourages predictions to remain consistent under both input-level and representation-level variations. At test time, VarMatch keeps the encoder and classifier frozen, aligns target representations with source class structure, and estimates target-aware class prototypes from confident target samples while excluding samples that provide little evidence of domain shift. Adaptation operates without network-parameter updates and uses deterministic latent representations. Across PACS, OfficeHome, Digits-DG, and miniDomainNet with 5 and 10 labels per class, VarMatch improves over the strongest semi-supervised domain-generalization baseline by 2.37-6.22 percentage points. Its frozen, source-aware adaptation outperforms evaluated parameter-free alternatives and is competitive with methods that update the encoder.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
semi-supervised domain generalization, domain generalization, semi-supervised learning, test-time adaptation, variational information bottleneck, latent-space augmentation, pseudo-labeling, consistency regularization, prototype adaptation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_jMmYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes VarMatch, a variational latent-space framework for semi-supervised domain generalization. During training, the method uses weakly augmented samples to supervise two additional views: an image-level augmented view and a representation-level augmented view generated through variance-scaled stochastic perturbations in the latent space. At test time, the method further adapts to the target distribution through a test-time adaptation strategy while keeping the encoder and classifier parameters frozen.

Critical strengths

Uncertainty-aware latent augmentation: Perturbing latent features based on per-dimension uncertainty is an interesting approach to representation-level augmentation. This allows the magnitude of perturbations to be adaptively controlled based on the estimated uncertainty.

Parameter-free test-time adaptation: The proposed test-time latent transformation and prototype interpolation are insightful approaches for adapting to unseen target distributions without updating the model parameters.

Strong quantitative results: The proposed method achieves substantial improvements over prior works across multiple benchmarks. However, these improvements are obtained with test-time adaptation (TTA), which should be considered when comparing against methods that do not employ TTA.

Critical weaknesses
  1. Potential violation of the SSDG evaluation protocol: The main comparisons are against semi-supervised domain generalization (SSDG) methods, which assume that no information from the target distribution is available during training or adaptation. However, the proposed test-time adaptation (TTA) strategy assumes access to the entire unlabeled test set (or a batch of test samples) to estimate the target distribution. This introduces additional information that is unavailable to conventional SSDG methods, making the comparison potentially unfair. Therefore, VarMatch without TTA would be a more appropriate comparison against existing SSDG methods. While Table 2 provides results for the raw model, I suggest reporting results without TTA across all datasets and evaluation settings.

  2. Separating the contributions of SSDG and TTA: The proposed TTA strategy is an interesting additional contribution and could be evaluated separately. To establish its effectiveness and ensure a fair comparison, it would be valuable to apply the proposed TTA method to other SSDG baselines and report their performance improvements. This would help distinguish improvements arising from the proposed training framework from those introduced by test-time adaptation.

  3. Limited backbone evaluation: The experimental evaluation would be strengthened by including additional modern backbones, such as CLIP ViT and DINO. These experiments would help assess whether the proposed approach generalizes across different feature representations. In particular, comparing against CLIP's strong zero-shot performance would provide a useful reference for evaluating the effectiveness of the proposed method.

Questions for authors

The main concern is the mismatch between the proposed evaluation protocol and the conventional SSDG setting. Existing SSDG methods assume no access to the target distribution, whereas the proposed approach exploits unlabeled target samples through test-time adaptation. This raises concerns about the fairness of the main comparisons. Could the authors clarify how this protocol aligns with the standard SSDG setting? Addressing this issue may require revising the main experimental comparisons to emphasize results without TTA and potentially reconsidering the paper's title to better reflect the actual problem setting.

2 comments
AuthorsOpenReview ↗

Response to Reviewer jMmY

We thank the reviewer for the thorough review and the helpful comments.

Evaluation protocol and comparison without test-time adaptation- The reviewer is right that the test-time stage uses unlabeled target data, which the SSDG baselines do not. This stage is optional in VarMatch. We therefore report VarMatch without it, i.e., the trained model evaluated directly on the target data with no test-time mechanism, on all benchmarks and label budgets (mean ± SD over three seeds; best baseline from our Tables 1 and 2). It is above the strongest baseline in seven of eight settings and on par with it on miniDomainNet with 5 labels.

BenchmarkLabelsBest baselineVarMatch, no TTAVarMatch, full
PACS1081.582.02 ± 0.5386.32 ± 0.20
PACS579.880.89 ± 1.4185.46 ± 0.77
OfficeHome1060.061.83 ± 0.3965.55 ± 0.30
OfficeHome556.858.39 ± 0.3162.31 ± 0.40
Digits-DG1082.283.61 ± 0.3684.57 ± 0.49
Digits-DG576.781.19 ± 3.3482.92 ± 1.93
miniDomainNet1056.557.88 ± 0.2160.23 ± 0.39
miniDomainNet553.753.66 ± 0.0556.66 ± 0.27

We will add this "no TTA" row, which reflects our training stage alone, to Tables 1 and 2.

Training and test-time contributions- We also applied the test-time stage, unchanged, to FixMatch on OfficeHome, trained with the official StyleMatch code base (58.15 without TTA, close to the 57.7 reported for FixMatch in Table 1). In addition, to isolate our training stage, we applied it to a deterministic variant of VarMatch on PACS that removes the variance head, the KL term and the latent view, so that the latent code is a deterministic projection of the backbone features, while keeping every other component. The deterministic variant keeps the latent layer, so the test-time stage runs on it exactly as in VarMatch; FixMatch has no latent layer, so the stage runs on its backbone features.

ModelBenchmarkNo TTAWith our TTAGain
FixMatchOfficeHome 1058.1561.48+3.33
VarMatchOfficeHome 1061.8365.55+3.72
DeterministicPACS 1080.3885.48+5.10
VarMatchPACS 1082.0286.32+4.30

The test-time stage improves every model, and the variational training adds a further gain on top of it. The results in the table are mean of 3 seeds.

Alignment with the SSDG protocol- Our training follows the standard SSDG protocol: no target data, labeled or unlabeled, is used during training, and the model without the test-time stage is evaluated exactly as the SSDG baselines. The test-time stage is an additional, optional step that uses the unlabeled target set without labels and without updating any parameter, as in test-time adaptation methods such as T3A and LAME. In the revision we will report the two protocols separately in the main tables and state clearly which results use the test-time stage.

Additional backbones and CLIP- The baseline results we compare with are taken from UPCSC (Lee et al., CVPR 2025), and the SSDG methods they cover: StyleMatch (Zhou et al., IJCV 2023), FBC-SA (Galappaththige et al., CVPR 2024) and UPCSC, all use ResNet-18 on these benchmarks, so we followed the same setting to keep the comparison direct and fair. CLIP and DINO are pretrained on very large web data that likely already covers the styles in PACS and OfficeHome, so their zero-shot accuracy does not measure generalization to unseen domains under the few-label setting that SSDG studies.

The title already names test-time adaptation as part of the method.

My main concern regarding the SSDG setting versus TTA has been adequately addressed. I encourage the authors to clearly incorporate these results into the revised manuscript. I will maintain my original rating of Weak Accept.

Reviewer_eFbnOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper addresses semi-supervised domain generalization (SSDG). In this setting, methods use a few labeled source samples, together with unlabeled source data, to generalize to an unseen target domain. The authors propose VarMatch, a variational latent-space framework that combines stochastic consistency training with source-aware, frozen test-time adaptation (TTA). At test time, VarMatch aligns target representations with the source class structure and estimates target-aware class prototypes from confident target samples. The method is evaluated on PACS, OfficeHome, Digits-DG, and miniDomainNet with 5 and 10 labels per class.

Critical strengths

The variational posterior mechanism is an interesting idea, since the same latent representation is used both for training-time consistency and for test-time alignment.

The authors introduce a frozen, source-aware test-time correction, and they report that it outperforms the parameter-free TTA methods they evaluate.

Critical weaknesses

I wonder why the semi-supervised DG setting is needed, since many works address fully unsupervised DG. Given the growing scale of work aimed at generalizing neural networks, this does not seem to me a particularly interesting direction to pursue. I am open to reading the authors' opinion on this.

In their related work, I didn't find papers that also use labels for Domain Generalization

Although the method builds on FixMatch, why do the authors choose to rely on labeled source data at all, and why use unlabeled target data at test time?

Questions for authors

The baselines do not seem recent to me. Several of the included methods are quite old, which makes the comparison less comprehensive.

I encourage the authors to demonstrate their method without any labels: take the most confident predictions as pseudo-labels, assume they are true labels, and test the method in that setting.

How much of the improvement comes from the few labeled samples and how much from the unlabeled data?

Why was only one backbone used? I would encourage evaluating at least two.

Moreover, I would like to question the '(Domain) Generalization' theme here. If the method ends up using labels during adaptation why not frame it as Partial Domain Adaptation?

2 comments
AuthorsOpenReview ↗

Response to Reviewer eFbn

We thank the reviewer for the detailed comments and for being open to our view on the setting.

Why semi-supervised DG- In standard DG every source sample is labeled, so most DG methods rely on labels heavily. Section 2.3 covers such methods (ERM, IRM, MetaVIB, IIB, RSC, SWAD), which assume fully labeled source domains. Unsupervised DG, e.g., in "Towards Unsupervised Domain Generalization", presented at CVPR 2022, removes labels from pretraining, but still needs labeled data to train the final classifier. SSDG sits between the two: the source domains are available, but only a few samples per class are labeled. This is a practical case, for example in medical settings, since collecting data from several domains is easy while labeling all of them is expensive or cannot be achieved. It is also an established setting with its own benchmarks and methods, including StyleMatch (Zhou et al., IJCV 2023), MultiMatch (Qi et al., ACM 2024), FBC-SA (Galappaththige et al., CVPR 2024) and UPCSC (Lee et al., CVPR 2025).

Usage of labels and target data- The few source labels are what defines the classes. Without any labels the model can group samples, but it cannot know which group is which class. At test time VarMatch uses the unlabeled target test batch together with the source training data, with frozen parameters and no target labels, as in parameter-free test-time adaptation methods such as T3A and LAME. No target data is used during training, and source and target share the same classes. Partial domain adaptation assumes access to target data during training and a target label space that is a subset of the source one, so neither assumption applies here.

The DG claim also holds without any test-time stage. Without TTA, VarMatch is above the best reported baseline in 7 of the 8 settings and ties on the eighth (miniDomainNet, 5 labels: 53.66 vs. 53.7).

Baselines- All baseline figures are taken from UPCSC (CVPR 2025), the most recent work that evaluates every SSDG method under a single protocol. They include the two most recent plug-in methods, FBC-SA (CVPR 2024) and UPCSC, each attached to FixMatch, FreeMatch and StyleMatch.

Contribution of labeled and unlabeled data- With no labels at all the task becomes clustering or unsupervised DG, which is evaluated under a different protocol, so we keep the SSDG setting. The contribution of the unlabeled data can be read from Table 1: ERM, trained on the labeled samples only, reaches 60.2 on PACS (10 labels), FixMatch, which adds the unlabeled data, reaches 76.8, and VarMatch reaches 82.0 without TTA (Table 4). The same holds on every benchmark (e.g., Digits-DG: 60.8 for ERM vs. 83.6 for VarMatch without TTA, see our response to Reviewer jMmY). Doubling the labels from 5 to 10 per class raises VarMatch only from 80.9 to 82.0. With the labels fixed, most of the gain comes from how the unlabeled data is used.

Backbone- ResNet-18 is the standard backbone in SSDG (StyleMatch, FBC-SA, UPCSC), which lets us compare directly with the reported results. Large pretrained models such as CLIP change the problem settings, since their pretraining data may already cover the target domains. The test-time stage is also not tied to our encoder: applied to a FixMatch model on OfficeHome with 10 labels it raises accuracy from 58.15 to 61.48.

Thanks for your efforts and responses to my question. Regarding semi-supervised domain generalization, why don't the authors cite any works for the sentence "We follow the standard SSDG protocol"? Citing here would have made it much clearer. These were hurdles in my reading earlier. Thanks for stating this and quoting Zhou et al., 2023.

I understand that the authors would like to stick to this setting because they could estimate their main equation 7. CLIP is not ideal; however, backbones such as ViT are also feasible.

In response, I have decided to improve the score. That said, I encourage the authors to improve the paper's overall presentation.

Reviewer_7GB9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper focuses on Semi-Supervised Domain Generalization (SSDG), addressing two challenges: the difficulty of learning reliable class structures due to limited labeled data and the degradation of classification performance caused by domain shifts in unseen target domains. To address these challenges, the authors propose VarMatch. During training, VarMatch performs latent-space consistency learning through Variance-Scaled Resampling (VSR), which utilizes the variance of the variational posterior and prediction confidence. At test time, Latent Transformation (LT) and Test-Time Prototype Interpolation (TT-PI) are employed to adapt to the target domain without updating network parameters.

Critical strengths
  1. The authors introduce VSR-based latent consistency learning to improve unstable feature representations caused by limited labeled data. At test time, they combine LT and TT-PI to align target features and refine class prototypes.

  2. The proposed framework integrates source training and test-time adaptation, enabling adaptation to the target domain without updating network parameters at test time.

Critical weaknesses
  1. Insufficient validation of the key design components of VSR. Although Table 3 evaluates the contribution of VSR through an ablation study that removes the entire VSR component, the individual effects of the learned posterior variance and confidence-based perturbation scaling are not separately analyzed. Therefore, the necessity of each design component and its additional benefits over simple feature perturbation remain unclear.

  2. Limited evaluation scope of the Ablation study. Since VSR relies on pseudo-label confidence and the learned posterior standard deviation, its effectiveness may vary depending on the number of classes, domain characteristics, and label budget. However, the VSR removal ablation is only reported on PACS with 10 labels per class, making it difficult to determine whether VSR consistently improves performance across different datasets and label budgets.

Questions for authors
  1. Could the authors provide additional ablation studies comparing the use of fixed Gaussian perturbation instead of the learned posterior standard deviation and the removal of confidence-based perturbation scaling? These experiments would help clarify whether the performance improvements arise from simply introducing latent perturbation and consistency learning or from the specific contributions of the proposed sample-specific posterior and confidence-based scaling.

  2. Could the authors conduct the same ablation studies on OfficeHome, Digits-DG, and miniDomainNet, as well as under the 5 labels/class setting? In particular, analyzing not only target accuracy but also pseudo-label accuracy and the Fisher ratio across these settings would help determine whether the effectiveness of VSR remains consistent across different numbers of classes, domain characteristics, and label budgets.

2 comments
AuthorsOpenReview ↗

Response to Reviewer 7GB9

We thank the reviewer for the focused questions on VSR.

W1 + Q1: Learned posterior vs. fixed Gaussian, and confidence scaling- On PACS (10 labels, seeds 1-2), a fixed Gaussian perturbation (keeping the confidence scaling) performs comparably to VSR when its magnitude is matched to the average VSR perturbation (86.9 vs. 86.4 with the test-time stage). However, this constant was measured from a trained VarMatch model on PACS. The matched constant is also dataset specific: measured the same way on Digits-DG it is 1.48, 2.6 times the PACS value (0.56). As a method, a fixed noise level would have to be set separately for every dataset: either by first training VarMatch to measure it, which means two training runs per dataset, or by searching over many noise levels for each dataset without access to the target domain, which adds many runs per dataset and removes the benefit of a single recipe across benchmarks. VSR avoids both: a single scale s0=4s_0=4 is used on all four benchmarks, and the posterior adapts the perturbation magnitude to each dataset and sample within one training run.

The posterior also contributes beyond the variational view. Compared to a deterministic variant with the same training (both without the variational view), which removes the variance head and the KL term, it adds +0.93±0.09 points on PACS with the test-time stage (three seeds, positive in every seed). The full VarMatch model is 1.6 points above the deterministic variant without the test-time stage and 0.8 points with it, in every seed.

Removing the confidence scaling (ω=0\omega=0) has little effect on the PACS average (81.7 vs. 81.5 without the test-time stage, 86.4 for both with it), but on Sketch, the target with the largest shift, it lowers accuracy in both seeds (by 1.5 and 1.8 points without the test-time stage). Without the scaling, low-confidence samples receive the same perturbation as confident ones, which can push them across the decision boundary and hurt their pseudo-labels. This matches its role of limiting the perturbation of uncertain samples, which are most frequent under large shifts.

W2 + Q2: Removing the variational view on other datasets and label budgets- We removed only the variational view, keeping all other training components active, on PACS (10 and 5 labels), OfficeHome and Digits-DG. The table reports target accuracy, pseudo-label accuracy on the unlabeled source set, the fraction of unlabeled source samples predicted with confidence ≥0.95\ge 0.95 (a fixed threshold for all variants, used only for this diagnostic), and the Fisher ratio (ground-truth labels, latent space).

BenchmarkSeedsVariantNo TTAFullRetainedPL acc.Fisher
PACS 103VarMatch82.0286.3297.0692.542.45
− variational view81.6186.4294.3191.731.85
PACS 52VarMatch81.0685.5396.8187.272.32
− variational view80.6085.9293.6187.231.77
OfficeHome 103VarMatch61.8365.5575.9675.411.36
− variational view61.6465.3367.4874.591.21
Digits-DG 102VarMatch83.7084.7099.0396.076.27
− variational view80.7682.0298.7493.995.86

On all four settings the variational view raises pseudo-label retention and the Fisher ratio, and pseudo-label accuracy is higher on all four (marginally on PACS 5). On Digits-DG it also improves target accuracy, by 2.9 points without the test-time stage and 2.7 with it (+1.2 and +4.1 in the two seeds); on PACS and OfficeHome the effect on target accuracy is within seed variation.

The variational view is also not redundant with pixel-level augmentation. On PACS, removing pixel augmentation keeps the Fisher ratio unchanged (2.44 vs. 2.45) but lowers pseudo-label accuracy by 4.6 points, while removing the variational view keeps accuracy but lowers the Fisher ratio by about 25%. The two views play complementary roles.

We can include both ablations in the paper's appendix section, and extend the latent-view ablation in Table 3 to the additional datasets and label budgets shown above.

We thank the authors for their detailed responses and additional ablation experiments. The authors have provided new experimental results addressing our concerns, which have helped clarify the design components and role of VSR.

However, the additional experiments show that Fixed Gaussian Perturbation achieves comparable target accuracy to VSR, leaving the additional benefits of learned posterior-based perturbation over Fixed Gaussian Perturbation unclear. The authors argue that dataset-specific noise scale tuning is necessary for Fixed Gaussian Perturbation, based on the different average VSR perturbation scales measured on PACS and Digits-DG. However, they have not experimentally verified whether applying the same fixed Gaussian noise scale across multiple datasets actually leads to performance degradation. Therefore, while we acknowledge the practical advantage of VSR in reducing dataset-specific noise scale tuning, we believe that further evidence is needed to establish the necessity of such tuning and the additional benefits of VSR over Fixed Gaussian Perturbation.

We appreciate the authors' additional experiments and explanations. Nevertheless, we believe that our concerns regarding the additional benefits of learned posterior-based perturbation over simpler alternatives have not been sufficiently addressed. Therefore, we maintain our original rating of Weak Rejection.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.