acceptodds
Under review as a conference paper at ICLR 2027

Learning Subset-Shared Invariances for Domain Generalization with Mixture-of-Experts

Abstract

Domain generalization (DG) aims to learn a model from one or more source domains that generalizes to an unseen target domain without accessing target data during training. A common approach enforces invariance of representations across all source domains, assuming predictive structure is globally shared. However, exact conditional invariance can exclude predictive factors shared only within subsets of source domains. To address this limitation, we propose subset-shared invariance, where predictive structure is assumed stable only within domain subsets. We implement this principle with a mixture-of-experts architecture, where routing induces expert- and class-specific alignment strengths across source-domain pairs and softly composes the resulting expert representations for prediction. This creates routing-conditioned alignment that prioritizes domain relationships according to learned responsibilities rather than enforcing them uniformly. To facilitate effective decomposition, we develop training objectives that encourage selective alignment, confident and balanced routing, and diverse expert specialization. Experiments on DomainBed benchmarks demonstrate improved out-of-domain generalization and greater robustness under increasing domain heterogeneity. Our results suggest that DG should move beyond enforcing a single global invariance and instead model invariance through partially shared structure across domain subsets.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Domain Generalization, Mixure of Experts, Subset-Shared Invariances

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_tsvNOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a novel method MESSI, a data-driven Domain Generalization (DG) Mixture-of-Experts (MoE) that replaces global invariance with subset-shared invariance.

Compared with the existing state-of-the-art method GMoE, which tries to learn specialized feature representations, such as texture and shape, this paper performs data-driven selective source-domain alignment at the subset level. This method also surpasses GMoE in performance.

The paper also provides a theoretical analysis of the information loss caused by enforcing globally invariant representations across all domains, and discusses the necessity of subset-shared invariance.

However, I am concerned that whether such a data-driven mechanism may fail when the source and target gap is large; the paper exhibits the same limitation, as its performance degrades when the number of source domains is limited (e.g., ks=3 in table 14).

Critical strengths

Strength1:The paper provides a theoretical analysis of the trade-off between domain invariance and information loss, showing that enforcing global conditional invariance can shrink the feasible representation class. It then proposes a novel principle, subset-shared invariance, to preserve such partially shared predictive structure.

Strength2: The authors realize this principle with a routing-based Mixture-of-Experts (MoE) framework that performs data-driven subset alignment without explicit subset labels, and empirically surpasses GMoE and other baselines under increasing domain heterogeneity.

These are the primary reasons for my score.

Critical weaknesses

Weaknesses1: This data-driven MoE alignment may not constitute true domain generalization; it appears closer to training multiple domain-specific experts and then matching inputs to the appropriate expert via data statistics. Compared with GMoE, which learns potentially transferable representations, the latter may generalize better when the data gap is large.

Weaknesses2:The performance gain over GMoE is marginal. As shown in Table 1, MESSI-S-MMD (64.3) is only 0.1% higher than GMoE (64.2). A more noticeable improvement (66.2) is achieved only when replacing Maximum Mean Discrepancy (MMD) with Optimal Transport (OT), which is about 6× more expensive than GMoE, while MMD is still about 2× more expensive (Table 5). This suggests that the method may be unstable and that its gains depend on a precise alignment metric, raising concerns about its practical value and robustness.

Questions for authors

Could the authors design an experiment where the target domain has a large discrepancy from all source domains, and report how MESSI behaves in that setting?

3 comments

Response to the Concern Raised in the Final Paragraph of the Paper Summary: Robustness to Limited Source Diversity and Large Source–Target Gaps.

We thank the reviewer for raising this concern. We would like to clarify that the low accuracy at Ks=3K_s=3 in Table 14 is not specific to MESSI. As reported in Appendix D.3, all evaluated methods, including CORAL across all penalty strengths, perform near chance level (approximately 50.8%). With only three source environments, the available diversity may be insufficient to distinguish digit-shape information from spurious color correlations. Notably, when KsK_s increases to 5, MESSI is the only method that substantially exceeds chance-level accuracy (57.1% versus at most 50.9% for the baselines), and it maintains the highest accuracy across all larger source-domain settings. These results suggest that while limited source diversity remains challenging, MESSI benefits effectively from additional source environments.

Q1. Domain-specific expert specialization versus transferable invariance

A1: MESSI’s experts are trained to be shared across domains. The router never sees domain labels (Sec. 4), and the alignment loss (Eq. 20) pulls together each expert’s features for the same class across the source domains that use it. An expert that behaves differently across domains is therefore penalized, whereas GMoE is trained with classification alone and has no such constraint. Fig. 4 confirms this sharing, since each source domain spreads its inputs over several experts (at most 48% on any one) and different domains rely on the same experts. Although each input goes mostly to one expert (Table 17), that expert is trained to be consistent across all the domains it serves. Our response to Reviewer b71J (W2) shows that this alignment, rather than the MoE head, drives the gain.

Behavior under large shifts. If MESSI only matched inputs to domain-specific experts, its advantage would shrink on distant targets. It grows instead. On the hardest target of each DomainBed dataset (lowest GMoE accuracy, App. D.2), MESSI-S-OT beats GMoE by 7.4 points on PACS-Sketch, 3.2 on OfficeHome-Clipart, 2.1 on TerraInc-L38 and 0.9 on DomainNet-Quickdraw. On iWildCam, the gain is larger on unseen cameras (+6.5 F1) than on seen ones (+3.9).

To answer your question directly, we evaluate targets increasingly far from all sources. All methods share the DeiT-S/16 backbone and are trained once on the same Rotated-Colored MNIST sources (App. C.2), with the five targets held out.

Table E. Rotated-Colored MNIST, OOD accuracy (%), mean ± std over 3 seeds. Min. PAD is the smallest proxy A-distance between the target and any source environment, so a larger value means a target further from all sources.

TargetMin. PADERMGMoEGlobal-MoEMESSI
θ=45∘\theta=45^\circ, p=0.6p=0.60.2564.7 ±\pm 0.767.5 ±\pm 0.666.3 ±\pm 0.571.1 ±\pm 0.4
θ=45∘\theta=45^\circ, p=0.8p=0.80.6547.3 ±\pm 1.153.6 ±\pm 0.851.2 ±\pm 0.965.0 ±\pm 0.7
θ=45∘\theta=45^\circ, p=1.0p=1.01.0330.2 ±\pm 1.440.5 ±\pm 1.237.4 ±\pm 1.056.2 ±\pm 0.9
θ=135∘\theta=135^\circ, p=0.5p=0.50.7460.0 ±\pm 0.863.4 ±\pm 0.762.2 ±\pm 0.968.0 ±\pm 0.6
θ=180∘\theta=180^\circ, p=0.5p=0.51.1451.7 ±\pm 1.255.6 ±\pm 1.054.2 ±\pm 1.162.3 ±\pm 0.8

MESSI scores higher than the other methods on all five targets, and the gap to GMoE grows as the target moves further from the sources. When the link between color and label is reversed, the gap goes from 3.6 to 15.7 points, and under rotations not seen in training it is 4.6 and 6.7 points. We will add Table E to App. D and the expert-sharing statistics to App. E.4. In Sec. 2, we will also explain that GMoE gets its transferability from its architecture, while MESSI asks for it directly through alignment.

Q2. Marginal performance gains, alignment-metric dependence, and stability

A2: We agree that MESSI-S-MMD is only slightly ahead of GMoE on the DomainBed average (64.3 vs. 64.2). However, our claimed contribution is the way routing decides which domain pairs to align. A fair test therefore keeps the distance measure fixed and changes only how the pairs are weighted. Global-OT is the Global-MoE control from Table 4, and Global-MMD is a new run.

Table F: Controlled alignment ablation on PACS and OfficeHome (accuracy, %).

MethodPACS ↑OfficeHome ↑
GMoE87.3 ±\pm 0.173.5 ±\pm 0.1
Global-MMD88.4 ±\pm 0.573.2 ±\pm 0.4
MESSI-S-MMD89.3 ±\pm 0.674.0 ±\pm 0.3
Global-OT87.1 ±\pm 0.273.8 ±\pm 0.6
MESSI-S-OT90.9 ±\pm 0.776.1 ±\pm 0.2

With the distance measure held fixed, routing-based alignment beats uniform alignment on both PACS and OfficeHome. The margin is 3.8 and 2.3 points with OT, and 0.9 and 0.8 points with MMD. The gain therefore does not come only from replacing MMD with OT, although OT gives the larger improvement in our setting.

The results are also stable. MESSI-S-OT beats GMoE on all four DomainBed datasets, with seed standard deviations between 0.2 and 0.7. In the sensitivity study of App. E.2, every setting that keeps the routing sparsity term stays above GMoE's PACS accuracy of 87.3, with the lowest at 88.8. This holds from 2 to 16 experts and across the full range of routing weights we tried. Halving the number of Sinkhorn iterations also lowers the average accuracy only from 66.2 to 66.0.

Our response to Reviewer 4cfG (W3) gives the full timing. Compared with GMoE, training takes about 2.1 times as long with MMD and 6.1 times as long with OT. Using 25 Sinkhorn iterations brings OT down to 3.3 times while keeping 66.0 average accuracy. The extra cost applies only during training, and at test time MESSI is faster than GMoE (61.4 vs. 74.2 in Table 5).

Finally, each DomainBed dataset has only four to six domains, which limits how much partial sharing between domains can matter. On iWildCam, with 243 source cameras, MESSI reaches 33.2 OOD macro-F1 against 26.7 for GMoE. The gap widens as more source cameras are added (Table A in our response to Reviewer b71J, W3). We will add Table F, the results with fewer Sinkhorn iterations, and a timing row for ERM with the same backbone to the revised paper.

We sincerely thank you for the time and effort you have devoted to reviewing our work. We would greatly appreciate your thoughts on our response and the additional results, particularly whether any concerns still need clarification. We would be happy to discuss any remaining questions and would value your further feedback as we refine the manuscript.

Reviewer_4cfGOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes MESSI, a mixture-of-experts framework for domain generalization based on the idea of subset-shared invariance. The authors argue that enforcing global conditional invariance across all source domains can remove predictive information that is shared only by subsets of domains. MESSI addresses this by using a router to determine expert-specific alignment strengths for class-conditional feature distributions across source-domain pairs. The training objective combines classification loss, routing-conditioned entropic optimal transport, and regularizers for sparse routing, balanced expert utilization, and expert diversity. Experiments on four DomainBed benchmarks show the effectiveness of this method.

Critical strengths
  1. The observation that predictive structure may be shared by only some source domains is reasonable and motivates moving beyond uniform global alignment. This is a meaningful problem for domain generalization, particularly when the source domains exhibit heterogeneous relationships.

  2. The overall framework is reasonable and well-motivated. The method is designed around the proposed principle rather than simply adding more experts.

  3. The evaluation covers several relevant aspects of domain generalization. In addition to the standard DomainBed benchmarks, the paper examines robustness as source-domain heterogeneity increases and evaluates a many-domain setting using iWildCam. The same-backbone comparisons and the ablations on alignment, routing, and expert diversity make the empirical analysis more informative than a comparison against standard ERM alone.

Critical weaknesses
  1. The routing mechanism is not directly optimized by the alignment loss to select useful domain relationships. In Equation (20), the pair weights are detached using a stop-gradient operation. This prevents the router from trivially reducing the alignment loss by lowering its responsibilities, but it also means that the alignment discrepancy does not directly update the router parameters through these weights. The router is instead trained through classification and the routing regularizers, while the alignment loss updates the experts and shared encoder. The shared-encoder updates can affect routing indirectly, but the distinction matters for the central claim that MESSI learns which domain relationships should be aligned. It remains unclear whether the learned routing structure is genuinely selected to preserve useful subset-shared invariances or whether it emerges indirectly from the other objectives. An ablation of the stop-gradient design, or a more direct analysis of how the alignment objective influences routing decisions, would clarify this point.

  2. The class-conditional OT estimates may be noisy, particularly on datasets with many classes. The alignment loss is computed over class-domain cells within minibatches. The reported batch size is 160 for DomainNet, which contains 345 classes, and the iWildCam setup samples only eight images from each of four active camera locations per minibatch. Consequently, many class-domain cells may contain very few examples, and some class-domain pairs may not be available for alignment in a given batch. In such cases, empirical OT can be dominated by individual samples rather than representing a reliable estimate of the underlying feature distributions. The paper does not report the distribution of cell occupancies, the fraction of potential alignment terms actually used, or sensitivity to minibatch size and the number of samples per class-domain cell. These details are important for assessing whether the proposed distribution-level alignment remains reliable in the more challenging benchmarks.

  3. The training cost of the primary OT version is substantial relative to the performance gain over the cheaper alternative. On PACS, MESSI-S-OT requires approximately 1,286 seconds per 1,000 optimization steps, compared with approximately 440 seconds for MESSI-S-MMD and 210 seconds for GMoE. The OT variant improves average DomainBed accuracy over the MMD variant by 1.9 percentage points, but requires roughly 2.9 times as much training time. The reported relative training times are normalized by a ResNet-50 ERM baseline even though the proposed MESSI-S model uses DeiT-S/16, so the headline overhead is not a strictly same-backbone comparison. Nevertheless, the absolute timings and comparisons between the two MESSI variants show that the OT solver introduces a significant cost. Since OT is the primary version of the method, the practical value of the additional accuracy gain over MMD deserves more discussion, especially for larger datasets and more numerous source domains.

Questions for authors

See weakness

3 comments

Q1. Stop-gradient routing and its interaction with alignment

A1: We agree that stop-gradient prevents the alignment loss from directly optimizing router-specific parameters. In our implementation, classification and routing regularizers train the router, while alignment updates the experts and shared encoder. Those encoder updates may affect subsequent routing outputs. The routing diagnostics show an association between the full objective and the observed routing patterns, but do not isolate the causal effect of alignment on router decisions. We conducted a stop-gradient ablation on PACS across three matched seeds. MESSI achieves 90.90 ±\pm 0.70 with stop-gradient, compared with 89.80 ±\pm 0.48 without it, with consistent improvements across all three seeds (+1.10 ±\pm 0.25 percentage points). We will state this limitation and describe the router as learning responsibilities through the predictive and routing objectives, with alignment strengths conditioned on those responsibilities.

Q2. Reliability of class-conditional OT under sparse minibatch sampling

A2: Thank you for raising concerns about class-domain occupancy and the reliability of minibatch OT. As specified in Appendix B.2, MESSI computes OT only for classes present in at least two source domains within a minibatch. Missing class-domain pairs are skipped, while singleton cells are permitted. For singleton-to-singleton pairs, OT reduces to squared feature distance.

On DomainNet, approximately 0.9% of potential class-domain pairs are active per iteration, with approximately 90% of active cells containing a single example. On iWildCam, our analysis across three seeds shows 11.182 ± 0.008 nonempty cells per minibatch, with 43.67 ±\pm 0.03% singleton cells. OT computes 3.070 ±\pm 0.007 valid pairs per step, corresponding to 0.2812% of the 1,092 potential class-domain pairs. Importantly, 94.45% of minibatches contain valid alignment pairs, and cumulative coverage reaches 87.57 ± 0.13% of source-eligible pairs, remaining 75.87 ±\pm 0.25% when excluding the empty-frame class.

We further evaluate DomainNet using larger minibatches and class-aware paired sampling with at least two examples per selected class-domain cell. The results show comparable OOD accuracy across all three configurations, indicating limited performance sensitivity to increased minibatch size and class-domain occupancy in the evaluated settings.

Table C: DomainNet OOD accuracy (%) across three seeds.

SettingOOD accuracy (%)
MESSI-S-OT (batch = 160)48.6 ± 0.3
Larger minibatch (batch = 320)48.8 ± 0.5
Paired-class sampling (min. occupancy = 2)48.7 ± 0.3

We will incorporate the occupancy distributions, active-pair and cumulative coverage statistics, and sampling sensitivity results into the revised manuscript, together with the corresponding implementation details.

Q3. OT computational cost and accuracy–efficiency trade-off

A3: Thank you for highlighting the cost–accuracy trade-off of OT. To assess its practical value, we report absolute training times measured on H100 GPUs alongside PACS accuracy over three seeds.

Table D: Training cost–accuracy trade-off on PACS (3 seeds).

MethodTime / 1K stepsRelative to GMoEPACS Acc. (%)
GMoE209.71 ± 0.49 s1.00×87.3 ± 0.1
MESSI-S-MMD439.68 ± 5.35 s2.10×89.3 ± 0.6
MESSI-S-OT (25 iter.)690 ± 15.23 s3.29×90.3 ± 0.5
MESSI-S-OT (50 iter.)1,286.26 ± 28.57 s6.13×90.9 ± 0.7

Full OT improves PACS accuracy by 1.6 percentage points over MMD at 2.93× training time. To improve efficiency, we reduce Sinkhorn iterations from 50 to 25 while keeping the architecture and other objectives unchanged. This reduces training time by 46.4%, with only a 0.6-point accuracy reduction. Compared with MMD, reduced OT achieves 1.0-point higher accuracy at 1.57× training time, providing a more efficient operating point.

We clarify that the previously reported 12.39× overhead was normalized against ResNet-50 ERM, rather than a same-backbone DeiT-S/16 baseline. Table D instead provides absolute timings and directly compares OT and MMD under the same MESSI architecture, isolating the computational trade-off between alignment objectives.

OT operates only on active class-conditional domain pairs within each minibatch. Thus, per-step overhead depends on the number of active pairs, their sample counts, and Sinkhorn iterations, rather than total dataset size alone. The reduced-iteration configuration provides a practical way to control this overhead without modifying the architecture. We will incorporate these results into Table 5 and clarify the training-cost normalization and scalability considerations in the revision.

We sincerely thank you for the time and effort you have devoted to reviewing our work. We would greatly appreciate your thoughts on our response and the additional results, particularly whether any concerns still need clarification. We would be happy to discuss any remaining questions and would value your further feedback as we refine the manuscript.

Reviewer_4cfGOpenReview ↗

Thanks for the response

The clarification is helpful, I thus keep my positive rating.

Reviewer_b71JOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MESSI, a mixture-of-experts framework for domain generalization based on subset-shared invariance. Instead of enforcing global feature alignment across all source domains, it uses routing-conditioned alignment to learn class-specific relationships between domain subsets. The framework also incorporates expert diversity and routing regularization. Experiments on DomainBed and controlled domain-growth settings show improved out-of-domain generalization.

Critical strengths
  1. The paper presents a clear motivation for relaxing global domain invariance. Modeling predictive structure shared by domain subsets is a reasonable direction for heterogeneous domain generalization.

  2. The routing-conditioned alignment mechanism is well motivated. Unlike conventional MoE approaches, routing directly influences which domain distributions are aligned.

  3. The experiments include competitive DG baselines, same-architecture ablations, and domain-growth analysis. These results provide useful evidence for the proposed mechanism.

Critical weaknesses
  1. The theoretical results mainly characterize exact invariance constraints and feasible representation classes. They do not directly establish that the proposed finite-weight optimization improves generalization to unseen domains.

  2. The learned subset structure is not sufficiently validated. Although routing analysis shows different alignment patterns, it remains unclear whether these patterns correspond to meaningful shared predictive factors across domains.

  3. The domain-growth analysis uses a controlled synthetic setting, leaving broader generalizability under real-world domain heterogeneity insufficiently established.

  4. The computational overhead is substantial. Table 5 reports training costs of 12.39× ERM for the OT version and 4.24× for MMD.

Questions for authors
  1. Provide stronger evidence for the learned domain subset structure.

  2. Explain how the theoretical analysis supports the practical optimization objective.

  3. Discuss the accuracy–cost tradeoff of OT-based alignment.

4 comments

Q1. Theoretical scope and finite-weight OOD generalization

A1: Thank you for highlighting the distinction between representational characterization and optimization-based generalization. Our theoretical analysis is intended to establish why subset-shared invariance provides a less restrictive learning principle, while the effectiveness of its practical realization is evaluated empirically.

Specifically, Theorem 3.3 shows that imposing additional exact invariance constraints cannot enlarge the feasible representation class, and Proposition 3.7 establishes a representational separation between complete and subset-supported expert alignment under the same model class and a fixed input-dependent router. Together, these results identify a structural limitation of global alignment: it can exclude predictive representations that remain feasible under subset-supported alignment. This establishes the theoretical motivation for relaxing uniform cross-domain invariance, without requiring a claim that every feasible subset-invariant solution yields lower target risk.

MESSI operationalizes this principle through routing-conditioned, class-conditional alignment with finite regularization weights. Rather than imposing binary subset assignments, the objective continuously adjusts the alignment strength of each expert-domain pair according to its learned routing responsibility. Although all valid pairs retain positive weights (Eq. 19), their contributions are non-uniform, allowing different experts to prioritize different cross-domain relationships instead of enforcing identical alignment pressure across all pairs. This is a deliberate soft formulation that makes subset-aware alignment jointly learnable with classification (Eq. 17), without requiring predefined domain subsets or discrete assignment optimization.

The connection to unseen domain generalization is supported by complementary empirical evidence. DomainBed evaluates performance on held-out target domains, controlled domain growth experiments examine robustness under increasing source heterogeneity, and comparisons with Global-MoE and Random-Subset-MoE in Table 4 assess the contribution of learned routing-aware alignment beyond architectural capacity and generic alignment strategies. Together, these experiments provide evidence that the proposed formulation improves OOD performance in the evaluated settings.

We will clarify this distinction in the revision: the theory establishes the representational rationale, the routing-conditioned objective provides its practical realization, and the controlled experiments establish its empirical generalization benefits.

Q2. Validity and predictive relevance of learned subset structures

A2: Thank you for highlighting this distinction. Our experiments support the predictive usefulness of routing-conditioned alignment, but do not establish that the learned responsibilities recover ground-truth shared predictive factors. We will distinguish these two claims explicitly.

Appendix E.5 shows that MESSI learns substantially different alignment structures from global and random controls (Jaccard overlap: 0.04–0.14 across DomainBed) while achieving lower class-conditional discrepancies on the same selected pairs (0.096 vs. 0.234 on PACS). Under the same MoE architecture on PACS (Table 4), MESSI achieves 90.9% accuracy, compared with 89.1% for classification-only MoE, 87.1% for global alignment, 87.0% for random-subset alignment, and 88.2% for domain-only routing, supporting the predictive usefulness of the complete objective rather than recovery of ground-truth shared factors.

Together, these findings support the predictive relevance of MESSI's learned subset structure, although they do not establish a unique correspondence between individual experts and ground-truth semantic factors. We will clarify this distinction and moderate the corresponding claims in the revised paper.

Q3. Generalization beyond synthetic domain-growth settings

A3: Thank you for this remark. Following your suggestion, we conducted an additional experiment on WILDS-iWildCam to evaluate the effect of increasing the number of source domains under real-world heterogeneity.

Real-world domain growth. To examine whether this trend extends beyond synthetic shifts, we conducted a fixed-budget experiment on WILDS-iWildCam with K∈{16,64,128,243}K\in\{16,64,128,243\} source cameras. We keep the total number of unique training images and optimization steps fixed, use the same held-out OOD cameras, and compare GMoE, OMoE, Global-MoE, and MESSI. Global-MoE uses the same architecture as MESSI but applies class-conditional alignment uniformly across all source-domain pairs for every expert, rather than using routing-conditioned alignment weights. We additionally audit source-class coverage to distinguish domain-growth effects from changes in available species.

Table A: Results for iWildCam domain growth. OOD test macro-F1 (%), mean ±\pm std over three seeds.

Source camerasGMoEOMoEGlobal-MoEMESSIΔ vs. GMoE
1622.0 ±\pm 0.922.4 ±\pm 0.921.8 ±\pm 0.823.5 ±\pm 0.9+1.5
6424.1 ±\pm 0.724.8 ±\pm 0.823.9 ±\pm 0.728.5 ±\pm 0.8+4.4
12825.4 ±\pm 0.726.2 ±\pm 0.725.0 ±\pm 0.631.1 ±\pm 0.7+5.7
24326.7 ±\pm 0.628.0 ±\pm 0.726.0 ±\pm 0.633.2 ±\pm 0.7+6.5

Table A shows that MESSI's advantage over GMoE increases from +1.5 to +6.5 F1 points as more real source environments are introduced. Its growing advantage over Global-MoE further supports the benefit of selective rather than uniform alignment.

Heterogeneity at a fixed domain count. Increasing the number of cameras also changes their composition. To examine heterogeneity at a fixed domain count, we additionally fix DomainNet Quickdraw as the target and evaluate all 10 combinations of three source domains selected from the remaining five. We keep the source-example budget and training protocol fixed, and quantify source heterogeneity using mean pairwise class-conditional MMD from a frozen DeiT-S encoder.

Table B: Results grouped by source heterogeneity OOD accuracy, mean ±\pm std over three seeds.(%).

Source heterogeneityGlobal-MoEMESSIΔ
Low (3 subsets)17.2 ±\pm 0.517.5 ±\pm 0.6+0.3
Medium (4 subsets)16.6 ±\pm 0.717.6 ±\pm 0.6+1.0
High (3 subsets)15.2 ±\pm 0.717.0 ±\pm 0.7+1.8

Across the ten source subsets, the MESSI–Global-MoE gain is positively associated with the frozen encoder source-heterogeneity measure (Spearman ρ=0.72\rho=0.72). Source composition also changes across subsets, so we interpret this as supporting association rather than an isolated causal effect. We will add the verified real-world domain-growth and fixed-count heterogeneity analyses to Appendix D, reference them in Section 5, and clarify the scope of our robustness claim.

Q4. Computational overhead and practical efficiency

A4: We agree that OT adds substantial training cost. The 12.39× in Table 5 is measured against a ResNet-50 ERM rather than an ERM with the same DeiT-S/16 backbone, please see our response to Reviewer 4cfG, W3 for the timing results under the same backbone. In short, using 25 Sinkhorn iterations cuts OT training time by 46% while average accuracy only moves from 66.2 to 66.0. The extra cost also applies only during training, since MESSI is faster than GMoE at test time. We will present this trade-off in the paper and keep MMD as the cheaper option.

Thank you for the response. I’ve updated my review to reflect your clarifications and the additional results, but I’ve decided to keep my original score.

Thank you for getting back to us and for your positive remarks on our work. We appreciate your feedback and keep your points in mind as we improve our work.

Comments on the paper

1 comment

We thank reviewers b71J, 4cfG, and tsvN for their constructive feedback. MESSI investigates subset-shared invariance for domain generalization through routing conditioned class conditional alignment. We summarize the main concerns and our responses below, with details in the reviewer specific replies.

The main concerns fall into six aspects:

  • (1) Theoretical scope and routing (b71J, 4cfG): The connection between exact invariance results, finite weight training, and routing learned without direct alignment gradients.
  • (2) Learned subsets (b71J, tsvN): Whether experts capture shared predictive structure rather than domain-specific assignments.
  • (3) Generalization (b71J, tsvN): Evidence beyond synthetic domain growth and under challenging source–target shifts.
  • (4) Alignment reliability (4cfG): Noisy OT estimates from sparsely populated minibatch class-domain cells.
  • (5) Metric dependence and stability (tsvN): Whether improvements reflect subset alignment or primarily the choice of OT.
  • (6) Computational efficiency (b71J, 4cfG, tsvN): Whether accuracy gains justify the additional training cost.

Regarding each concern, we summarize our responses as follows:

  • (1) We distinguish representational results from target-generalization guarantees. Classification and routing regularization train the router, while alignment updates experts and the shared encoder, influencing routing indirectly.
  • (2) Architecture-matched controls and expert-sharing diagnostics support the predictive usefulness of routing-conditioned alignment, without establishing a unique correspondence between experts and semantic factors.
  • (3) Additional fixed budget iWildCam domain growth, fixed count DomainNet heterogeneity, and controlled target shift evaluations provide complementary evidence beyond synthetic source expansion.
  • (4) Occupancy analysis confirms the sparse-estimation regime. Larger batches and paired-class sampling yield comparable accuracy, we interpret empirical OT as a stochastic regularizer, not evidence of accurate population-discrepancy estimation.
  • (5) Routing-conditioned alignment outperforms global alignment with both MMD and OT under matched discrepancies. Seed and hyperparameter analyses provide stability evidence, while we acknowledge MMD’s marginal average gain over GMoE and OT’s stronger results.
  • (6) We clarify the timing normalization and accuracy cost trade-off. Reducing Sinkhorn iterations from 50 to 25 lowers measured PACS training time by 46%, retaining 66.0% versus 66.2% average DomainBed accuracy. Alignment remains training-only.

We will incorporate these additions and clarify their scope and limitations in the revised manuscript. We appreciate the reviewers’ feedback and welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.