acceptodds
Withdrawn from ICLR 2027

Collaborative Dual-Evidence Representation Learning for Domain Generalization

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Domain Generalization (DG) seeks to induce a model from several source domains that generalizes to unseen target domains. Existing alignment-based DG methods strive to promote statistical invariance by reducing representation discrepancies across source domains. However, such alignment alone may not be sufficient for reliable generalization, since different representations may achieve similar levels of source-domain alignment while preserving different discriminative structures, and consequently behave differently on unseen domains. This motivates us to complement the statistical alignment with an additional inductive bias that provides a distinct discriminative signal for more reliable prediction. To this end, we propose **Co**llaborative **D**ual-**E**vidence **R**epresentation Learning (CoDER), which implements the two inductive biases through collaborative pathways and integrates dual predictive evidence for final prediction. Specifically, one pathway derives evidence from a CORAL-regularized representation, while the other utilizes a shared latent dictionary to generate context-aware modulation signals for this representation, yielding adaptive discriminative evidence. Each pathway employs a different classifier to provide Dirichlet evidence, which is fused at inference. Extensive experiments on five benchmarks demonstrate that CoDER consistently outperforms strong competitors, with evidence fusion *effectively enhancing* predictive performance and reliability.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Domain Generalization, Collaborative Representation, Dual Evidence Learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_3xTyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces CoDER, a representation learning framework that combines representations through two distinct representation learning pathways, i.e., a statistical-invariance pathway and a context-aware pathway. The final predictions are later combined through evidence fusion. Experiments across five domain generalization benchmarks show that CoDER has better performance for domain generalization.

Critical strengths
  1. The overall motivation and the approach are reasonable. The proposed approach is motivated by the limitation of using only the invariance representation, which may not capture all the useful predictive information. CoDER addresses this problem by combining invariant and contextual representations.
  2. The experiments are extensive and demonstrate the effectiveness of the proposed approach. The ablation and analysis are thorough for the two branches.
  3. The paper is overall well-written and easy to follow.
Critical weaknesses
  1. The novelty and technical contribution of the paper seem somewhat limited. While the experiments show the effectiveness of CoDER, the framework consists mostly of available components. The paper would benefit from a clearer explanation of the technical contribution of the paper compared to prior works.
  2. The motivation that statistical alignment can be limiting makes sense. However, Figure 1 is mostly conceptual, and the experiments do not provide enough insight into when and why statistical alignment can be insufficient. The paper would benefit from a more concrete illustration of how using statistical alignment alone can be insufficient and how the proposed contextual representations address this limitation.
  3. While the ablations support the effectiveness of combining two different pathways, it is not clear what predictive information each pathway encodes and whether they are complementary. The paper should incorporate some analysis demonstrating how these are complementary beyond performance improvement and what information they capture differently.
  4. The citation format is not correct in many cases, using narrative citations instead of citations in parentheses.
Questions for authors

Please see weaknesses.

Reviewer_D78BOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes CoDER, a dual-pathway method for domain generalization. One pathway learns invariant features using CORAL, while the other retrieves sample-dependent information from a latent dictionary and uses it to modulate the invariant representation. Predictions from the two pathways are combined through a Dirichlet-evidence formulation. This method obtains strong average performance on the five reported DomainBed datasets, with a particularly notable improvement on TerraIncognita.

Critical strengths

The paper is clearly written, and the motivation for retaining an invariant representation while introducing sample-specific modulation is reasonable. The ablations are useful and indicate that the dictionary, FiLM, orthogonality regularization, and class-conditional HSIC each contribute to performance.

Critical weaknesses

My first concern is novelty. The method combines several established ingredients, including CORAL, dictionary retrieval, FiLM, HSIC, and evidential fusion. The paper does not make sufficiently clear how its dual-pathway formulation differs conceptually from prior methods that jointly model domain-invariant and domain-specific information, particularly Bui et al. (2021).

My second concern is the evidential fusion mechanism. The two classifiers are trained with standard cross-entropy, while their logits are converted into Dirichlet evidence only at inference. It is therefore unclear whether the resulting evidence is meaningfully calibrated. More importantly, the paper does not compare this fusion rule against simple probability averaging or logit averaging. Without these baselines, I cannot determine whether the improvement comes from the proposed evidential formulation or simply from ensembling two predictors. The analysis of modulation magnitude is also correlational. Correct predictions may have larger modulation for reasons unrelated to a causal benefit from modulation. An inference-time intervention that scales or removes the modulation would make this claim more convincing.

Minor comments: The paper is generally well written, but several minor grammatical issues should be corrected, including

“is the Gram matrices” --> “are the Gram matrices” (p. 5),

“number of class” --> “number of classes” (p. 6),

“experiments have been constructed” --> “experiments have been conducted” (p. 6),

“these dataset” --> “these datasets” (p. 7),

and “across all benchmark” --> “across all benchmarks” (p. 8).

Questions for authors
  1. How does Dirichlet fusion compare with probability and logit averaging?

  2. Are the individual branches and fused predictors calibrated according to ECE or NLL?

  3. What is the key conceptual distinction from earlier invariant/domain-specific representation methods?

Reviewer_4aTzOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

CoDER is a domain-generalization model under the DomainBed protocol: ResNet-50, training-domain validation, on VLCS, PACS, OfficeHome, TerraIncognita, and DomainNet. A shared encoder feeds two heads. Pathway A projects the features and aligns them with CORAL. Pathway B retrieves a context vector from a shared dictionary of R = 2N bases (six bases when three sources are used, ten on DomainNet) by a softmax over cosine similarity, then applies that context as FiLM scale and shift on the aligned features (Eqs. 3–6). Orthogonality on the bases and a class-conditional HSIC penalty between the context and the aligned features are the extra regularizers (Eqs. 5 and 7). Training is cross-entropy on both heads plus those three terms (Eq. 8). At test time a Softplus turns each head's logits into Dirichlet evidence, and Eq. 11 combines the beliefs by weighting each head with the other head's uncertainty.

Critical strengths
  • The experimental scaffold matches this literature. Table 1 covers five datasets and a long baseline list, with means and standard deviations. Tables 7–11 give the per-domain numbers. Table 2 ablates the dictionary, FiLM, orthogonality, and class-conditional HSIC, with ERM and CORAL as reference rows.
  • The two heads are measurably different. In Table 4, ginv is ahead only on VLCS (79.1 vs. 78.6); gdis is ahead on the other four datasets. Fusion beats both heads everywhere, and Section 4.2 states that a plain average of the two predictions stays behind fusion. Jointly correct predictions are kept in full (F.B.C. = 100%).
  • The extra head is cheap on the reported clock. Table 3 gives CoDER the lowest per-batch time of the five methods listed: 0.209 s on PACS, 0.237 s on OfficeHome, and 0.392 s on DomainNet, below TCRI, I3C, GGA, and DA-SAM.
  • Against CORAL alone, Table 1 is higher on every dataset. TerraIncognita moves from 47.6 to 53.5.
Critical weaknesses
  • Novelty is the core problem. CORAL, FiLM, a softmax over a codebook, Dirichlet evidence, orthogonality, and HSIC are existing tools, and the paper cites them as such. Equation 8 trains both heads with cross-entropy. Dirichlet evidence is a Softplus applied at inference, and Eq. 11 is the usual uncertainty-weighted combination of two beliefs. Figure 1 is a shape-versus-color cartoon. No experiment measures those factors, or any other named factor, inside either pathway. Table 4 records agreement of the predicted labels.
  • The DomainBed gain is about one point, and it is uneven. The Table 1 average is 67.4 against 66.3 for GGA. On DomainNet, the largest set in the paper, CoDER at 44.4 is behind GGA (45.2), SAGM (45.0), and DA-SAM (44.5). OfficeHome is 70.4 ± 0.4 against I3C at 70.2 ± 0.5. VLCS is 80.4 ± 0.2 against SAGM at 80.0 ± 0.4. TerraIncognita is the comparison that moves the average (53.5 vs. 51.2).
  • The class-conditional HSIC term carries 2.6 of the 2.8-point gap over CORAL. Dropping it in Table 2 leaves 78.6, 85.6, 68.5, 49.8, and 41.5, which average to 64.8, next to CORAL at 64.6. VLCS, PACS, and OfficeHome sit at or below the CORAL row. DomainNet returns to CORAL's 41.5. TerraIncognita falls from 53.5 to 49.8, which is still above CORAL (47.6) and below TCRI (51.2). Appendix B records this collapse.
  • Fusion, the step named in the title, adds little once both heads exist. Over the better head in Table 4 the margins are +1.3 (VLCS), +0.6 (PACS), +0.2 (OfficeHome), +1.1 (TerraIncognita), and +0.5 (DomainNet). The heads disagree on 6.9%, 4.7%, 1.4%, 5.5%, and 4.0% of test samples (100 − B.C. − B.Inc.). Cross-coverage is 3.0% and 1.7% on OfficeHome, and 3.9% and 3.2% on DomainNet. On all five datasets, fused accuracy lines up with the jointly correct mass plus the retained one-head wins, so both-wrong recovery is about zero at the printed precision.
  • Figure 3 bins the post-training magnitude ∆ . Errors sit in the small-∆ bins, and the fused confidence lies above either head. The association is measured after training, with no intervention on the FiLM maps. The abstract's reliability claim rests on this plot. The paper reports no calibration error, selective prediction, or risk-coverage number.
  • Table 1 puts a dagger only on TCRI ("our reproduced results"). The 1.1-point ranking compares CoDER with GGA, SAGM, DA-SAM, and I3C entries that the caption does not mark as re-runs. Appendix C replaces CORAL with MMD, DANN, and RDM on PACS only (Table 6); the gain on the strongest of those baselines is +1.4 (RDM, 87.2 to 88.6). Appendix E's limitation is that Pathway B has many components. Separately, Table 1 lists GGA on PACS at 87.3 ± 0.4, while the domain scores in Table 8 average to the 86.8 printed there.
Questions for authors
  1. Please re-run GGA, SAGM, DA-SAM, and I3C in the same codebase, with the same ResNet-50, training-domain validation, three trials, and 20 hyperparameter samples used for CoDER. What are the five dataset scores and the average, and does the DomainNet order change?
  2. Table 4 lines up with fusion that keeps every jointly correct call and a fraction of the one-head wins. Please give, per dataset, the fraction of samples both heads miss that fusion gets right.
  3. For the full model and the No/cHSIC row in Table 2, please report the dependence between zc and zinv, and a measurement of a factor that one pathway retains and the other drops.
  4. If γ and β are scaled or zeroed at test time with zinv held fixed, how does accuracy move with the modulation magnitude in Figure 3?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.