- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper extends Rademacher complexity-based generalization theory to domain generalization by treating source domains as random distributions and defining a distribution-level Rademacher complexity on the Wasserstein space.
Critical strengths
- The idea of connecting optimal transport geometry and curvature to DG generalization is new and interesting, and may inspire further work.
- The proofs are written out in full in the appendix and are mostly easy to follow.
- The authors try to validate the theory with experiments on three standard DG benchmarks, and openly discuss the cases where results do not match (PACS Photo, Office-Home Clipart)
Critical weaknesses
- The hierarchical setting, where domains are drawn from a meta-distribution and then samples from each domain, and the resulting domain-level and sample-level complexity terms, are well established in learning-to-learn and DG theory [1,2].
- In classification, Y is a finite set, so Z = X × Y is not a length space and CD(K,∞) with K > 0 cannot hold. The paper does not explain how Theorem 3 applies in its own experiments.
- The DG error values in (Figures 1 and 2). With a linear classifier, a smaller radius gives smaller logits and therefore smaller loss gaps, regardless of generalization. Since the source and target accuracy are not reported, there is a speculation that the smaller gap may come from underfitting.
Questions for authors
- How does a union of disjoint spheres satisfy CD(K,∞) with K > 0? What is the actual K of the latent measure in your experiments?
- How do you handle discrete labels in Z?
- How do you account for the fact that B/√K is invariant to rescaling the metric? Under what condition does higher curvature truly reduce the bound?
- How does your two-level bound compare to the meta-learning bounds of [1] and [2]?
- What exactly is the “DG error” in Figures 1 and 2? Can you report source and target accuracy for each radius?
REFS:
[1] Maurer et al, The Benefit of Multitask Representation Learning
[2] Blanchard et al, Domain Generalization by Marginal Transfer Learning
4 comments
Question 1: In our setting, the representation space is a Riemannian manifold, the base manifold is a union of hyperspheres equipped with the round metric. For a standard sphere of radius , the Ricci tensor is where is its dimension, is the radius, and is the Riemannian metric tensor. In our experiments, we set to the sectional curvature . Then, by the equivalence theorem given in Theorem 7.3(b) ((1) and (3)) of [5], one can directly verify that our experimental setting satisfies the condition.
Question 3: We thank the reviewer for this insightful question. We agree that the curvature-dependent term indeed couples with the Lipschitz constant and the domain regularity constant , and these quantities need not remain fixed when the representation changes.
To address the coupling with , we will remove the Lipschitz assumption on the loss and replace the Kantorovich--Rubinstein duality step in the proof of Theorem 3 with the domain adaptation inequality in Lemma 1 of [1] (Redko et al. (2019)), inheriting its assumptions. This modification eliminates the explicit factor from the bound. We will add an appendix summarizing the relevant lemma (Appendix D in the revised paper) and give the revised derivation of Theorem 3 (Eqn. (31) in the revised paper).
To address the coupling with , we will no longer treat as a fixed constant but keep it as an explicit, representation-dependent factor in the bound, around which we build an explicit invariance interpretation. See the revised version of the paper for details (Definition 4, Eqn. (21), (23), (24)). Guided by this invariance interpretation, we revise the claim about curvature. The original statement that "higher curvature leads to better domain generalization'' is replaced with "domain generalization is jointly affected by the curvature of the representation space and its degree of cross-distribution invariance.'' This statement directly connects our bound to invariant representation learning for domain generalization.
[1] Maurer et al, The Benefit of Multitask Representation Learning.
[2] Blanchard et al, Domain Generalization by Marginal Transfer Learning.
[3] Gouk et al, On the limitations of general purpose domain generalisation methods.
[4] Redko et al, Theoretical analysis of domain adaptation with optimal transport.
[5] Lott et al, Ricci curvature for metric-measure spaces via optimal transport.
[6] Albuquerque et al, Generalizing to unseen domains via distribution matching.
[7] Zhao et al, On learning invariant representations for domain adaptation.
Thank you authors for the rebuttal. Most of my concerns, particularly regarding (W2,Q2), (W3,Q5), and Q3 are still unaddressed. Infact, since that the authors acknowledge for the change the assumption (in response to W2,Q2), and removing assumptions from Theorem 3, how it has affected the error bound is unanswered. Furthermore, the speculation of underfitting due to use of small radius (of single hypersphere) and the mitigation through use of mixture of hypersphsre is not quantified in the response to W3,Q5. Due to these reasons, i choose to stick with my inital rating of weak reject (2).
We thank the reviewer for the response, and we would still like to provide some explanations regarding (W2, Q2), Q3, and our revisions to these issues. First, we have submitted a revised version of the manuscript, and in each response below we indicate where the corresponding revision was made. For further details, please refer to the revised version.
-
Our revision for (W2, Q2) indeed addresses this concern of yours: following Redko et al. (2017), Albuquerque et al. (2019), and Zhao et al. (2019), we assume that each domain has a labeling function and adopt the covariate shift assumption for all . We then define domain distributions only on the representation space , and the risk functional becomes where is the shared labeling function. The same replacement is made in all subsequent formulas, and the derivations are modified accordingly. This revision indeed removes the discrete space from the definition of the underlying sample space, so that represents only the representation space (which is a standard length space). The effect on the bound is as follows. The mathematical form of the final bound is unchanged, but its interpretation and validity are materially different: is now the Ricci curvature lower bound of the representation space , the domain space is , and all Wasserstein-geometric steps (including the condition) now act on , which is a length space. In the original manuscript, the curvature assumption was imposed on , where it cannot hold for finite label sets. In the revision, the assumptions and the conclusion are consistent on the same space .
-
For Q3, our understanding of your question is as follows: changing the curvature causes the representation space to change, making not a constant but a variable that depends on the representation space, so the curvature couples with and in the original manuscript (). To address the coupling with , we remove the Lipschitz assumption on the loss and replace the Kantorovich–Rubinstein duality step in the proof of Theorem 3 with the domain adaptation inequality in Lemma 1 of Redko et al. (2019): This revision directly replaced the content in the original manuscript (the content below Eq. (30) and above Eq. (31)) with Eq. (31) in the revised manuscript, thereby eliminating . As for the bound itself, this only eliminates and has no other impact. To address the coupling with , we no longer treat as a fixed constant. Instead, we keep it as an explicit representation-dependent factor and use it to define the invariance measure The final bound resulting from these two revisions is as follows: Please refer to Theorem 4 in the revised version for details. Therefore, we believe that, with these solutions, we can answer Q3: higher curvature reduces the bound under the condition that the invariance does not degrade, i.e., does not increase. Equivalently, simultaneously learning invariant representations and increasing the curvature of the representation space reduces the domain generalization gap. If the metric is rescaled, and change together, and one cannot claim a benefit from curvature alone. Our revised claim is therefore that domain generalization is jointly affected by the curvature of the representation space and its degree of cross-distribution invariance, rather than that higher curvature always helps.
we hope this response addresses the reviewer's concerns. We welcome any further questions or suggestions. If the concerns are resolved, we would be grateful if the reviewer considers updating their score accordingly.
Weaknesses 1 and Question 4: We thank the reviewer for pointing us to these important references. We agree that the hierarchical setting (drawing domains from a meta-distribution and then samples from each domain) is well established in learning-to-learn and domain generalization theory, including Maurer et al. (2016) and Blanchard et al. (2021). But we do not claim this hierarchical framework as an independent contribution; in Section 2.2 we introduce it following [3] (Gouk et al. (2022)) for completeness.
Regarding the comparison with the bounds in [1] and [2], we would like to clarify the relationship. We explicitly restrict the domain space to a C-regular subset of the 2-Wasserstein space, whereas [1] and [2] do not impose this Wasserstein restriction. This directly lead to our modification of McDiarmid's inequality and to the curvature-dependent confidence term, which is not present in the bounds of [1] and [2]. We therefore view our bound as complementary to [1] and [2]: it does not aim to replace their more general or potentially tighter bounds, but provides a geometric perspective that connects the curvature of the representation space to domain generalization.
Weaknesses 2 and Question 2: We thank the reviewer for this insightful comment, and we fully agree with the criticism. In classification, Y is a finite set, so Z=X×Y is not a length space. We propose the following revision. Following [4], [6], and [7], we assume that each domain has a labeling function y=fD(x), and adopt the covariate shift assumption (common in domain generalization theory) fD1(x)=fD2(x) for all D1,D2∈DC. Under this assumption, we define the domain distributions only on the representation space Z, i.e., we have changed the meaning of the symbol Z from the product space X×Y to the representation space. The risk functional becomes ϕℓ,h(D)=Ex∼D[ℓ(h(x),f(x))], where f is the shared labeling function. See the beginning of Section 2.2 in the revised manuscript for details.
Weaknesses 3 and Question 5: We thank the reviewer for these detailed concerns. We agree that using a single hypersphere may lead to underfitting, and that the mixture-of-hyperspheres construction was introduced to mitigate this issue but may not eliminate it entirely; indeed, variations in training accuracy are observed in practice. Therefore, we can only verify the relationship between curvature and generalization error by showing how this empirical generalization gap varies, which does introduce a limitation in experimental rigor. We will add explicit discussion of this experimental limitation in the revised version (Appendix G.4).