acceptodds
Under review as a conference paper at ICLR 2027

How Much Source Geometry to Trust? Adaptive Spiked Supports for Open-Set Domain Generalization

Abstract

Open-set domain generalization (OSDG) requires acceptance regions to be loose enough to accommodate domain-shifted known samples yet compact enough to reject unknowns. Relying more on source-domain class geometry makes these regions more compact but less tolerant of domain shift, and since the reliability of that geometry varies across classes and source-domain configurations, no fixed degree of reliance suits all cases. This raises a central question: how much source geometry should we trust? We propose Adaptive Spiked Support (AdaSpike), which adapts this trust through cross-domain geometric consistency. Our key insight, which our robust OSDG analysis formalizes, is that greater source consistency suggests more stable target geometry and warrants greater trust, while greater discrepancy suggests more uncertainty and warrants less. Specifically, we show that the optimal acceptance region under a robust risk bound has a closed-form spiked structure, which preserves anisotropic scales along the leading source directions and assigns a shared isotropic scale to the rest, interpolating between isotropic and fully anisotropic supports. AdaSpike determines from its cross-domain discrepancy, having no need of target data or manual tuning. The supports are learned with objectives formulated from an OSDG risk bound, and a single support score serves both classification and rejection. Across three OSDG benchmarks and two backbones, AdaSpike outperforms existing methods in all six settings, improving average OSCR by 2.16 points over state-of-the-art methods.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Open set domain generalization, Domain generalization, Open set recognition

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_h5qnOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies how acceptance regions should retain source class geometry when recognizing known classes and rejecting unknown classes in an unseen domain. AdaSpike represents each class by a spherical tangent space support with distinct scales along selected principal directions and a shared residual scale. A robust optimization of a coverage and volume risk bound yields a spiked spectrum whose number of retained directions depends on an uncertainty radius. The implementation estimates this radius from the largest discrepancy between source class second moments and selects the dimension before training. It then learns representations and support scales through classification, coverage on held out source domains, and log volume objectives. The CLIP version additionally uses generated pseudo unknowns. Experiments cover PACS, OfficeHome, and VLCS with ResNet 18 and CLIP. Reported average results outperform the compared methods, and supplementary experiments examine fixed dimensions, alternative support parameterizations, objective ablations, individual target domains, and computational overhead.

Critical strengths
  1. The support construction offers a clear interpretation of the tradeoff between directional source structure and isotropic protection against uncertain shifts. The spectral derivation in Theorem 1 and Appendix A.5 provides a coherent solution to the stated robust coverage and volume optimization problem, rather than merely introducing an arbitrary matrix parameterization.

  2. The empirical evaluation includes two substantially different representation settings and three benchmarks. Tables 1 and 2 show consistent gains in average OSCR, while Table 3, Table 6, Figures 4 to 6, and Tables 9 to 14 provide useful evidence about support structure, objective interactions, dimension choices, and variation across target domains.

Critical weaknesses
  1. Observed source discrepancy does not calibrate target uncertainty. Proposition 1 and Appendix A.3 require a population domain radius plus estimation error, whereas Section 3.1 uses the observed source diameter. Even infinitely sampled sources can share identical geometry while an unseen domain differs substantially. Appendix C acknowledges omitted estimation error but does not justify estimating the population radius this way. Figure 3 offers benchmark evidence, not a calibrated guarantee. The central trust rule therefore does not inherit the theorem's robustness.

  2. The guarantee does not cover learned geometry. Appendix A.3 assumes independent tangent displacements under a fixed mapping, but AdaSpike estimates prototypes and trains representations on the same samples. No uniform moment concentration or independent estimation split is supplied. Stopping gradients does not restore statistical independence, and the classifier uniformity in Appendices A.6 and A.7 does not resolve this gap. Moreover, the initially selected dimension remains fixed as geometry changes, without validating that its trust estimate remains appropriate.

  3. The practical objective can favor support collapse. Equation (7) omits the robustness term that diverges as residual scales shrink, replacing theoretical coverage and volume with a boundary hinge and log volume. If class features collapse to distinct prototypes, coverage vanishes and classification can improve while log volume tends to negative infinity. Appendix B.4 gives no scale floor preventing this. This does not imply reported runs diverge, but the implemented optimization lacks the protection underlying its robust design.

  4. Support volume need not control unknown acceptance. Assumption 1 requires bounded unknown density relative to full spherical surface measure. Learned representations can instead concentrate unknowns on lower dimensional sets or near known supports, making volume reduction ineffective or the bound uninformative. The paper neither validates this assumption nor tests such controlled unknown distributions. Since volume regularization is the ResNet version's only explicit unknown control, the applicability of the conditional rejection argument is central rather than incidental.

  5. The adaptive trust rule's contribution is not isolated. Fixed dimension and support family comparisons do not test matched alternatives based on spectral energy, constant uncertainty, or source validation under the same objective. Table 6 studies losses only on PACS with ResNet. CLIP additionally uses pseudo unknown generation and exclusion loss without an ablation separating their benefits from adaptive spiked supports. The results demonstrate the full pipeline's performance, but do not establish that source discrepancy based trust explains its gains.

Questions for authors

See Weaknesses.

6 comments
AuthorsOpenReview ↗

Response to Reviewer h5qn (1/6)

We appreciate the reviewer for the careful and constructive assessment of our work, particularly for recognising the theoretical derivation of the spiked support structure (Strength 1). In response to the reviewer's concerns, we have revised our theoretical analysis and conducted additional empirical evaluations addressing the practical method. The main theoretical revisions are detailed in our response to Weakness 1. Before addressing the specific concerns, we would like to clarify the scope of our theoretical and empirical contributions.

As stated in the original Introduction, AdaSpike is a “theoretically motivated framework”, without claiming end-to-end theoretical guarantees for the practical method. Our theoretical analysis considers fixed representations and class prototypes under explicit assumptions, including the two-stage generative model adopted from Blanchard et al. (2021) and a bounded-density assumption for unknowns. Within this setting, our analysis motivates the support structure and dimension-selection rule under explicit assumptions, while the effectiveness of the learned method is assessed empirically. We have clarified the role and scope of our theory in the revised version.

Our contributions can be distinguished at three levels.

  1. Structural results: Revised Theorem 1 characterizes the optimal spiked support from a robust optimisation problem and establish how source discrepancy determines the number of retained directions.

  2. Conditional bounds: Revised Proposition 1 provides a probabilistic bound relating observed source discrepancy to unseen target geometry, while revised Theorem 2 accounts for target domains outside the resulting uncertainty set in an OSDG risk bound.

  3. Practical implementation: AdaSpike updates representations and class prototypes using surrogate objectives while retaining the initially selected spike dimension. While our theoretical analysis does not provide an end-to-end training guarantee, we have provided additional empirical evidence concerning dimension stability (W2), training dynamics alongside clarification of the positive scale floor (W3), and the effect of volume regularisation on unknown false acceptance (W4). We also add matched comparisons with alternative dimension-selection rules and CLIP loss ablations to assess their respective contributions (W5).

We acknowledge that our theoretical analysis has limitations. In particular, it assumes fixed representations and class prototypes, whereas these are updated during practical training. Moreover, the probabilistic bound in Proposition 1 can be loose when the number of source domains SS is small. We have acknowledged and discussed these limitations in the revised Limitations section (Appendix C).

References

Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.

AuthorsOpenReview ↗

W1. Linking observed source discrepancy to target uncertainty and dimension selection (2/6)

We acknowledge that the original theoretical analysis did not fully justify replacing the population uncertainty radius with the observed source discrepancy b^c\hat b_c. Under the two-stage generative model of Blanchard et al. (2021), our revised theoretical analysis establishes a probabilistic connection between observed source discrepancy and unseen target geometry (Rev. Proposition 1), and links observed source discrepancy to adaptive spike dimension selection through robust optimization (Rev. Theorem 1). Revised Theorem 2 further retains an contribution from target domains outside the uncertainty set in the OSDG risk bound. The key results of Proposition 1 and Theorem 1 are summarised below:

Proposition 1. Probabilistic target coverage

P(TcPT∈Uc(b^c,rcobs))≥[1−3S+1−δc]+P(T_c^{P_T} \in \mathcal U_c(\hat b_c,r_c^{\mathrm{obs}})) \geq [1-\frac{3}{S+1}-\delta_c]_+

Theorem 1. Adaptive spike dimension selection

k_c^\star = \\#\\{j: \hat\gamma_{c,j}>\tau_c\\}, \qquad \sum_j (\tau_c-\hat\gamma_{c,j})_+ = \hat b_c

Specifically, we replace the original Assumption 2 with the two-stage generative model of Blanchard et al. (2021), under which source and target domains are independently sampled from a common meta-distribution. Under this assumption, revised Proposition 1 constructs an uncertainty set using the observed source discrepancy b^c\hat b_c together with an additional estimation radius, and establishes a probabilistic lower bound on target coverage. Revised Theorem 1 shows that robust optimisation over this uncertainty set yields an optimal spike dimension determined by the observed source spectrum and b^c\hat b_c, with the additional estimation radius leaving the selected directions unchanged. Revised Theorem 2 introduces an additional risk term to account for target domains outside the uncertainty set, accounting for cases where the target geometry falls outside the uncertainty set.

However, our theoretical analysis is under explicit assumptions and the probabilistic coverage bound can be loose when the number of source domains SS is small. We therefore provided complementary empirical evidence in Figure 3 (both Orig. and Rev.), showing that source-estimated geometry captures much of the observed target-domain geometric variation on the evaluated datasets.

References

Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.

AuthorsOpenReview ↗

W2. Theoretical scope and stability of dimension selection under learned representations (3/6)

We acknowledge that our theoretical analysis is conducted in a fixed feature space. Our intention is to characterise how source discrepancy informs the selection of the support dimension. We do not claim that our theory provides guarantees throughout the neural representation learning process. We have acknowledged this limitation in the Limitations section (Rev. Appendix C).

To empirically assess whether the initial dimension selection remains appropriate as the representation changes during training, we recomputed k⋆k^\star (averaged across all classes and folds) using the learned representations after training and compared it with the initial selection (Rev. Appendix B.7, Table 12). The results are summarised below.

Table: Stability of dimension selection under representation learning.

DatasetInitial k⋆k^\starRe-estimated k⋆k^\starAffected Spectral Mass (%) ↓
PACS46.141.81.68
OfficeHome50.743.12.63
VLCS52.239.93.69
Average49.741.62.70

Although the re-estimated k⋆k^\star differs from its initial value, the affected dimensions account for only 1.68%, 2.63%, and 3.69% of the total post-training spectral variance on PACS, OfficeHome, and VLCS, respectively, with an average of 2.70%. These results suggest that the initial dimension selection remains relatively stable in terms of the affected spectral variance, providing empirical support for its use under representation learning.

Reviewer_ock6OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper investigates the problem of open-set domain generalization, where a model is trained on multiple source domains to identify unknown classes in unseen target domains. The authors argue that the threshold for rejecting unknown classes should adapt to the degree of feature dispersion. Specifically, the threshold should be tightened when source-domain features are highly similar and relaxed when they exhibit greater dispersion. Based on this observation, the authors propose a method that dynamically adjusts the rejection threshold by feature dimensions accordingly. Experimental results demonstrate the effectiveness of the proposed approach.

Critical strengths
  1. Adjusting the decision boundary according to feature dispersion along each dimension, rather than treating all dimensions equally, seems reasonable.
  2. The paper provides extensive mathematical derivations to explain and support the proposed method.
  3. The experimental results demonstrate the effectiveness of the proposed approach.
Critical weaknesses

1.Some terminology is overly specialized, making parts of the paper difficult to follow without consulting additional references. Examples include “geometry,” “anisotropic,” “isotropic,” and “spike dimension.” The authors are encouraged to use more accessible terminology or introduce a dedicated preliminary section to explain these concepts.

2.The motivation and proposed method are relatively clear, while the theoretical analysis seems unnecessary in the main text. I suggest moving it to the appendix.

Questions for authors

1.Why do (k^*=48) and (k=48) yield different performance in Fig. 5?

2.How does the proposed method perform when using ResNet-50 as the backbone?

6 comments
AuthorsOpenReview ↗

Response to reviewer ock6 (1/4)

We thank the reviewer for the thoughtful assessment of our work and the constructive suggestions. We have revised the presentation and conducted additional experiments to address these concerns, as detailed below.

W1. Clarification and consistency of terminology

We have added a dedicated terminology subsection (Rev. Section 2.1, Problem Setup and Terminology) to provide clearer explanations of key concepts, including geometry, anisotropic and isotropic supports, and spike dimension. We have also standardised terminology throughout the paper to avoid using different terms for the same mathematical concept. Specifically, we consistently use source discrepancy for b^c\hat b_c, replacing terms such as source uncertainty, geometric discrepancy, and geometric uncertainty, while reserving uncertainty for the uncertainty set and its interpretation. We also consistently use spike dimension for kck_c, reserving trusted spike dimension for intuitive explanations in the abstract and introduction.

AuthorsOpenReview ↗

W2. Simplification of the theoretical presentation (2/4)

We have substantially shortened the theoretical analysis in the main text and moved the full statements of Assumptions 1–2, Definition 1, Proposition 1, and Theorem 2 to Appendix A. We retain Theorem 1 as the central theoretical result in the main text, highlighting how source discrepancy determines the adaptive spike dimension. These revisions make the main text more focused on the motivation and proposed method, while preserving the complete theoretical analysis in the appendix.

AuthorsOpenReview ↗

Q1. Clarification and Updates to Figure 5 (3/4)

We identified typos in the originally reported k⋆k^\star values and inconsistencies in the fixed-kk evaluation protocol in Figs. 5 and 6 of the original submission, which led to the reported performance discrepancy. We have corrected these issues and updated the corresponding figures and hyperparameter settings in Table 7. To maintain consistency with the revised experimental reporting throughout the manuscript, we have also updated Figure 4, Figure 5, and Figure 6 to report the mean and standard deviation of adaptive k⋆k^\star performance over three random seeds.

Reviewer_SqLYOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes AdaSpike, an adaptive geometric support framework for Open-Set Domain Generalization (OSDG). The method uses cross-domain geometric discrepancies to determine how much source-domain structure should be preserved in class-specific acceptance regions. The authors provide a theoretical analysis motivating the spiked support structure and develop learning objectives for balancing known-class recognition and unknown rejection. Experiments on three OSDG benchmarks with ResNet-18 and CLIP demonstrate consistent improvements over existing methods, supported by ablation studies on the adaptive support design.

Critical strengths
  1. The paper addresses a well-motivated problem in OSDG by adaptively determining how much source-domain geometry should be trusted. The proposed spiked support provides a reasonable way to balance known-class coverage and unknown rejection.

  2. The authors provide a theoretical analysis to motivate the proposed support structure and learning objectives, establishing a clear connection between the problem formulation and method design.

Critical weaknesses
  1. The theoretical analysis considers both cross-domain variation and finite-sample estimation error, but the implementation only uses empirical source discrepancy. This may underestimate uncertainty in unseen target domains, raising questions about whether the theoretical guarantees still apply in practice.

  2. The method assumes that geometric consistency across source domains indicates reliable geometry in unseen target domains. However, similar source domains do not necessarily imply a similar target distribution. The robustness of this assumption under more challenging domain shifts remains unclear.

  3. The anisotropic dimension is estimated before training and remains fixed, while the backbone representations and class geometry evolve during optimization. It is unclear whether the initially selected dimension remains appropriate as the feature space changes.

Questions for authors
  1. The CLIP experiments introduce an additional exclusion loss using pseudo-unknown samples. Could the authors provide an ablation without this loss to clarify how much of the improvement comes from the proposed adaptive support?

  2. Could the authors report mean and standard deviation across multiple independent runs? In particular, how stable are the selected anisotropic dimensions and the resulting performance under different random seeds?

9 comments
AuthorsOpenReview ↗

Response to reviewer SqLY (1/5)

We thank the reviewer for highlighting these concerns. We have revised our theoretical analysis and conducted additional experiments to address them, as detailed below.

W1. Estimation error and adaptive dimension selection

We acknowledge that the original theoretical analysis did not fully justify using the observed source discrepancy b^c\hat b_c for dimension selection without explicitly accounting for finite-sample estimation error. Our revised theoretical analysis establishes a direct connection between the observed source discrepancy b^c\hat b_c and the optimal spike dimension kc⋆k_c^\star. Specifically, revised Theorem 1 yields the following adaptive dimension selection rule:

Theorem 1. Adaptive spike dimension selection

k_c^\star = \\#\\{j: \hat\gamma_{c,j}>\tau_c\\}, \qquad \sum_j (\tau_c-\hat\gamma_{c,j})_+ = \hat b_c

Specifically, we replace the original Assumption 2 with the two-stage generative model of Blanchard et al. (2021). Under this framework, revised Proposition 1 constructs an uncertainty set using the observed source discrepancy b^c\hat b_c together with an additional estimation radius, and establishes a probabilistic coverage bound for unseen target geometry. Revised Theorem 1 shows that robust optimisation over this uncertainty set yields an optimal spike dimension determined by b^c\hat b_c and the observed source spectrum, while the additional estimation radius does not affect the selected spike directions. This establishes the theoretical connection between the observed source discrepancy and our practical dimension selection rule.

References

Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.

AuthorsOpenReview ↗

W2. Generalisation of source geometry to unseen targets (2/5)

We acknowledge that geometric consistency across source domains does not necessarily guarantee similar geometry in an unseen target domain. Under the two-stage generative model of Blanchard et al. (2021), where source and target domains are independently sampled from a common meta-distribution, our revised theoretical analysis establishes a probabilistic coverage bound for unseen target geometry (Rev. Proposition 1) and accounts for target domains outside the uncertainty set in the OSDG risk bound (Rev. Theorem 2), as follows:

Proposition 1. Probabilistic target coverage

P(TcPT∈Uc(b^c,rcobs))≥[1−3S+1−δc]+P(T_c^{P_T} \in \mathcal{U}_c(\hat{b}_c,r_c^{\mathrm{obs}})) \geq \left[1-\frac{3}{S+1}-\delta_c\right]_+

Theorem 2. Finite-sample OSDG risk bound

E(hA⋆,f)≤B^cls+B^cov+B^vol+∑c∈KΞc+Baux\mathcal{E}(h_{A^\star,f}) \leq \hat{B}_{\mathrm{cls}} + \hat{B}_{\mathrm{cov}} + \hat{B}_{\mathrm{vol}} + \sum_{c\in K}\Xi_c + B_{\mathrm{aux}}

Specifically, revised Proposition 1 provides a lower bound on the probability that unseen target geometry lies within the source-estimated uncertainty set, while revised Theorem 2 introduces an additional risk term to account for target domains outside this set.

However, this probabilistic bound can be loose when the number of source domains SS is small, as in our benchmarks. We therefore provide complementary empirical evidence in Figure 3 of the original submission, showing that source-estimated uncertainty captures much of the observed target-domain geometric variation on the evaluated benchmarks. These results support the use of source discrepancy to guide adaptive dimension selection under the stated assumption, rather than providing a guarantee for arbitrary target shifts. We have further clarified the theoretical assumptions and limitations in Appendix C.

References

Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.

AuthorsOpenReview ↗

W3. Stability of dimension selection under representation learning (3/5)

We acknowledge that our theoretical analysis is conducted in a fixed feature space. Our intention is to characterise how source discrepancy informs the selection of the support dimension. We do not claim that our theory provides guarantees throughout the neural representation learning process. We have acknowledged this limitation in the Limitations section (Rev. Appendix C).

To empirically assess whether the initial dimension selection remains appropriate as the representation changes during training, we recomputed k⋆k^\star using the learned representations after training and compared it with the initial selection (Rev. Appendix B.7, Table 12). The results are summarised below.

Table: Stability of dimension selection under representation learning.

DatasetInitial k⋆k^\starRe-estimated k⋆k^\starAffected Spectral Mass (%) ↓
PACS46.141.81.68
OfficeHome50.743.12.63
VLCS52.239.93.69
Average49.741.62.70

Although the re-estimated k⋆k^\star differs from its initial value, the affected dimensions account for only 1.68%, 2.63%, and 3.69% of the total post-training spectral variance on PACS, OfficeHome, and VLCS, respectively, with an average of 2.70%. These results suggest that the initial dimension selection remains relatively stable in terms of the affected spectral variance, providing empirical support for its use under representation learning.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.