- Rating
- 1
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper studies how acceptance regions should retain source class geometry when recognizing known classes and rejecting unknown classes in an unseen domain. AdaSpike represents each class by a spherical tangent space support with distinct scales along selected principal directions and a shared residual scale. A robust optimization of a coverage and volume risk bound yields a spiked spectrum whose number of retained directions depends on an uncertainty radius. The implementation estimates this radius from the largest discrepancy between source class second moments and selects the dimension before training. It then learns representations and support scales through classification, coverage on held out source domains, and log volume objectives. The CLIP version additionally uses generated pseudo unknowns. Experiments cover PACS, OfficeHome, and VLCS with ResNet 18 and CLIP. Reported average results outperform the compared methods, and supplementary experiments examine fixed dimensions, alternative support parameterizations, objective ablations, individual target domains, and computational overhead.
Critical strengths
-
The support construction offers a clear interpretation of the tradeoff between directional source structure and isotropic protection against uncertain shifts. The spectral derivation in Theorem 1 and Appendix A.5 provides a coherent solution to the stated robust coverage and volume optimization problem, rather than merely introducing an arbitrary matrix parameterization.
-
The empirical evaluation includes two substantially different representation settings and three benchmarks. Tables 1 and 2 show consistent gains in average OSCR, while Table 3, Table 6, Figures 4 to 6, and Tables 9 to 14 provide useful evidence about support structure, objective interactions, dimension choices, and variation across target domains.
Critical weaknesses
-
Observed source discrepancy does not calibrate target uncertainty. Proposition 1 and Appendix A.3 require a population domain radius plus estimation error, whereas Section 3.1 uses the observed source diameter. Even infinitely sampled sources can share identical geometry while an unseen domain differs substantially. Appendix C acknowledges omitted estimation error but does not justify estimating the population radius this way. Figure 3 offers benchmark evidence, not a calibrated guarantee. The central trust rule therefore does not inherit the theorem's robustness.
-
The guarantee does not cover learned geometry. Appendix A.3 assumes independent tangent displacements under a fixed mapping, but AdaSpike estimates prototypes and trains representations on the same samples. No uniform moment concentration or independent estimation split is supplied. Stopping gradients does not restore statistical independence, and the classifier uniformity in Appendices A.6 and A.7 does not resolve this gap. Moreover, the initially selected dimension remains fixed as geometry changes, without validating that its trust estimate remains appropriate.
-
The practical objective can favor support collapse. Equation (7) omits the robustness term that diverges as residual scales shrink, replacing theoretical coverage and volume with a boundary hinge and log volume. If class features collapse to distinct prototypes, coverage vanishes and classification can improve while log volume tends to negative infinity. Appendix B.4 gives no scale floor preventing this. This does not imply reported runs diverge, but the implemented optimization lacks the protection underlying its robust design.
-
Support volume need not control unknown acceptance. Assumption 1 requires bounded unknown density relative to full spherical surface measure. Learned representations can instead concentrate unknowns on lower dimensional sets or near known supports, making volume reduction ineffective or the bound uninformative. The paper neither validates this assumption nor tests such controlled unknown distributions. Since volume regularization is the ResNet version's only explicit unknown control, the applicability of the conditional rejection argument is central rather than incidental.
-
The adaptive trust rule's contribution is not isolated. Fixed dimension and support family comparisons do not test matched alternatives based on spectral energy, constant uncertainty, or source validation under the same objective. Table 6 studies losses only on PACS with ResNet. CLIP additionally uses pseudo unknown generation and exclusion loss without an ablation separating their benefits from adaptive spiked supports. The results demonstrate the full pipeline's performance, but do not establish that source discrepancy based trust explains its gains.
Questions for authors
See Weaknesses.
6 comments
W1. Linking observed source discrepancy to target uncertainty and dimension selection (2/6)
We acknowledge that the original theoretical analysis did not fully justify replacing the population uncertainty radius with the observed source discrepancy . Under the two-stage generative model of Blanchard et al. (2021), our revised theoretical analysis establishes a probabilistic connection between observed source discrepancy and unseen target geometry (Rev. Proposition 1), and links observed source discrepancy to adaptive spike dimension selection through robust optimization (Rev. Theorem 1). Revised Theorem 2 further retains an contribution from target domains outside the uncertainty set in the OSDG risk bound. The key results of Proposition 1 and Theorem 1 are summarised below:
Proposition 1. Probabilistic target coverage
Theorem 1. Adaptive spike dimension selection
k_c^\star = \\#\\{j: \hat\gamma_{c,j}>\tau_c\\}, \qquad \sum_j (\tau_c-\hat\gamma_{c,j})_+ = \hat b_c
Specifically, we replace the original Assumption 2 with the two-stage generative model of Blanchard et al. (2021), under which source and target domains are independently sampled from a common meta-distribution. Under this assumption, revised Proposition 1 constructs an uncertainty set using the observed source discrepancy together with an additional estimation radius, and establishes a probabilistic lower bound on target coverage. Revised Theorem 1 shows that robust optimisation over this uncertainty set yields an optimal spike dimension determined by the observed source spectrum and , with the additional estimation radius leaving the selected directions unchanged. Revised Theorem 2 introduces an additional risk term to account for target domains outside the uncertainty set, accounting for cases where the target geometry falls outside the uncertainty set.
However, our theoretical analysis is under explicit assumptions and the probabilistic coverage bound can be loose when the number of source domains is small. We therefore provided complementary empirical evidence in Figure 3 (both Orig. and Rev.), showing that source-estimated geometry captures much of the observed target-domain geometric variation on the evaluated datasets.
References
Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.
W2. Theoretical scope and stability of dimension selection under learned representations (3/6)
We acknowledge that our theoretical analysis is conducted in a fixed feature space. Our intention is to characterise how source discrepancy informs the selection of the support dimension. We do not claim that our theory provides guarantees throughout the neural representation learning process. We have acknowledged this limitation in the Limitations section (Rev. Appendix C).
To empirically assess whether the initial dimension selection remains appropriate as the representation changes during training, we recomputed (averaged across all classes and folds) using the learned representations after training and compared it with the initial selection (Rev. Appendix B.7, Table 12). The results are summarised below.
Table: Stability of dimension selection under representation learning.
| Dataset | Initial | Re-estimated | Affected Spectral Mass (%) ↓ |
|---|---|---|---|
| PACS | 46.1 | 41.8 | 1.68 |
| OfficeHome | 50.7 | 43.1 | 2.63 |
| VLCS | 52.2 | 39.9 | 3.69 |
| Average | 49.7 | 41.6 | 2.70 |
Although the re-estimated differs from its initial value, the affected dimensions account for only 1.68%, 2.63%, and 3.69% of the total post-training spectral variance on PACS, OfficeHome, and VLCS, respectively, with an average of 2.70%. These results suggest that the initial dimension selection remains relatively stable in terms of the affected spectral variance, providing empirical support for its use under representation learning.
Response to Reviewer h5qn (1/6)
We appreciate the reviewer for the careful and constructive assessment of our work, particularly for recognising the theoretical derivation of the spiked support structure (Strength 1). In response to the reviewer's concerns, we have revised our theoretical analysis and conducted additional empirical evaluations addressing the practical method. The main theoretical revisions are detailed in our response to Weakness 1. Before addressing the specific concerns, we would like to clarify the scope of our theoretical and empirical contributions.
As stated in the original Introduction, AdaSpike is a “theoretically motivated framework”, without claiming end-to-end theoretical guarantees for the practical method. Our theoretical analysis considers fixed representations and class prototypes under explicit assumptions, including the two-stage generative model adopted from Blanchard et al. (2021) and a bounded-density assumption for unknowns. Within this setting, our analysis motivates the support structure and dimension-selection rule under explicit assumptions, while the effectiveness of the learned method is assessed empirically. We have clarified the role and scope of our theory in the revised version.
Our contributions can be distinguished at three levels.
Structural results: Revised Theorem 1 characterizes the optimal spiked support from a robust optimisation problem and establish how source discrepancy determines the number of retained directions.
Conditional bounds: Revised Proposition 1 provides a probabilistic bound relating observed source discrepancy to unseen target geometry, while revised Theorem 2 accounts for target domains outside the resulting uncertainty set in an OSDG risk bound.
Practical implementation: AdaSpike updates representations and class prototypes using surrogate objectives while retaining the initially selected spike dimension. While our theoretical analysis does not provide an end-to-end training guarantee, we have provided additional empirical evidence concerning dimension stability (W2), training dynamics alongside clarification of the positive scale floor (W3), and the effect of volume regularisation on unknown false acceptance (W4). We also add matched comparisons with alternative dimension-selection rules and CLIP loss ablations to assess their respective contributions (W5).
We acknowledge that our theoretical analysis has limitations. In particular, it assumes fixed representations and class prototypes, whereas these are updated during practical training. Moreover, the probabilistic bound in Proposition 1 can be loose when the number of source domains S is small. We have acknowledged and discussed these limitations in the revised Limitations section (Appendix C).
References
Blanchard, G., Deshmukh, A. A., Dogan, Ü., Lee, G., & Scott, C. (2021). Domain generalization by marginal transfer learning. Journal of Machine Learning Research, 22(2), 1–55.