acceptodds
Under review as a conference paper at ICLR 2027

Anticipating Domain Shifts Via Geometry-Guided Functional Probing

Abstract

Domain generalization (DG) aims to learn a model from multiple source domains that remains reliable on unseen target domains. Recent studies have sought flat minima in the loss landscape to enhance robustness to diverse input variations, thereby mitigating domain shift in classification tasks. However, these methods typically rely on generic parameter-space perturbations to explore flatness, which not only lacks explicit grounding in class-relevant predictive variations but also suffers from additional perturbation-searching overhead. This motivates us to leverage the learned geometry of the classifier's decision boundary to randomly construct diverse predictive variations in anticipation of unseen domain shifts. To this end, we propose Anticipating Shifts via Functional Probing (ASFP), a lightweight framework that promotes local flatness along the explored functional directions by jointly optimizing a family of perturbed predictive functions. Specifically, ASFP constructs perturbation directions that evolve with the classifier by randomly combining pairwise differences between its class-weight vectors, enabling structured functional exploration without costly perturbation search. *Theoretically*, with the SmoothMax objective, we characterize the induced curvature and sensitivity regularization and establish a local bound on loss increases under unseen-domain perturbations covered by this geometry. *Empirically*, we demonstrate through extensive experiments on five benchmarks that ASFP consistently outperforms strong baselines while maintaining low training overhead.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
Domain Generalization, Functional Probes, Random Perturbation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_HmHLOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ASFP which is a classifier-aware perturbation method that does pairwise differences between class-weight vectors and uses it to perturbed classifier heads with a SmoothMax-style objective to improve domain generalization. The experiments shows ASFP evaluation on popular DG benchmarks : VLCS, PACS, OfficeHome, TerraIncognita, and DomainNet using a ResNet-50 backbone.

Critical strengths

The strength of the paper are:

  1. ASFP is lightweight and jointly optimizes with the standard network, with no extra cost for gradient search.
  2. The method is computationally cheaper to run than its counterparts.
Critical weaknesses

The weakneses are as follows:

  1. Perturbation-based training improving generalization is already an established idea in the field of DG and OOD-G hence not something novel.
  2. The most important unresolved issue is whether the specific geometry-guided construction provides a meaningful advantage over simpler perturbation-based regularization. The conditional theory does not settle this question, and the current ablations do not isolate the claimed mechanism.
  3. Replacing Smoothloss with Ave. loss has substantial decrement in accuracy. Therefore, it is not sure the gain is coming from the perturbation based regularization or not.
  4. In the paper it is claimed that : ASFP promotes local functional flatness along the explored perturbation directions. There is no any loss landscape comaparison with SAM or SAGM that advocate similar idea. After joint optimization of perturbed function with the network, how does the loss change is not clear.
  5. Much of the reported baseline results are not evaluation on the similar experimental configuration. Can the authors provide paired comparisons and uncertainty estimates against the SAM baselines under the same evaluation protocol and training budget?

Minor weaknesses:

  1. In line 151, shouldnt it be s^(0) = g_w^(0)(z), relative to Fig. 1?
  2. line 154 not eqn numbered?
  3. Results are reported only on RN-50. GEneralization to other network is not substantiated.
Questions for authors
  1. Does the geometry based perturbation outperform random direction perturbation in the same subspace under similar perturbation budget and compute? This should have ben the main motivating experiment in the paper.
  2. Can you explain Eqn (3) more intuitively?
  3. Do you have results for RotatedMNIST? In the source code it passed as a dataset argument, but cant find in the supplementary material. Also see weaknesses for other questions.
4 comments
AuthorsOpenReview ↗

Response to Reviewer HmHL (1/2)

We sincerely thank the reviewer for the insightful feedback.

[W1] Perturbation-based training improving ...

We agree that perturbation-based training is an established strategy for improving generalization. Within this broad family, however, ASFP makes a distinct methodological contribution through geometry-guided functional probing. Unlike conventional approaches relying on generic parameter perturbations, ASFP exploits the learned classifier geometry to construct diverse functional probes through pairwise class-weight differences, explicitly targeting predictive variations relevant to inter-class discrimination under potential domain shifts. Furthermore, our additional ablations demonstrate that geometry-guided probing provides benefits beyond generic perturbation-based regularization.

[W2&W3&Q1] Contributions of Geometry-Guided Probing and SmoothMax

We thank the reviewers for these valuable comments. To better disentangle the contributions of geometry-guided probing and SmoothMax, we have introduced additional ablation studies in Tab. 2 of the revised manuscript.

First, regarding geometry-guided probing, as shown in Tab. R1, we compare ASFP with Isotropic and Shuffle. In particular, Isotropic and Shuffle preserve the perturbation count, magnitude, and SmoothMax objective, while modifying the perturbation directions. ASFP consistently outperforms these alternatives across all five benchmarks, demonstrating that the improvements cannot be attributed solely to generic perturbation-based regularization.

Second, regarding SmoothMax, replacing it with Ave Loss or Max Loss leads to substantial performance degradation, indicating that the loss aggregation strategy also plays an important role. Unlike averaging or hard maximization, SmoothMax emphasizes challenging probing directions while retaining contributions from the others.

Together, these results highlight the distinct roles of the two components that geometry-guided probing determines which classifier-relevant functional variations to explore, whereas SmoothMax determines how these variations contribute to optimization. Their combination enables more effective functional exploration and improves generalization under unseen domain shifts.

More detailed analyses are presented in Lines 429-468 of the revised manuscript.

Tab. R1: Ablation Studies

MethodVLCSPACSOfficeHomeTerraDomainNet
Baseline77.5±0.485.5±0.266.5±0.346.1±1.840.9±0.3
Isotropic78.9±0.287.3±0.267.3±0.149.2±1.742.6±0.5
Shuffle79.0±0.386.4±0.168.0±0.349.8±2.042.9±0.4
Ave. Loss78.0±0.586.3±0.566.5±0.247.3±2.141.1±0.5
Max Loss77.8±0.885.7±0.668.1±0.648.0±1.841.3±0.5
Ours80.4±0.088.3±0.469.8±0.153.6±0.744.3±0.1

[W4] In the paper it ...

We thank the reviewer for this valuable suggestion. We would like to clarify that ASFP aims to promote local functional flatness along classifier-geometry-guided directions, rather than generic parameter-space flatness. Unlike SAM and SAGM, which primarily investigate sharpness under parameter perturbations, our analysis focuses on loss sensitivity along classifier-relevant functional directions.

To better illustrate this behavior, we provide empirical analyses in Figs. 3 and 6 of the revised manuscript. Specifically, Fig. 3(A) shows how the classification loss evolves during joint optimization, while Fig. 3(B)(C) demonstrates that the curvature and sensitivity regularization terms gradually decrease toward convergence, consistent with Prop. 2. Furthermore, Fig. 6(A)(D) shows that the loss range across different probing directions generally decreases during training on both toy data and PACS, indicating increasingly consistent losses across the explored functions. Additionally, Fig. 6(C)(F) examines how closely the explored functional variations approximate target-induced predictive deviations, further connecting the probing geometry to unseen domain shifts.

Together, these observations provide empirical support for the local functional flatness promoted by ASFP along classifier-relevant directions.

AuthorsOpenReview ↗

Response to Reviewer HmHL (2/2)

[W5] Much of the reported baseline ...

We would first like to clarify that all reported results follow the DomainBed evaluation protocol. To further ensure a fair comparison, we additionally reproduce SAM and SAGM under more strictly controlled experimental settings, as detailed in Sec. F.4 of the revised manuscript. Specifically, we adopt matched training budgets and random seeds, fix the batch size at 32, and sample the remaining hyperparameters from their respective predefined search spaces.

As shown in Tab. R2, our ASFP achieves the highest average accuracy. In particular, ASFP outperforms SAGM on VLCS and PACS while remaining competitive on OfficeHome and DomainNet. These results further support the effectiveness of geometry-guided functional probing under matched experimental settings.

Tab. R2: Paired performance comparison with SAM-based methods under controlled experimental settings.

VLCSPACSOfficeHomeDomainNetAvg
SAM78.1±0.585.3±0.367.8±0.543.0±0.568.6
SAGM78.9±0.886.6±0.468.9±0.243.8±0.169.6
ASFP79.5±0.387.8±0.568.8±0.343.1±0.269.8

[MW1] In line 151, shouldnt it ...

Yes, we have revised the corresponding text in the manuscript accordingly.

[MW2] line 154 not eqn numbered?

Yes, it is intentionally left unnumbered, as it is not referenced elsewhere in the manuscript.

[MW3] Generalization to other network is not substantiated.

In Sec. F.3 of the revised manuscript, we evaluate ASFP using the ViT-B/16 architecture. From Tab. R3, ASFP achieves the best overall performance and outperforms competing methods in most individual cases, further demonstrating its effectiveness across different backbone architectures.

Tab. R3: Accuracy Comparison on PACS using ViT-B/16 backbone. The results of all competing methods reported are taken from DA-SAM.

ACPSAvg
ERM96.595.396.286.593.7
SWAD----91.3
SMA----92.1
DA-SAM97.694.499.689.595.3
ASFP97.495.799.689.695.6

[Q2] Can you explain Eqn (3) more intuitively?

Intuitively, Eq. (3) transforms the geometry-guided direction obtained from Eq. (2) into a controlled classifier perturbation. Specifically, the normalization separates the perturbation direction from its magnitude, while γ∣∣wc∣∣2\gamma||\mathbf{w}_c||_2 controls the relative perturbation strength across classes. As the classifier weights evolve during training, these directions adapt to the learned classifier geometry, enabling diverse functional exploration that reflects the changing discriminative relationships between classes. We have clarified this intuition in the revised manuscript.

[Q3] Do you have results for RotatedMNIST?

In Tab. 7 of the revised manuscript, we report the results on RotatedMNIST. From Tab. R4, our ASFP achieves the best average performance and outperforms competing methods on almost cases, further demonstrating its effectiveness under diverse domain shifts. Notably, the improvements are more pronounced on the 0° and 75° domains, whose rotation angles fall outside the ranges covered by the source domains, suggesting the effectiveness of geometry-guided functional probing in challenging extrapolation scenarios. More detailed analyses are presented in Lines 1188-1198 of the revised manuscript.

Tab. R4: Overall results on the RotatedMNIST benchmark.

01530456075Avg
ERM95.1±0.698.7±0.298.7±0.298.7±0.298.8±0.195.6±0.497.6
IRM95.5±0.498.7±0.298.7±0.198.5±0.398.7±0.196.1±0.197.7
VREx95.7±0.698.9±0.298.7±0.198.9±0.298.9±0.195.8±0.497.8
GGA95.1±0.899.0±0.199.0±0.398.8±0.198.8±0.296.1±0.497.8
SAGM95.4±0.498.9±0.198.9±0.198.9±0.198.9±0.195.9±0.597.8
ASFP96.4±0.298.8±0.099.2±0.199.1±0.199.2±0.096.5±0.398.2

Thank you authors for the careful rebuttal of my comments. Much of my concerns have been addressed through additional experiments and explanation.

While some of the gains such as across PACS is marginal compared to SAGM, the methodology is seemingly sound and explores new direction in perturbation based training complementing its other counterparts SAM, SAGM etc. On the other note, since no any comparison has been made with the baselines regarding computational complexity, it is better to remove the term "lightweight framework" from abstract and wherever it appears.

Overall, due to methodological soundness, experimental rigor, and supporting theoretical analysis, I have decide to update my assessment to weak accept (3).

We sincerely thank the reviewer for their thoughtful reassessment and positive feedback. We greatly appreciate the recognition of our methodological contribution, experimental rigor, and theoretical analysis.

Regarding computational efficiency, we would like to clarify that Tab. 3 provides an empirical comparison of per-batch training costs with several competing methods, including SAGM and DA-SAM, where ASFP consistently demonstrates lower training overhead. Nevertheless, we appreciate the reviewer's suggestion and will refine the wording of "lightweight framework" to avoid potential ambiguity regarding computational complexity.

Reviewer_RjpCOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes a lightweight framework for domain generalization that promotes local flatness along diverse functional perturbation directions. ASFP constructs classifier-adaptive perturbations by randomly combining pairwise differences between class-weight vectors, enabling structured functional exploration without gradient-based perturbation search. The resulting predictive functions share the same backbone and are jointly optimized using a temperature-controlled SmoothMax objective that emphasizes high-loss functional variations. Experiments on five domain generalization benchmarks demonstrate competitive performance with lower per-batch training costs than the compared methods.

Critical strengths
  1. The paper is clearly written, with an intuitive illustration of the proposed framework and a detailed theoretical analysis.

  2. The method is simple to implement and shares backbone features across perturbed classifiers, avoiding gradient-based perturbation search.

  3. Experiments cover five widely used domain generalization benchmarks.

  4. The paper includes ablation studies, hyperparameter sensitivity analysis, and training-time comparisons, and provides code in the supplementary materials.

Critical weaknesses
  1. The method relies on an affine classification head, as its perturbations are constructed from pairwise differences between class-weight vectors. It is unclear how this construction and the associated theoretical analysis extend to other models. Although a nonlinear network may still have a final linear layer, applying ASFP only to that layer limits the scope of the proposed functional probing, and this architectural restriction deserves further discussion.

  2. The experimental comparison omits some closely related method, e.g., SFT, MeCAM. SFT reports an average accuracy of 67.1% across the same five benchmarks with ResNet-50, close to ASFP’s 67.3%. A comparison under matched settings is needed to establish the claimed performance advantage over strong existing methods.

  3. The current ablation shows that SmoothMax outperforms averaging the losses, but does not compare geometry-guided directions with ordinary random directions under the same perturbation count, magnitude, and SmoothMax objective.

  4. The experiments are limited to ResNet-50, leaving the effectiveness of ASFP across different backbone architectures unverified.

  5. The gains over the strongest reported baselines are uneven across datasets. ASFP improves the average accuracy by 1.0 percentage point, corresponding to a relative improvement of 1.51%, and the relative gains on VLCS and PACS are only 0.63% and 0.91%, respectively, while performance is lower on OfficeHome and DomainNet.

Questions for authors
  1. Can ASFP extend to classifiers without an affine output head, or is this structure a necessary assumption?

  2. How does ASFP compare with more sota baselines?

  3. What explains the uneven gains across datasets?

3 comments
AuthorsOpenReview ↗

Response to Reviewer RjpC

We sincerely thank the reviewer for the insightful feedback.

[W1&Q1] Can ASFP extend to classifiers without an affine output head ...

We would like to clarify that the affine structure is required for our geometry-guided perturbation construction, but does not restrict the overall classifier to an affine function. Specifically, the affine readout allows us to explicitly exploit pairwise class-weight differences to construct classifier-relevant functional probes, which is central to the motivation of our method. Although the geometric characterization in Lemma 1 relies on this structure, the proposed probing strategy can still be applied to more complex nonlinear classifiers.

To further demonstrate this applicability, we extend ASFP to two nonlinear classifier architectures involving nonlinear feature transformations and nonlinear decision functions, respectively, as detailed in Sec. F.1 (Page 20) of the revised manuscript. From Tab. R1, our ASFP consistently outperforms ERM across all five benchmarks with both nonlinear heads. These results demonstrate the effectiveness of geometry-guided functional probing beyond affine classifiers, including architectures with nonlinear feature transformations and nonlinear decision functions.

Tab. R1: Performance comparison between ERM and ASFP with different classifier architectures across five benchmarks. DNH denotes the DomainBed nonlinear head with nonlinear feature transformations, and CNH denotes the custom nonlinear head with nonlinear decision functions.

HeadMethodVLCSPACSOfficeHomeTerraIncognitaDomainNetAvg
DNHERM79.6±0.387.0±0.466.5±0.250.6±0.242.2±0.365.2
DNHASFP81.0±0.188.6±0.370.5±0.254.4±0.345.9±0.268.1
CNHERM76.0±0.586.9±0.165.7±0.146.9±2.140.6±0.563.2
CNHASFP79.6±0.188.5±0.268.1±0.253.8±1.044.4±0.266.9

[W2&Q2] How does ASFP compare with more sota baselines?

In the revised manuscript, we have included MeCAM and SFT as additional baselines in Tab. 1, along with the training cost of SFT in Tab. 3. We further clarify that all reported results are evaluated under the same DomainBed protocol to ensure a fair comparison.

Overall, as shown in Tab. 1, our ASFP (67.3%) outperforms MeCAM (66.6%) and achieves comparable yet slightly better average performance than SFT (67.1%), with varying advantages across individual benchmarks. More importantly, ASFP incurs substantially lower per-batch training costs than SFT, as shown in Tab. 3, highlighting its favorable balance between generalization performance and computational efficiency. More detailed analyses are presented in Lines 429-468 and 499-502 of the revised manuscript.

[W3] Additional comparisons with random direction.

In Tab. 2 of the revised manuscript, we introduce some additional ablation studies to assess the contribution of classifier geometry. From Tab. R2, our ASFP consistently outperforms Isotropic, highlighting the effectiveness of class-geometry-guided perturbations over generic random perturbations. More detailed analyses are presented in Lines 429-468 of the revised manuscript.

Tab. R2: Ablation Studies

VLCSPACSOfficeHomeTerraDomainNet
Baseline77.5±0.485.5±0.266.5±0.346.1±1.840.9±0.3
Isotropic78.9±0.287.3±0.267.3±0.149.2±1.742.6±0.5
Ours80.4±0.088.3±0.469.8±0.153.6±0.744.3±0.1

[W4] The experiments are limited to ResNet-50.

In Sec. F.3 of the revised manuscript, we evaluate ASFP using the ViT-B/16 architecture. From Tab. R3, ASFP achieves the best overall performance and outperforms competing methods in most individual cases, further demonstrating its effectiveness across different backbone architectures.

Tab. R3: Accuracy Comparison on PACS using ViT-B/16 backbone. The results of all competing methods reported are taken from DA-SAM.

ACPSAvg
ERM96.595.396.286.593.7
SWAD----91.3
SMA----92.1
DA-SAM97.694.499.689.595.3
ASFP97.495.799.689.695.6

[W5&Q3] What explains the uneven gains across datasets?

We thank the reviewer for this question. We believe that varying relative gains across benchmarks are expected, as different datasets exhibit distinct domain shifts and inter-class relationships, which may favor different generalization strategies. Specifically, ASFP constructs functional probes based on the classifier geometry learned from source domains, and their effectiveness may vary depending on how unseen domain shifts affect the learned discriminative relationships. Indeed, strong methods such as SFT also exhibit varying advantages across benchmarks. However, ASFP consistently improves upon ERM across all benchmarks and achieves the highest overall average accuracy. Furthermore, ASFP incurs substantially lower training costs than SFT, highlighting its favorable balance between generalization performance and computational efficiency.

Thank you for the revisions and additional experiments, which address several of my concerns. However, my concern about the empirical advantage remains. Relative accuracy changes over the best baseline are +0.12%, 0%, −1.55%, +5.72%, and −3.70%, with a meaningful improvement on only one dataset. With ViT-B/16, the changes across the four PACS domains are −0.20%, +0.42%, 0%, and +0.11%, respectively, offering limited additional support for a convincing performance advantage. The reported efficiency gains are encouraging, although the modest and uneven accuracy improvements leave the overall advantage over baselines less clear. Therefore, I am inclined to maintain my original score.

We sincerely thank the reviewer for acknowledging the computational efficiency of ASFP. While we recognize that the accuracy gains vary across benchmarks, we believe that achieving competitive or even better generalization performance with substantially lower training costs (approximately one-third of SFT's per-batch training cost) represents a meaningful contribution, particularly in terms of the performance-efficiency trade-off.

We appreciate the reviewer's careful assessment and hope this perspective helps highlight the overall empirical value of ASFP.

Reviewer_RbZaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes ASFP, a flatness oriented domain generalization method. Instead of searching for parameter-space perturbations as SAM-style methods do, ASFP builds M perturbed copies of the linear classification head. For each class c, the perturbation direction is a fixed random combination of the pairwise class-weight contrasts Gc=[wj−wc]j≠cG_c = [w_j − w_c]_{j≠c}. The perturbation is rescaled proportionally to ‖wc‖‖w_c‖. All heads share the backbone and are trained jointly, placing more weight on high-loss heads.

Empirically, ASFP is evaluated on a rotated toy problem (3 classes) and on DomainBed benchmarks. It reports the best average accuracy.

Critical strengths
  • Perturbations can be computed without gradient descent. Table 3 shows lower cost than SAGM and DA-SAM, meaning computational efficiency. It can a practical plug-in regularizer if its performances are good enough, and it is the reason why this paper is worth considering.
Critical weaknesses
  • The claim that the classifier-geometry-guided directions are what drive the gains is not isolated experimentally. Spefically, Table 2 contains no ablation in which the contrast-based directions GcG_c are replaced by random or isotropic directions under the same experimental conditions. Also, there is no comparison showing SAM applied only to the classification head, or against simple margin or confidence regularizers.

  • Table 2 shows that averaging the head losses gives results close to ERM, while SmoothMax with τ=0.1\tau = 0.1 gives the full improvement. With τ=0.1\tau = 0.1, SmoothMax is close to a hard max over heads. It means that the gain could be from a worst of M last-layer perturbation, which acts as a cheap margin- or adversarial-type regularizer on the features.

  • The theory does not connect to the regime or the benchmarks where the method is used. Prop. 1 is essentially Lipschitz continuity + cross entropy convexity. dist(eT,Ci)dist(e_T, C_i) is neither controlled by the training objective nor measured on any real benchmark.

  • Furthermore, Prop. 2 is an asymptotic expansion as ϵ→0\epsilon\rightarrow0 for fixed τ\tau. As τ=0.1\tau = 0.1 in practice, the logit perturbations are much larger than τ\tau, it is outside the regime the expansion describes.

Questions for authors

See above.

(+ SWAD is cited as a closely related flatness method but is not compared against. Please include it, and ideally ASFP + SWAD to test whether the two are complementary.)

2 comments
AuthorsOpenReview ↗

Response to Reviewer RbZa (1/2)

We sincerely thank the reviewer for the insightful feedback.

[W1] Additional comparisons with random perturbations or isotropic, and classifier-head-only SAM.

In Tab. 2 of the revised manuscript, we introduce three additional ablation studies to assess the contribution of classifier geometry, including isotropic random perturbations (Isotropic), shuffled class-geometry-guided perturbations (Shuffle), and classifier-head-only SAM (SAM-Head).

From Tab. R1, our ASFP consistently outperforms Isotropic, Shuffle, and SAM-Head, highlighting the effectiveness of class-geometry-guided perturbations over generic random perturbations and classifier-head-only sharpness optimization. More detailed analyses are presented in Lines 429-468 of the revised manuscript.

Tab. R1: Ablation Studies

MethodVLCSPACSOfficeHomeTerraDomainNet
Baseline77.5±0.485.5±0.266.5±0.346.1±1.840.9±0.3
Isotropic78.9±0.287.3±0.267.3±0.149.2±1.742.6±0.5
Shuffle79.0±0.386.4±0.168.0±0.349.8±2.042.9±0.4
SAM-Head78.5±0.485.7±0.967.9±0.647.5±1.043.0±0.2
Ave. Loss78.0±0.586.3±0.566.5±0.247.3±2.141.1±0.5
Max Loss77.8±0.885.7±0.668.1±0.648.0±1.841.3±0.5
Ours80.4±0.088.3±0.469.8±0.153.6±0.744.3±0.1

[W2] With τ\tau = 0.1, SmoothMax is close to a hard max over heads.

In Tab. 2 of the revised manuscript, we also introduce the Max Loss as an ablation, which exclusively optimizes the worst-case perturbation direction.

Although τ=0.1\tau=0.1 increases the emphasis on high-loss probes, SmoothMax differs from hard maximum optimization by retaining contributions from all probing directions, thereby promoting flatness across diverse functional variations. In contrast, Max Loss exclusively optimizes the worst-case direction. As shown in Tab. R1, although Max Loss improves upon the baseline, it consistently underperforms our SmoothMax objective across all five benchmarks. Furthermore, the revised Fig. 6 (A) and (D) show that the temperature-normalized head-loss range gradually approaches zero during training, indicating diminishing loss differences across probing directions and further supporting the effectiveness of our joint optimization strategy.

[W3] Practical Relevance of Prop. 1.

We thank the reviewer for this insightful comment. We would like to clarify that Prop. 1 establishes a coverage-dependent loss bound, connecting the SmoothMax objective and the coverage of target-induced predictive variations to the target loss. Specifically, SmoothMax directly controls the losses of the explored functional probes, while the convex-hull distance characterizes how well these probes cover unseen predictive variations. Together, they provide explicit loss control for well-covered domain shifts, offering a theoretical interpretation of geometry-guided functional probing. We have revised the corresponding discussion in Lines 270–276 of the revised manuscript to clarify this interpretation.

To further connect this analysis to practical scenarios, we have added empirical evaluations in Fig. 6 (F) of the revised manuscript. Beyond the controlled rotated toy dataset, we further estimate the convex-hull distance dist(eT,Ci)dist(e_T, C_i) on the real-world PACS benchmark. These experiments provide empirical insights into the coverage of geometry-guided functional probes under realistic domain shifts. More detailed analyses are presented in Lines 1052-1064 of the revised manuscript.

[W4] Furthermore, Prop. 2 is ...

We would like to clarify that Prop. 2 provides a local interpretation of the SmoothMax objective, revealing its implicit curvature- and sensitivity-dependent regularization along the explored functional directions. In practice, our ASFP directly optimizes the original SmoothMax objective at finite perturbation scales, rather than relying on its Taylor approximation.

To further examine its practical behavior, we provide additional empirical analyses in Fig. 6 of the revised manuscript, including both the controlled toy dataset and the real-world PACS benchmark. As shown in Figs. 6(A) and (B), both the temperature-normalized head-loss range and the Taylor approximation error gradually approach zero under the controlled setting, consistent with our theoretical analysis. On PACS, although the Taylor approximation error remains noticeable in Fig. 6(E), it does not increase further, while the head-loss range in Fig. 6(D) steadily decreases during training. These results provide empirical evidence that the original SmoothMax objective effectively reduces loss disparities across geometry-guided probing directions at practical perturbation scales, consistent with the local regularization mechanism characterized in Prop. 2.

AuthorsOpenReview ↗

Response to Reviewer RbZa (2/2)

[Q1] Please include SWAD, and ideally ASFP+SWAD.

In Tab. 5 of the revised manuscript on Page 21, we further evaluate several methods combined with SWAD. The results demonstrate that ASFP also benefits from SWAD and achieves the best performance across the evaluated benchmarks, highlighting the compatibility of geometry-guided functional probing with weight averaging.

Tab. R2: Performance Comparison w/ and w/o SWAD. All results except ours are taken from DA-SAM.

MethodVLCSPACSTerraAvgOfficeHomeDomainNet
ERM77.5±0.485.5±0.246.1±1.869.766.5±0.340.9±0.3
ERM+SWAD79.1±0.188.1±0.150.0±0.372.470.6±0.246.5±0.1
SAM78.2±0.585.3±1.045.7±0.969.768.0±0.843.4±0.1
SAM+SWAD78.587.145.370.3--
DA-SAM79.6±0.887.5±0.450.0±0.272.468.4±0.344.5±0.2
DA-SAM+SWAD79.188.050.972.7--
ASFP80.4±0.088.3±0.453.6±0.774.169.8±0.144.3±0.1
ASFP+SWAD80.0±0.189.0±0.154.3±0.274.471.6±0.147.1±0.1

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.