- Rating
- 3(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ASFP which is a classifier-aware perturbation method that does pairwise differences between class-weight vectors and uses it to perturbed classifier heads with a SmoothMax-style objective to improve domain generalization. The experiments shows ASFP evaluation on popular DG benchmarks : VLCS, PACS, OfficeHome, TerraIncognita, and DomainNet using a ResNet-50 backbone.
Critical strengths
The strength of the paper are:
- ASFP is lightweight and jointly optimizes with the standard network, with no extra cost for gradient search.
- The method is computationally cheaper to run than its counterparts.
Critical weaknesses
The weakneses are as follows:
- Perturbation-based training improving generalization is already an established idea in the field of DG and OOD-G hence not something novel.
- The most important unresolved issue is whether the specific geometry-guided construction provides a meaningful advantage over simpler perturbation-based regularization. The conditional theory does not settle this question, and the current ablations do not isolate the claimed mechanism.
- Replacing Smoothloss with Ave. loss has substantial decrement in accuracy. Therefore, it is not sure the gain is coming from the perturbation based regularization or not.
- In the paper it is claimed that : ASFP promotes local functional flatness along the explored perturbation directions. There is no any loss landscape comaparison with SAM or SAGM that advocate similar idea. After joint optimization of perturbed function with the network, how does the loss change is not clear.
- Much of the reported baseline results are not evaluation on the similar experimental configuration. Can the authors provide paired comparisons and uncertainty estimates against the SAM baselines under the same evaluation protocol and training budget?
Minor weaknesses:
- In line 151, shouldnt it be s^(0) = g_w^(0)(z), relative to Fig. 1?
- line 154 not eqn numbered?
- Results are reported only on RN-50. GEneralization to other network is not substantiated.
Questions for authors
- Does the geometry based perturbation outperform random direction perturbation in the same subspace under similar perturbation budget and compute? This should have ben the main motivating experiment in the paper.
- Can you explain Eqn (3) more intuitively?
- Do you have results for RotatedMNIST? In the source code it passed as a dataset argument, but cant find in the supplementary material. Also see weaknesses for other questions.
4 comments
Response to Reviewer HmHL (2/2)
[W5] Much of the reported baseline ...
We would first like to clarify that all reported results follow the DomainBed evaluation protocol. To further ensure a fair comparison, we additionally reproduce SAM and SAGM under more strictly controlled experimental settings, as detailed in Sec. F.4 of the revised manuscript. Specifically, we adopt matched training budgets and random seeds, fix the batch size at 32, and sample the remaining hyperparameters from their respective predefined search spaces.
As shown in Tab. R2, our ASFP achieves the highest average accuracy. In particular, ASFP outperforms SAGM on VLCS and PACS while remaining competitive on OfficeHome and DomainNet. These results further support the effectiveness of geometry-guided functional probing under matched experimental settings.
Tab. R2: Paired performance comparison with SAM-based methods under controlled experimental settings.
| VLCS | PACS | OfficeHome | DomainNet | Avg | |
|---|---|---|---|---|---|
| SAM | 78.1±0.5 | 85.3±0.3 | 67.8±0.5 | 43.0±0.5 | 68.6 |
| SAGM | 78.9±0.8 | 86.6±0.4 | 68.9±0.2 | 43.8±0.1 | 69.6 |
| ASFP | 79.5±0.3 | 87.8±0.5 | 68.8±0.3 | 43.1±0.2 | 69.8 |
[MW1] In line 151, shouldnt it ...
Yes, we have revised the corresponding text in the manuscript accordingly.
[MW2] line 154 not eqn numbered?
Yes, it is intentionally left unnumbered, as it is not referenced elsewhere in the manuscript.
[MW3] Generalization to other network is not substantiated.
In Sec. F.3 of the revised manuscript, we evaluate ASFP using the ViT-B/16 architecture. From Tab. R3, ASFP achieves the best overall performance and outperforms competing methods in most individual cases, further demonstrating its effectiveness across different backbone architectures.
Tab. R3: Accuracy Comparison on PACS using ViT-B/16 backbone. The results of all competing methods reported are taken from DA-SAM.
| A | C | P | S | Avg | |
|---|---|---|---|---|---|
| ERM | 96.5 | 95.3 | 96.2 | 86.5 | 93.7 |
| SWAD | - | - | - | - | 91.3 |
| SMA | - | - | - | - | 92.1 |
| DA-SAM | 97.6 | 94.4 | 99.6 | 89.5 | 95.3 |
| ASFP | 97.4 | 95.7 | 99.6 | 89.6 | 95.6 |
[Q2] Can you explain Eqn (3) more intuitively?
Intuitively, Eq. (3) transforms the geometry-guided direction obtained from Eq. (2) into a controlled classifier perturbation. Specifically, the normalization separates the perturbation direction from its magnitude, while controls the relative perturbation strength across classes. As the classifier weights evolve during training, these directions adapt to the learned classifier geometry, enabling diverse functional exploration that reflects the changing discriminative relationships between classes. We have clarified this intuition in the revised manuscript.
[Q3] Do you have results for RotatedMNIST?
In Tab. 7 of the revised manuscript, we report the results on RotatedMNIST. From Tab. R4, our ASFP achieves the best average performance and outperforms competing methods on almost cases, further demonstrating its effectiveness under diverse domain shifts. Notably, the improvements are more pronounced on the 0° and 75° domains, whose rotation angles fall outside the ranges covered by the source domains, suggesting the effectiveness of geometry-guided functional probing in challenging extrapolation scenarios. More detailed analyses are presented in Lines 1188-1198 of the revised manuscript.
Tab. R4: Overall results on the RotatedMNIST benchmark.
| 0 | 15 | 30 | 45 | 60 | 75 | Avg | |
|---|---|---|---|---|---|---|---|
| ERM | 95.1±0.6 | 98.7±0.2 | 98.7±0.2 | 98.7±0.2 | 98.8±0.1 | 95.6±0.4 | 97.6 |
| IRM | 95.5±0.4 | 98.7±0.2 | 98.7±0.1 | 98.5±0.3 | 98.7±0.1 | 96.1±0.1 | 97.7 |
| VREx | 95.7±0.6 | 98.9±0.2 | 98.7±0.1 | 98.9±0.2 | 98.9±0.1 | 95.8±0.4 | 97.8 |
| GGA | 95.1±0.8 | 99.0±0.1 | 99.0±0.3 | 98.8±0.1 | 98.8±0.2 | 96.1±0.4 | 97.8 |
| SAGM | 95.4±0.4 | 98.9±0.1 | 98.9±0.1 | 98.9±0.1 | 98.9±0.1 | 95.9±0.5 | 97.8 |
| ASFP | 96.4±0.2 | 98.8±0.0 | 99.2±0.1 | 99.1±0.1 | 99.2±0.0 | 96.5±0.3 | 98.2 |
Thank you authors for the careful rebuttal of my comments. Much of my concerns have been addressed through additional experiments and explanation.
While some of the gains such as across PACS is marginal compared to SAGM, the methodology is seemingly sound and explores new direction in perturbation based training complementing its other counterparts SAM, SAGM etc. On the other note, since no any comparison has been made with the baselines regarding computational complexity, it is better to remove the term "lightweight framework" from abstract and wherever it appears.
Overall, due to methodological soundness, experimental rigor, and supporting theoretical analysis, I have decide to update my assessment to weak accept (3).
We sincerely thank the reviewer for their thoughtful reassessment and positive feedback. We greatly appreciate the recognition of our methodological contribution, experimental rigor, and theoretical analysis.
Regarding computational efficiency, we would like to clarify that Tab. 3 provides an empirical comparison of per-batch training costs with several competing methods, including SAGM and DA-SAM, where ASFP consistently demonstrates lower training overhead. Nevertheless, we appreciate the reviewer's suggestion and will refine the wording of "lightweight framework" to avoid potential ambiguity regarding computational complexity.
Response to Reviewer HmHL (1/2)
We sincerely thank the reviewer for the insightful feedback.
We agree that perturbation-based training is an established strategy for improving generalization. Within this broad family, however, ASFP makes a distinct methodological contribution through geometry-guided functional probing. Unlike conventional approaches relying on generic parameter perturbations, ASFP exploits the learned classifier geometry to construct diverse functional probes through pairwise class-weight differences, explicitly targeting predictive variations relevant to inter-class discrimination under potential domain shifts. Furthermore, our additional ablations demonstrate that geometry-guided probing provides benefits beyond generic perturbation-based regularization.
We thank the reviewers for these valuable comments. To better disentangle the contributions of geometry-guided probing and SmoothMax, we have introduced additional ablation studies in Tab. 2 of the revised manuscript.
First, regarding geometry-guided probing, as shown in Tab. R1, we compare ASFP with Isotropic and Shuffle. In particular, Isotropic and Shuffle preserve the perturbation count, magnitude, and SmoothMax objective, while modifying the perturbation directions. ASFP consistently outperforms these alternatives across all five benchmarks, demonstrating that the improvements cannot be attributed solely to generic perturbation-based regularization.
Second, regarding SmoothMax, replacing it with Ave Loss or Max Loss leads to substantial performance degradation, indicating that the loss aggregation strategy also plays an important role. Unlike averaging or hard maximization, SmoothMax emphasizes challenging probing directions while retaining contributions from the others.
Together, these results highlight the distinct roles of the two components that geometry-guided probing determines which classifier-relevant functional variations to explore, whereas SmoothMax determines how these variations contribute to optimization. Their combination enables more effective functional exploration and improves generalization under unseen domain shifts.
More detailed analyses are presented in Lines 429-468 of the revised manuscript.
Tab. R1: Ablation Studies
We thank the reviewer for this valuable suggestion. We would like to clarify that ASFP aims to promote local functional flatness along classifier-geometry-guided directions, rather than generic parameter-space flatness. Unlike SAM and SAGM, which primarily investigate sharpness under parameter perturbations, our analysis focuses on loss sensitivity along classifier-relevant functional directions.
To better illustrate this behavior, we provide empirical analyses in Figs. 3 and 6 of the revised manuscript. Specifically, Fig. 3(A) shows how the classification loss evolves during joint optimization, while Fig. 3(B)(C) demonstrates that the curvature and sensitivity regularization terms gradually decrease toward convergence, consistent with Prop. 2. Furthermore, Fig. 6(A)(D) shows that the loss range across different probing directions generally decreases during training on both toy data and PACS, indicating increasingly consistent losses across the explored functions. Additionally, Fig. 6(C)(F) examines how closely the explored functional variations approximate target-induced predictive deviations, further connecting the probing geometry to unseen domain shifts.
Together, these observations provide empirical support for the local functional flatness promoted by ASFP along classifier-relevant directions.