acceptodds
Under review as a conference paper at ICLR 2027

Navigating Potholes with Geometry-Aware Sharpness Minimization

Abstract

Sharpness-aware minimization (SAM) biases optimization toward flat neighborhoods by evaluating gradients at adversarially perturbed parameters, but standard SAM defines these perturbations in Euclidean geometry. Geometry-aware optimizers instead precondition the descent direction using curvature information. We combine these ideas using LLQR, which learns a structured, slowly varying inverse metric from layerwise network dynamics. LLQR+SAM uses this metric both to shape the SAM perturbation and to precondition the gradient evaluated at the perturbed parameters. On a two-scale quadratic model, we show that the learned geometry enlarges the SAM exploration scale in directions that are flat under the background geometry but locally sharp, providing a mechanism for escaping narrow sharp wells (“potholes”). Empirically, LLQR+SAM consistently improves over SAM and LLQR across CIFAR-10/100, TinyImageNet, ImageNet, and IWSLT14, with analogous gains when combined with recent SAM variants such as F-SAM. These results support the view that slow learned geometry and fast sharpness-aware perturbations can be complementary.

Primary area:
optimization
Keywords:
SAM, deep learning, optimization, geometry-aware

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_cPgsOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper propose SAM with LQLR that combines slowly updated learned geometry with sharpness-aware minimization. The method use LLQR to learn a structed inverse metric and use it to construct anisotropic perturbations. Among various experiment settings, the results show improvements over SAM and various SAM variants.

Critical strengths
  1. It is clear and reasonable to combine slowly updated geometry estimates with per-step sharpness-aware perturbations, reusing the same metric for perturbation and descent.

  2. The experiments cover multiple architectures and settings. The method improves performance in several settings and can also be combined with variants such as F-SAM and ASAM.

Critical weaknesses

1.LLQR+SAM primarily combines the existing LLQR with SAM. The main adaptation is to construct anisotropic perturbation with LLQR metric and reuse it during the descent stage. However, beyond choosing LLQR as an metric estimator, I think this paper does not clearly establish a distinctive design specifically addressing localized sharp basins. Meanwhile, their are many existing algorithms such as FisherSAM discuss anisotropic geometry for the SAM perturbation. Thus, I think the contribution beyond the combination therefore remains insufficiently differentiated.

2.The ablation results do not sufficiently establish the need to apply the learned geometry to both perturbation and descent(Table16 and 19). The authors should clarify when this component helps and better establish the value of sharing geometry.

3.Evaluations on domain generalization, single-source domain generalization, or distribution-shift benchmarks would help clarify whether the method have better generalization ability under difficult generalization settings.

Minor: Some text in the figures is too small to read comfortably at normal viewing size.

Questions for authors

See weaknesses.

1 comment

Thank you for the suggestions.

W1 — Novelty relative to FisherSAM. The question we study is the combination of coarse geometry learned on a slow timescale with per-step sharpness-aware optimization, rather than anisotropic probing alone. FisherSAM modifies the perturbation using a diagonal Fisher; LLQR provides a structured outer preconditioner and can also shape the probe. On TinyImageNet/ResNet-50, FisherSAM attains 71.69 ± 0.42%, near SAM's 71.63 ± 0.59%, versus 73.14 ± 0.26% for full LLQR+SAM. Adding LLQR outer preconditioning to FisherSAM improves CIFAR-100/ResNet-18 by 0.67 points [0.30, 1.02]; restricting LLQR to a diagonal metric reduces outer-only accuracy by 0.75 points [−1.08, −0.42]. The Fisher K-FAC comparison further shows that the improvement is not exclusive to LLQR (reply to eVJg). These findings distinguish the contribution of structured outer geometry from that of the probe.

W2 — Is a shared metric needed? No: outer-only already captures most of the benefit (general reply, Table 1). Full coupling changes the exploration geometry, but does not always help: on CIFAR-10-C, full trails outer-only by 0.68 points [−1.11, −0.24], and on TinyViT by 1.85 points (reply to eVJg). We will revise the manuscript to avoid presenting full coupling as necessary.

W3 — Generalization under distribution shift. We evaluated final checkpoints on CIFAR-10.1, CIFAR-10-C, and CIFAR-100-C, without tuning on shifted data (Tables E–F). LLQR+SAM improves over SAM on CIFAR-10.1 (+0.78 points [0.47, 1.09]) and on CIFAR-100-C on average (+0.47 [0.23, 0.72]), but not on CIFAR-10-C on average (−0.37 [−1.10, 0.36]). Results differ markedly by corruption: gains on blur and weather, losses on additive noise and pixelation. We will report both the improvements and this limitation without claiming uniform robustness or a verified causal explanation.

Table E. Shifted test sets, ResNet-18, final checkpoints, five seeds. Accuracy (%) ↑; differences are LLQR+SAM − SAM with 95% paired t-intervals.

DatasetTest setSAMLLQR+SAMDifference
CIFAR-10, ρ = 0.1Clean96.68 ± 0.1296.80 ± 0.04+0.12 [−0.04, +0.28]
CIFAR-10.191.39 ± 0.2792.17 ± 0.04+0.78 [+0.47, +1.09]
Drop, clean − CIFAR-10.1 ↓5.29 ± 0.174.63 ± 0.07−0.66 [−0.84, −0.47]
CIFAR-10-C mean75.82 ± 0.3375.45 ± 0.32−0.37 [−1.10, +0.36]
CIFAR-100, ρ = 0.2Clean80.96 ± 0.1581.76 ± 0.17+0.80 [+0.52, +1.09]
CIFAR-100-C mean51.86 ± 0.1052.33 ± 0.19+0.47 [+0.23, +0.72]

CIFAR-C means average 15 corruptions over five severities. Clean entries are final-checkpoint accuracies, not the “best” scores used in the placement ablation and submission.

Table F. LLQR+SAM − SAM by corruption type (points), mean over five severities; five seeds, 95% paired t-intervals.

FamilyCorruptionCIFAR-10-CCIFAR-100-C
NoiseGaussian−6.53 [−8.31, −4.75]−3.80 [−5.19, −2.41]
Shot−5.85 [−7.71, −3.99]−4.10 [−5.25, −2.96]
Impulse−4.79 [−7.29, −2.29]−3.07 [−4.48, −1.67]
BlurDefocus+2.27 [+1.19, +3.34]+1.46 [+0.70, +2.23]
Glass+2.37 [+0.38, +4.36]+2.83 [+1.85, +3.80]
Motion+1.71 [+1.00, +2.43]+1.49 [+0.99, +1.98]
Zoom+3.25 [+1.54, +4.97]+2.13 [+1.11, +3.15]
WeatherSnow+1.39 [+1.08, +1.70]+2.80 [+2.72, +2.88]
Frost+1.92 [+0.66, +3.18]+2.71 [+1.89, +3.52]
Fog+0.48 [−0.02, +0.98]+1.38 [+1.04, +1.72]
Brightness+0.35 [+0.25, +0.45]+1.09 [+0.73, +1.45]
DigitalContrast−0.42 [−1.07, +0.22]+1.53 [+0.86, +2.20]
Elastic+0.56 [+0.10, +1.01]+1.57 [+1.34, +1.80]
Pixelate−1.81 [−3.41, −0.21]−0.87 [−1.31, −0.43]
JPEG−0.41 [−0.82, +0.01]−0.06 [−0.67, +0.55]

Minor. We will enlarge the figure labels.

References

Recht et al., Do CIFAR-10 Classifiers Generalize to CIFAR-10? CoRR abs/1806.00451 (2018)

Hendrycks and Dietterich, Benchmarking Neural Network Robustness to Common Corruptions and Perturbations (ICLR 2019)

Reviewer_eVJgOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper integrates SAM with LLQR—a structured inverse preconditioner UU that is learned from layer-wise dynamics and updated slowly. LLQR+SAM employs UU in two ways: first, to shape the SAM perturbation (ϵ=ρUg/∥g∥U\epsilon = \rho Ug/\|g\|_U); and second, to precondition the gradient at the perturbed point (Equation 1, Algorithm 1). Section 4 analyzes a two-scale quadratic model, consisting of a background curvature Hˉ\bar{H} plus a sharp component. Under the idealized assumption that U=Hˉ−1U = \bar{H}^{-1} and using standard SGD, the "hovering scale" becomes ρ/λˉ\rho/\sqrt{\bar{\lambda}}, compared to ρ\rho for standard SAM. The authors interpret this as a mechanism for escaping narrow, sharp minima ("potholes"). Experiments cover CIFAR-10/100, TinyImageNet, ImageNet, IWSLT14, various SAM variants, scenarios with label noise, and Vision Transformers (ViT). The paper reports consistent performance improvements over both standalone SAM and LLQR.

Critical strengths
  1. The problem is clearly and reasonably framed: can a single learned metric serve both the perturbation step and the descent step? The algorithm is concise.
  2. The scope of applicability is addressed transparently (Section 6, Appendix E). The scalar recurrence formulas (Appendix A.3–A.4) are clearly presented.
  3. The experiments include reports on computational overhead and memory usage.
  4. It demonstrates significant improvement over standard SAM on the CIFAR-100 dataset, covering a wide range of architectures.
Critical weaknesses

1.The novelty of the paper lies in the "geometry-aware SAM probe." However, factorial ablation analysis reveals that the probe contributes almost nothing. Therefore, the performance gains stem from the combination of LLQR preconditioning and SAM, rather than from the "coupling" emphasized in the title and abstract. 2.The paper does not report the actual Euclidean perturbation norms of SAM and LLQR+SAM, nor does it provide a SAM baseline using a larger or layer-by-layer parameter-tuned ρ. The gain may simply be due to the difference in effective radius. 3.The most closely related methods—Riemannian SAM, preSAM, and those proposed by Zhong et al. (2026)—are discussed but not empirically compared. Furthermore, there is a lack of baselines involving "SAM + other preconditioners" (e.g., using K-FAC, Shampoo, or SOAP as the outer preconditioner, combined with either Euclidean or Fisher-based probes). Without such comparisons, it is impossible to determine whether the effectiveness stems specifically from LLQR or if any high-quality preconditioner would yield similar results. FisherSAM appears only in Table 1, and no results are provided for it on ImageNet, even though the corresponding hyperparameters for ImageNet are listed in Appendix C.1. 4. On TinyImageNet, LLQR alone performs worse than SGDM (57.02 vs. 57.97), yet LLQR+SAM outperforms SAM by 1.07. This superadditive interaction was not predicted in Section 4, nor is it accounted for.

Questions for authors
  1. Please report the ‖ε‖₂ values ​​for SAM and LLQR+SAM during the training process (on CIFAR-100/ResNet-18). Additionally, provide results for SAM when its ρ parameter is adjusted to match the effective radius used in LLQR+SAM.
  2. Why does the "outer-layer-only" variant outperform the full method on TinyViT (Table 19)? In the ablation studies for TinyImageNet/CIFAR-100, are the gains of the full method over the "outer-layer-only" variant statistically significant (i.e., beyond the level of noise, as assessed by a paired test)?
  3. Could you include a baseline—such as "SAM + a non-LLQR preconditioner" (e.g., using K-FAC or Fisher-based preconditioning for the outer-layer step), preSAM, or Riemannian SAM—on at least one benchmark?
  4. Regarding the super-additive effect observed on TinyImageNet (where LLQR alone underperforms SGDM, yet LLQR+SAM outperforms SAM by 1.07), are there any explanations for this beyond the scaling of the radius?
2 comments

Thank you for the detailed questions. Confidence intervals below use matched seeds unless noted.

W2/Q1 — Is the gain simply a larger perturbation radius? We measured ∣ϵ∣2|\epsilon|_2, swept SAM's radius, and evaluated an LLQR+SAM probe with exactly the same Euclidean norm as SAM (Table B). Eq. (1) uses ϵ=ρUg/g⊤Ug\epsilon=\rho Ug/\sqrt{g^\top Ug}; replacing its denominator by ∣Ug∣2|Ug|_2 fixes ∣ϵ∣2=ρ|\epsilon|_2=\rho without changing the perturbation direction. At ρ=0.2\rho=0.2, this radius-matched LLQR+SAM reaches 81.80%, versus 81.04% for SAM (+0.77 points [0.46, 1.07]). No radius in the tested SAM sweep (0.05–0.70) closes this gap. The original probe has Euclidean norms of 0.19/0.17/0.17 at 5%/50%/100% of training and comparable accuracy. Thus the gain over SAM is not explained by overall perturbation length. The separate placement ablation, rather than radius matching, measures the contribution of the learned probe beyond outer preconditioning. We did not optimize per-layer radii; equal radii per parameter group gave 79.11 ± 0.34% at ρ=0.2\rho=0.2.

Table B. Probe radius, CIFAR-100/ResNet-18, five seeds. Best accuracy is the highest test accuracy during training.

Probe ϵ\epsilonρ‖ε‖₂ during trainingBest accuracy (%) ↑Difference to radius-matched LLQR+SAM (pp)
SAM, ρ g/∥g∥2\rho\,g/\lVert g\rVert_20.100.1080.89 ± 0.26−0.91 [−1.26, −0.57]
0.200.2081.04 ± 0.13−0.77 [−1.07, −0.46]
0.300.3081.00 ± 0.15−0.80 [−0.99, −0.61]
LLQR+SAM, radius-matched, ρ Ug/∥Ug∥2\rho\,Ug/\lVert Ug\rVert_20.200.2081.80 ± 0.13—
LLQR+SAM, Equation 1, ρ Ug/∥g∥U\rho\,Ug/\lVert g\rVert_U0.200.19 / 0.17 / 0.1781.82 ± 0.35+0.02 [−0.32, +0.35]

By construction, ∣ϵ∣2=ρ|\epsilon|_2=\rho at every step for SAM and radius-matched LLQR+SAM. SAM at ρ=0.05\rho=0.05 and 0.40–0.70 performed no better than 80.45%. The LLQR+SAM rows are reruns paired with this sweep and can differ slightly from the submission.

W1/Q2 — Why does outer-only outperform full on TinyViT? The placement experiments show that outer preconditioning supplies most of the improvement; the learned probe need not improve every setting (general reply). On TinyViT/AdamW, full trails outer-only by 1.85 points (submitted Table 19). AdamW's coordinatewise scaling may interact with the anisotropic perturbation, whose radius was not separately tuned, but we have not tested this explanation. We will report the result without claiming that shared geometry is always preferable.

W3/Q3 — Does the effect extend to other preconditioners? We implemented SAM with Fisher K-FAC following kfac-jax (Botev and Martens), updating Fisher factors every step and refreshing damped inverses every 500 steps. Its learning rate and damping were tuned jointly with SAM (0.08 and 1.0), while LLQR+SAM reused the independently tuned component settings. On CIFAR-100/ResNet-18, full Fisher K-FAC + SAM matches full LLQR+SAM in accuracy but takes roughly twice as long (Table C). In the K-FAC placement ablation, outer-only reaches 81.74 ± 0.12% and full 81.81 ± 0.18%, again indicating that the probe's incremental effect is small. These results support the value of structured preconditioning with SAM, while highlighting LLQR's lower measured cost in this setting.

Table C. K-FAC preconditioner, CIFAR-100/ResNet-18, ρ = 0.2, five seeds.

MethodBest accuracy (%) ↑Elapsed (h/run) ↓
SAM81.13 ± 0.210.94 ± 0.00
LLQR79.96 ± 0.220.90 ± 0.04
Fisher K-FAC79.89 ± 0.121.99 ± 0.03
LLQR+SAM (full)81.82 ± 0.141.16 ± 0.01
Fisher K-FAC + SAM (full)81.81 ± 0.182.46 ± 0.21

"Full" uses the metric in both steps. the probe and the outer step. Accuracies for SAM, LLQR, and LLQR+SAM are those of submitted Table 1; wall-clock times are from reruns of the same recipes on one GPU, including startup, compilation, and evaluation.

W4/Q4 — Why does LLQR underperform SGDM on TinyImageNet? We identified a protocol error: the submitted ResNet-18 and WRN-28-10 runs used SGD without momentum, although the intended tuned recipe specified momentum 0.9. We reran all four methods with the correct SGDM recipe (Table D). LLQR now exceeds SGDM (+0.39 and +0.24 points), and LLQR+SAM still exceeds SAM (+1.09 and +0.66 points). We will replace the affected table and discussion; the anomalous LLQR-versus-SGDM comparison no longer holds.

Table D. TinyImageNet-200 with momentum. Best top-1 test accuracy (%, mean ± std, 3 seeds).

ArchitectureSGDMLLQRSAMLLQR+SAM
ResNet-1866.77 ± 0.1367.16 ± 0.3168.47 ± 0.3569.56 ± 0.26
WRN-28-1070.55 ± 0.4070.79 ± 0.3472.56 ± 0.3873.22 ± 0.19

F-SAM naming. The ImageNet hyperparameters in Appendix C.1 concern Friendly-SAM (F-SAM), not FisherSAM. F-SAM achieves 77.92 ± 0.07% versus 77.96 ± 0.01% for SAM; LLQR+F-SAM achieves 78.61 ± 0.16%. We will clarify the distinction and add the omitted values to the appendix.

Reference: Botev and Martens, KFAC-JAX (2022) GitHub repository, https://github.com/google-deepmind/kfac-jax

Reviewer_b6WPOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes LLQR+SAM, which combines sharpness-aware minimization (SAM) with LLQR, a method that learns a structured, slowly varying inverse metric from layerwise network dynamics. The learned metric is used both to define the SAM perturbation and to precondition the gradient evaluated at the perturbed parameters. The authors motivate this design using a two-scale quadratic model, arguing that the learned geometry can enlarge the effective exploration scale in directions that are flat under the background curvature but locally sharp.

Overall, the work presents an interesting idea along with a nice theoretical analysis, and I believe this work advances our understanding.

Critical strengths

The paper proposes a natural combination of LLQR and SAM. Using the learned metric both to define the perturbation and to precondition the outer update is a reasonable idea, and the two-scale quadratic model provides an intuitive explanation for why the combination might be beneficial. The experiments cover a range of datasets and architectures, including ImageNet, and the authors make an effort to account for computational overhead. Overall, the results suggest that combining learned geometry with SAM can be useful in practice.

Critical weaknesses

The paper does not clearly establish how much of its contribution comes from the geometry-aware perturbation, as opposed to simply combining SAM with LLQR preconditioning. For example, the ablations on TinyImageNet suggest that outer preconditioning accounts for much of the improvement, while geometry-aware perturbations provide a smaller additional gain. The theoretical explanation is also based on an idealized quadratic model with an ideal metric, and it remains unclear how well this mechanism describes the behavior of the learned metric in actual neural-network training. In particular, geometry-aware and preconditioned SAM have been studied in prior work and the paper does not make the advantages of its particular LLQR-based construction sufficiently clear.

Questions for authors

The ablations suggest that much of the improvement comes from outer preconditioning, with a smaller contribution from changing the perturbation geometry. Can the authors provide an argument that the geometry-aware perturbation itself is beneficial, beyond the effect of preconditioning the outer update?

1 comment

Thank you for the constructive assessment.

Value of the learned perturbation. The new factorial ablations (general reply, Table A) show that outer preconditioning accounts for most of the gain over SAM; modifying the probe has a smaller, setting-dependent effect. This does not undermine the broader question addressed by the paper: whether a slowly learned geometry can complement SAM's response to localized sharpness. Section 4 proposes a possible mechanism, rather than predicting the ranking of the variants. We will make that distinction explicit.

What is specific to LLQR? The geometric interaction is not unique to LLQR. Its contribution is a structured, slowly learned inverse metric, updated periodically and smoothed over time, that can be reused for SAM at moderate computational cost. FisherSAM instead modifies the probe, while Riemannian SAM and preSAM provide more general geometric formulations. On CIFAR-100/ResNet-18, Fisher K-FAC + SAM matches full LLQR+SAM in accuracy (81.81% versus 81.82%) but takes 2.46 versus 1.16 hours per run in our implementation (Table C, reply to eVJg). We will sharpen this positioning in the related work and discussion.

Comments on the paper

3 comments
AuthorsOpenReview ↗

General Reply

We thank the reviewers for their careful and constructive feedback. We first address the concern shared by all three reviewers, then respond to their other questions individually.

Unless specified otherwise, results are mean ± sample standard deviation over five seeds; exceptions are marked by nn. Brackets denote 95% paired tt-confidence intervals for method differences over matched seeds.

What does combining learned geometry with SAM contribute?(cPgs, eVJg, b6WP)


We study how a slowly learned, coarse optimization geometry can complement SAM. Beyond the convergence benefits expected from preconditioning, our motivating hypothesis is that their combination can also improve the solutions reached: SAM probes local sharpness, while the learned metric accounts for broader curvature, potentially helping the optimizer escape narrow sharp wells (“potholes”) within otherwise flat valleys. Section 4 illustrates this mechanism in a two-scale model; it does not establish that it causes the gains observed in neural networks.

The expanded ablations clarify the roles of the two steps. We independently apply LLQR's metric to the SAM perturbation, the outer update, both, or neither (Table A). Outer-only consistently improves over SAM in the CIFAR-100 and TinyImageNet settings shown. Adding learned geometry to the perturbation brings smaller, variable changes: full matches outer-only on clean CIFAR-100 and modestly improves its mean on the label-noise and TinyImageNet settings, without a clear statistically established advantage. Results from other settings, including unfavorable TinyViT and new CIFAR-10-C comparisons, are addressed in the individual replies.

Table A. LLQR metric placement., ResNet-18 unless stated. Accuracy (%) ↑; bold is the largest reported mean in each row.

DatasetEvaluationSAMOuter onlyProbe onlyFull LLQR+SAM
CIFAR-100, ρ = 0.2Clean, best81.04 ± 0.1381.82 ± 0.1781.13 ± 0.1181.82 ± 0.14
20% label noise, best (n = 3)72.86 ± 0.0973.65 ± 0.1772.45 ± 0.2473.87 ± 0.18
40% label noise, best (n = 3)64.70 ± 0.5167.59 ± 0.2064.98 ± 0.5068.13 ± 0.44
60% label noise, best (n = 3)61.33 ± 0.4362.47 ± 0.6361.74 ± 0.6263.07 ± 0.43
CIFAR-100-C mean, final51.86 ± 0.1052.34 ± 0.0851.51 ± 0.1252.33 ± 0.19
TinyImageNet/ResNet-50, ρ = 0.1Clean, best71.63 ± 0.5972.79 ± 0.2071.80 ± 0.3373.14 ± 0.26
TinyImageNet/ResNet-18, ρ = 0.1Clean, best (n = 3)68.47 ± 0.3569.05 ± 0.1968.70 ± 0.4269.56 ± 0.26

Outer only applies LLQR to the descent step, probe only to the perturbation, and full to both. Under label noise rate ff, each training label is redrawn uniformly over all classes with probability ff; test labels remain clean. CIFAR-100 results are new runs (hence small differences from submitted Table 16, differences with Table 4 come from maintaining data augmentation in label-noise experiments for a uniform setup across experiments Table A); TinyImageNet/ResNet-50 uses submitted Table 6. TinyImageNet/ResNet-18 uses the corrected momentum recipe detailed in the reply to eVJg.

The central interaction does not require a shared probe and descent metric. In the two-scale model, outer-only already combines a background-preconditioned update with SAM's response to local sharpness. The geometry-aware probe further changes exploration anisotropically: along a single dominant direction ii, its scale becomes ρ/λˉi\rho/\sqrt{\bar\lambda_i} instead of ρ\rho. The preferred placement may depend on the loss geometry; our analysis does not predict which performs best in a given experiment. We will add the outer-only calculation and revise the abstract and discussion to distinguish the benefits of combining geometry with SAM from the additional effect of coupling both steps.

AuthorsOpenReview ↗

Revised manuscript

We updated our manuscript, with edits marked in red within the text. A complete changelog is provided in the comment below.

With the rebuttal period drawing to a close, we remain eager to engage in further discussion if any concerns remain.

Main changes in the revised manuscript

  • TinyImageNet ResNet-18 and WRN-28-10: the submitted runs used SGD without momentum instead of the intended SGDM recipe. All four methods were rerun with SGDM (Table 2, Section 5.1); Appendix C.1 states the recipe, and Appendix C.2 the chunked metric updates for WRN-28-10. No-momentum results without a rerun are removed: the TinyImageNet row of Table 7 and its mention in Section 5.4, the TinyImageNet rows of Table 9, and submitted Table 12 with its sentence in Appendix D.2. Submitted Tables 13–19 are now Tables 12–18.
  • Two-scale model: outer-only calculation added (Section 4, "Comparison with SAM"; new Appendix A.7). Section 4 now presents the model as a possible mechanism that does not predict which placement performs best.
  • Placement of the learned metric: new Appendix F.1 (Table 20). Section 5.3 rewritten; full coupling is no longer presented as necessary for the gain. Appendix D.4 relates the new runs to Table 15.
  • Abstract, introduction and conclusion: the benefit of combining learned geometry with SAM is distinguished from the effect of coupling both steps.
  • Probe radius: new Appendix F.2 (Table 21).
  • Positioning against other geometry-aware SAM methods: Related Work and Section 6; Fisher K-FAC and FisherSAM in new Appendix F.3 (Table 22).
  • Distribution shift: new Appendix F.4 (Tables 23–24); the limitation is stated in Section 6.

Minor

  • ImageNet/ResNet-50: the SAM and LLQR+SAM uncertainties are now sample SDs over three seeds, instead of standard errors; means unchanged (Table 2, Section 5.1, Tables 7 and 11).
  • Figure 2: larger text. Same layout and data.
  • Figure 4: redrawn at full text width with legible labels. Same points and fitted slopes.
  • Figures 5 and 6 (Appendix A): larger text, with one shared legend below the panels. Same layout and data.
  • F-SAM naming (Appendix C.1): the ImageNet protocol's F-SAM is Friendly-SAM, not FisherSAM. ImageNet F-SAM and LLQR+F-SAM results added in new Appendix F.5 (Table 25).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.