acceptodds
Under review as a conference paper at ICLR 2027

Chameleon: Learning Anisotropic Perturbation Geometry in Sharpness-Aware Optimization

14 views

Abstract

Sharpness-Aware Minimization (SAM) improves generalization by optimizing the worst-case loss within a neighborhood of model parameters, thereby biasing training toward flatter minima. Despite its effectiveness, standard SAM uses a fixed isotropic perturbation region, which can be mismatched with the highly anisotropic curvature of modern loss landscapes. This geometric mismatch becomes particularly limiting under distribution shifts, where sensitive directions may drift over time and a spherical neighborhood can either over-regularize flat directions or under-explore sharp ones. We propose Chameleon, a lightweight sharpness-aware optimizer that learns an anisotropic perturbation geometry. Chameleon replaces SAM’s fixed-radius ball with a learnable ellipsoidal neighborhood by tensorizing perturbation scales into adaptive parameters. These scales are optimized through a normalized sharpness surrogate that preserves first-order efficiency while encouraging perturbations to align with curvature-sensitive directions. Theoretically, we characterize the learned geometry as a curvature-sensitive axis-aligned metric that coordinate-wise reweights dominant spectral components, establish sublinear convergence under clipped adaptive scales, and derive containment-based PAC-Bayes generalization bounds complemented by a matched-budget curvature analysis. Extensive experiments evaluating multi-modal capabilities across vision, language, and industrial click-through rate (CTR) prediction tasks demonstrate consistent improvements in cross-domain adaptation. Specifically, Chameleon achieves up to 3.91% cross-domain and 2.09% in-domain accuracy gains on vision tasks, 4.19% F1 improvement on language tasks, and 0.71% AUC gains on CTR tasks.

Primary area:
optimization
Keywords:
sharpness-aware minimization, optimization, generalization, flat minima, PAC-Bayes, distribution shift

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8wDVOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Chameleon, a sharpness-aware optimizer that improves generalization by learning anisotropic perturbation geometry. Unlike standard SAM, which employs a fixed spherical perturbation neighborhood, Chameleon introduces a learnable ellipsoidal neighborhood with coordinate-wise adaptive perturbation scales. These scales are optimized through a normalized sharpness surrogate that implicitly captures curvature information without explicit second-order computations. The authors provide theoretical analyses of the learned perturbation geometry, convergence properties, and PAC-Bayes generalization bounds, along with a matched-budget curvature analysis. Extensive experiments validate the effectiveness of the proposed method.

Critical strengths
  • The paper clearly identifies a key limitation of standard SAM: its uniform perturbation radius across parameter directions fails to account for the anisotropic curvature of the loss landscape. Motivated by this observation, the authors introduce a learnable anisotropic perturbation geometry, providing a natural and well-justified extension of SAM. The overall motivation is clear, and the proposed design aligns well with the identified problem.
  • The experimental evaluation is comprehensive, covering a wide range of tasks, including image classification, domain generalization, natural language understanding, and click-through rate prediction, across diverse datasets and model architectures. The consistent improvements over existing baselines demonstrate the effectiveness and generalizability of the proposed method across different learning scenarios.
Critical weaknesses
  • The derivation of step (i) in Appendix C.5 is insufficiently justified. The approximation 1/ρt≈∥∇L1∥/(L1−L0)1/\rho_t \approx \Vert\nabla L_1\Vert/(L_1-L_0) equates a vector-valued quantity with a scalar, without explaining the necessary assumptions. Furthermore, even accepting this approximation, substituting it into (h) appears to yield a denominator of (L1−L0)∥∇L1∥(L_1-L_0)\Vert\nabla L_1\Vert, rather than (L1−L0)∥∇L1∥2(L_1-L_0)\Vert\nabla L_1\Vert^2 as written in (i). These inconsistencies undermine the justification of the proposed scale update.

  • The experimental comparisons are insufficient. While several SAM variants are evaluated on vision tasks, experiments on language understanding and CTR prediction only compare against vanilla SAM and ERM. Moreover, most baselines are relatively outdated. Including more recent SAM variants, such as XSAM [1], RSAM [2], and TSAM [3], would help better establish the competitiveness of the proposed method.

[1] Revisiting Sharpness-Aware Minimization: A More Faithful and Effective Implementation, ICLR 2026.

[2] Rényi Sharpness: A Novel Sharpness That Strongly Correlates with Generalization, ICLR 2026.

[3] Tilted Sharpness-Aware Minimization, ICML 2025.

Questions for authors
  • Could the authors provide a more rigorous derivation of Equation (17), particularly the transitions from (h) to (i) and from (i) to (j)? What assumptions justify these approximations, and how is the discrepancy in the gradient norm factor explained?
  • Could the authors provide comparisons with more recent SAM variants, such as XSAM, RSAM, and TSAM, particularly on language understanding, CTR prediction, and domain generalization benchmarks? These comparisons would help establish the competitiveness of Chameleon beyond standard SAM.
3 comments
AuthorsOpenReview ↗

Rigorous Scale-Gradient Derivation and Comparisons with Recent SAM Variants

We thank the reviewer for the constructive feedback and address both concerns.

W1&Q1: Derivation of Eq. (17) and scale updates.

We agree that steps (i)–(j) in Appendix C.5 are not rigorously justified. Let v=g0/∥g0∥2v=g_0/\|g_0\|_2, w1=w+ρ⊙vw_1=w+\rho\odot v, g1=∇L(w1)g_1=\nabla L(w_1), H1=∇2L(w1)H_1=\nabla^2L(w_1), s=∥g1∥2>0s=\|g_1\|_2>0, and ΔL=L1−L0\Delta L=L_1-L_0. Holding w,g0w,g_0 fixed, the exact gradient of J=L1/sJ=L_1/s is

∇ρJ=v⊙[g1s−L1H1g1s3].\nabla_\rho J=v\odot\left[\frac{g_1}{s}-\frac{L_1H_1g_1}{s^3}\right].

Define the displacement and direction:

δ=ρ⊙v,r=∥δ∥2,u=δ/r.\delta=\rho\odot v,\quad r=\|\delta\|_2,\quad u=\delta/r.

For a Lipschitz Hessian and small rr,

g1−g0r=H1u+O(r).\frac{g_1-g_0}{r}=H_1u+O(r).

Replacing H1(g1/s)H_1(g_1/s) also requires g1/s≈ug_1/s\approx u, beyond smoothness. Taylor expansion gives

ΔLr=g0⊤u+O(r).\frac{\Delta L}{r}=g_0^\top u+O(r).

Thus, 1/r≈s/ΔL1/r\approx s/\Delta L additionally requires g0⊤u≈sg_0^\top u\approx s, i.e., alignment between the displacement and gradient directions. Coordinate-wise scaling does not generally preserve this alignment. Consequently, the transition from (h) to (i) is conditional, not a general identity. We also agree that the subsequent transition to (j) introduces an inconsistent gradient-norm factor that cannot be absorbed into the learning rate.

For completeness, centering the numerator defines a different objective JΔ=ΔL/sJ_\Delta=\Delta L/s, whose exact gradient is

∇ρJΔ=v⊙[g1s−ΔLH1g1s3].\nabla_\rho J_\Delta=v\odot\left[\frac{g_1}{s}-\frac{\Delta L H_1g_1}{s^3}\right].

This alternative does not algebraically recover Eq. (6b). We will correct Appendix C.5, remove the unsupported equivalence, and explicitly characterize Eq. (6b) as a first-order-computable heuristic rather than the exact gradient of JJ.

We further compare the update with exact HVP-based gradients. The cosine similarity reaches 0.93. On CIFAR-100/CutMix and RTE, the HVP variant achieves 83.04%/81.68%, versus 82.98%/81.59% for Chameleon and 82.49%/80.51% for the damped uncentered variant. The corresponding relative FLOPs are 1.84× for HVP and 1.09× for Chameleon. We will report relative gradient errors and objective changes alongside cosine similarity to characterize the approximation more completely.

W2&Q2: Comparisons with recent SAM variants.

We agree that comparisons beyond vanilla SAM are important. We therefore evaluate three recent methods using matched data splits, training steps, model selection, and hyperparameter search-trial counts, following their official implementations. Specifically, XSAM [1] estimates the direction toward the neighborhood maximum; RSAM [2] regularizes Rényi sharpness; TSAM [3] uses exponential tilting to reweight local losses. In contrast, Chameleon learns parameter-wise, axis-aligned perturbation scales.

Table 1. GLUE (RoBERTa-base), CTR (DeepFM), and relative runtime (SAM=1).

MethodCoLAMRPCSTS-BRTEAvazuCriteoTaobaoCost
TSAM63.1588.36/91.5689.41/89.2880.87/78.2437.620/78.68744.343/80.79119.310/63.7752.45
RSAM63.2988.42/91.6189.58/89.4781.05/78.4137.623/78.68544.348/80.78819.314/63.7581.38
XSAM63.5488.55/91.6889.84/89.7281.23/78.6537.618/78.68944.338/80.79519.306/63.7961.26
Chameleon64.0388.73/91.7990.29/90.1481.59/79.0137.614/78.69244.329/80.80319.297/63.8411.08

CoLA: MCC; MRPC/RTE: accuracy/F1; STS-B: Pearson/Spearman; CTR: LogLoss/AUC (×100). Lower LogLoss and higher AUC are better.

Table 2. PACS and DomainNet accuracy (%).

MethodPCPhSClInPaQuReSk
TSAM87.9673.8498.6464.4880.6145.3881.8341.8881.8972.78
RSAM88.0173.9298.6864.5580.6845.4281.8841.9381.9472.86
XSAM88.0974.1598.7564.7180.7645.4981.9541.9982.0172.98
Chameleon88.2774.7698.9265.0980.9245.6382.0742.1182.1473.27

PACS: Painting/Cartoon/Photo/Sketch. DomainNet: Clipart/Infograph/Painting/Quickdraw/Real/Sketch. ERM and SAM results are also provided in the manuscript's Tables 2–4.

Chameleon achieves the best reported results across all listed benchmarks. Relative to the strongest recent baseline, gains range from 0.11–0.49 points on GLUE and 0.06–0.61 points on PACS/DomainNet. CTR improvements are smaller but consistent in both LogLoss and AUC. Chameleon also incurs the lowest reported cost (1.08×), compared with 1.26× for XSAM, 1.38× for RSAM, and 2.45× for TSAM. We will clarify the evaluation protocols, computational-cost measurement, and seed-level variability. These results establish competitiveness under the tested settings without implying universal dominance across all architectures.

I appreciate the authors' thorough response and efforts to address my concerns. Their clarifications have resolved most of my questions, and I will therefore retain my positive assessment of the paper.

We sincerely thank you for the positive assessment and valuable suggestions, which have helped us improve our work. We greatly appreciate your continued support.

Reviewer_EQHWOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

This paper proposes Chameleon, an extension of Sharpness-Aware Minimization (SAM) with learnable perturbation scales. The authors replace the fixed isotropic perturbation region with an adaptive ellipsoidal neighborhood. They use a normalized sharpness objective to update the perturbation scales during training. The paper also presents theoretical results on curvature, convergence, and generalization. Numerical results show that Chameleon improves predictive performance over SAM and several related methods.

Critical strengths

The idea of learning different perturbation scales for different parameters is interesting. Standard SAM uses a fixed perturbation radius, which may not reflect differences in parameter sensitivity. The proposed method provides a way to adjust these scales during training. I also appreciate that the method does not require explicit Hessian computation.

The numerical experiments cover several models. Tables 1-4 (pp. 7-8) report improvements in image classification, language understanding, cross-domain transfer, and CTR prediction. The authors report results over five independent runs. The experiments with different batch sizes and label-noise levels are also useful. Table 7 (p. 9) shows smaller Hessian eigenvalues and traces for Chameleon than for SAM. These results provide evidence that adaptive perturbation scales may improve training performance.

Critical weaknesses

I have two main concerns.

The first concern is about the theoretical formulation. Definition 3.1 (p. 5) minimizes the objective over both model parameters and perturbation scales. However, the inner objective involves maximization over the perturbation region. Enlarging this region cannot reduce the maximum perturbed loss. Thus, the formulation appears to favor smaller perturbation scales, at least for this part of the objective. In contrast, Algorithm 1 (p. 20) updates the scales by gradient ascent. I am not sure how these two formulations are consistent. I also have concerns about Proposition 3.2 (p. 5). In Appendix E (p. 21), the matrix used in the quadratic expansion comes from the Hessian of the log loss. However, Eq. (4) defines a different normalized objective. It is unclear how the stated quadratic expansion follows from this objective.

My second concern is about the generalization analysis. Theorem 3.4 (p. 6) provides a PAC-Bayes generalization bound. However, I am not convinced by the proof in Appendix G (p. 24). Assumption 5 bounds the original population loss by its Gaussian-smoothed version. This does not appear sufficient to justify the upper bound on the smoothed loss in Eq. (49). The matched-budget analysis on p. 25 also assumes a positive diagonal Hessian and optimally allocated perturbation scales. It is unclear whether the learned scales satisfy this condition. The numerical results are good, but I would like more evidence for the claimed improvement under changing distributions. In particular, Tables 16 and 17 (pp. 30-31) compare standard SAM with a multi-step reference. They do not directly report the performance of Chameleon in these settings.

Questions for authors

My questions are based on my concerns regarding the critical weaknesses.

(1) In Definition 3.1 (p. 5), the optimization is formulated as minimization over both model parameters and perturbation scales. However, Algorithm 1 (p. 20) uses gradient ascent for the scales. How should these two formulations be reconciled? I would also like to understand how the quadratic expansion in Proposition 3.2 follows from the normalized objective in Eq. (4).

(2) Can the authors clarify the proof of Theorem 3.4? In Appendix G (p. 24), it is unclear how Eq. (49) follows from Assumption 5 and the preceding PAC-Bayes inequality. I would also like to know whether the learned perturbation scales approach the curvature-dependent allocation used in the matched-budget analysis on p. 25.

(3) The experiments report improvements across several datasets. However, the paper emphasizes adaptation under changing distributions. In Appendix K.4 (pp. 30-31), the results mainly compare SAM with a multi-step reference. Can the authors report direct comparisons involving Chameleon in these experiments? It would also help to compare the learned scales with fixed anisotropic or adaptive isotropic scales under the same training budget.

1 comment
AuthorsOpenReview ↗

Clarifying the Optimization Formulation, PAC-Bayes Analysis, and Empirical Evidence

We thank the reviewer for the helpful input.

W1 & Q1: Optimization formulation and Proposition 3.2.

(1) Definition 3.1 vs. Algorithm 1. We agree Eq. (5) minimizes over both ww and ρ\rho, whereas Algorithm 1 updates ρ\rho by ascent. Algorithm 1 instead uses coupled updates: ρt+1=ΠR(ρt+αρqt),wt+1=wt−ηtdt,\rho_{t+1}=\Pi_{\mathcal R}(\rho_t+\alpha_\rho q_t),\quad w_{t+1}=w_t-\eta_t d_t, where R=[ρmin⁡,ρmax⁡]d\mathcal R=[\rho_{\min},\rho_{\max}]^d, and qtq_t, dtd_t denote the scale/model directions in Eqs. (6b)/(6a). We will revise Definition 3.1 to state that Algorithm 1 is an approximate alternating procedure, with qtq_t a finite-difference rather than exact-gradient direction.

(2) Proposition 3.2. We agree that H/f−gg⊤/f2H/f-gg^\top/f^2 is the Hessian of log⁡f\log f, not generally of J=f/∥g∥2J=f/\|g\|_2. For strictly positive, sufficiently smooth ff, a valid auxiliary proxy is

=\tfrac12\epsilon^\top\nabla^2\log f(w)\epsilon+o(\|\epsilon\|^2).$$ This proxy is distinct from the scale-learning objective. Proportionality between $\|g\|$ and $f$ does not turn their ratio into $\log f$, nor does $|g\|\to0$ justify dropping terms containing inverse powers of $\|g\|$. Existing SAM analyses do not establish this equivalence. We will withdraw the unsupported derivation from Eq. (4), and clarify that the log-loss expansion characterizes an auxiliary symmetric curvature proxy, not an exact Hessian expansion of $J$. **W2 & Q2: PAC-Bayes proof and matched-budget analysis.** **(1) Theorem 3.4.** Eq. (49) does not follow directly from Eq. (48) and Assumption 5. Instead, define $\widetilde Q=Q*\mathcal N(0,b^2I)$ and apply PAC-Bayes to it: $$\mathbb E_{\widetilde Q}R\leq\mathbb E_{\widetilde Q}\widehat R+C,\quad C=4\sqrt{\frac{\mathrm{KL}(\widetilde Q\|P)+\log(2m/\alpha)}{m}},$$ with probability at least $1-\alpha$, for a data-independent prior $P$ and loss in $[0,1]$. Smoothing dominance and an ellipsoid-tail decomposition then yield $$\mathbb E_QR\leq\mathbb E_Q[\widehat R+\widehat h+p_{\rm out}]+C,$$ where $\widehat h=\max_{\delta\in\mathcal E(\rho(w))}[\widehat R(w+\delta)-\widehat R(w)]) and (p_{\rm out}(w)=\Pr_{\delta\sim\mathcal N(0,b^2I)}[\delta\notin\mathcal E(\rho(w))]$. Control this tail conditionally on the learned ellipsoid. For deterministic $w_T$, take $Q=\delta_{w_T}$, but retain $\mathrm{KL}(\widetilde Q\|P)$. The bounded-loss condition must be explicit; unbounded cross-entropy needs clipping or a tail assumption. We will revise the proof accordingly. **(2) Matched-budget analysis.** For $H=\operatorname{diag}(h_i)\succ0$ and $\prod_i\rho_i=r^d$, the quadratic oracle minimizes $$M_{\rm ani}=\frac{1}{2}\max_i h_i\rho_i^2$$ and gives the optimal scales $$\rho_i^*=r\left(\prod_{j=1}^d h_j\right)^{1/(2d)}h_i^{-1/2}.$$ The resulting anisotropic penalty satisfies $$M_{\rm ani}^*=\frac{r^2}{2}\left(\prod_{i=1}^d h_i\right)^{1/d} \leq M_{\rm iso}=\frac{r^2}{2}\max_i h_i.$$ This proves an oracle advantage for an idealized positive diagonal quadratic model, not convergence of Eq. (6b) to $\rho^*$. In our measurements, 94.8% of sampled Hessian diagonal entries are positive. We use $|H_{ii}|+\epsilon_0$ as a positive diagonal proxy, not as evidence that the full Hessian is positive definite. Under this proxy, penalty variance decreases by 68.4%, and the measured normalized penalty ratio is $0.61\pm0.04$, versus the oracle 0.54. We will define these metrics and avoid implying a learning guarantee. **Q3: Direct comparisons under distribution shifts.** Appendix K.4 lacked direct Chameleon comparisons, we thus added fixed-anisotropic and dynamic-isotropic controls. **Table 1. Final non-stationary performance.** | Method | TTA $E$/Acc. | CL $E$/Acc. | Forget. | |---|---:|---:|---:| | SAM | 0.083/78.2 | 0.129/72.9 | 8.7 | | Fixed anisotropic | 0.076/78.6 | 0.118/73.2 | 8.4 | | Dynamic isotropic | 0.064/79.3 | 0.095/73.8 | 7.9 | | Chameleon | 0.047/80.5 | 0.069/74.5 | 7.2 | | 10-step reference | 0.043/80.8 | 0.064/74.7 | 7.1 | TTA: CIFAR-10-C, step 15; CL: 10-split CIFAR-100, task 10. $E$ is the one-step versus 10-step projected-ascent loss gap on the same trained trajectory, probe batch, and feasible set, not the reference solver's residual. The 10-step result is diagnostic only. **Table 2. Stationary accuracy (%).** | Method | CIFAR-100/Cutout | CIFAR-100/CutMix | PACS | |---|---:|---:|---:| | SAM | 81.21±0.24 | 82.15±0.04 | 78.02±0.45 | | Fixed anisotropic | 81.29±0.21 | 82.22±0.08 | 78.10±0.48 | | Dynamic isotropic | 81.48±0.18 | 82.51±0.07 | 78.45±0.39 | | Chameleon | 81.86±0.19 | 82.98±0.07 | 79.26±0.34 | Fixed anisotropic holds coordinate scales fixed; dynamic isotropic adapts one radius; Chameleon adapts coordinate scales. We will specify tuning and report shape-only and radius-dynamics controls with matched budgets. One-step methods match architectures, data exposure, schedules, and training budgets; runtime is separate. TTA labels are evaluation-only.
Reviewer_6zw1OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Chameleon, a sharpness-aware optimization method that learns anisotropic perturbation scales instead of using SAM's fixed isotropic neighborhood. The method dynamically adjusts parameter-wise perturbation scales through a normalized sharpness surrogate to better capture local curvature information. The authors provide convergence and PAC-Bayes generalization analyses and evaluate Chameleon on vision, language, and CTR prediction tasks, showing improvements in generalization and robustness over several SAM baselines.

Critical strengths

The paper addresses the limitations of fixed isotropic perturbations in SAM and proposes a learnable anisotropic geometry to adapt to different parameter sensitivities.

Chameleon uses first-order gradient information to update perturbation scales without explicitly computing the Hessian, making it relatively efficient.

The paper provides theoretical analyses of convergence, curvature-related geometry, and generalization under stated assumptions.

The experiments cover diverse tasks and datasets, including ImageNet, CIFAR, GLUE, PACS, DomainNet, and CTR prediction. The reported results show improvements over several SAM and curvature-aware baselines, with additional robustness and flatness evaluations.

Critical weaknesses
  1. Chameleon constructs a dynamic anisotropic ellipsoid using learnable scales, but it has not fully demonstrated the unique advantages of this design compared to existing scaling mechanisms. It is recommended to compare fixed anisotropic scaling, layerwise scaling, and learned anisotropic scaling, and clarify the additional value of dynamic updates. Besides the methods provided in the main text, ASAM (ICML 2021), P² (2024), Fisher SAM (ICML 2022) hope the authors can have additional discussions and comparisons with the mechanisms and effects of article [a].

  2. Chameleon is closely related to Curvature-aware SAM, such as CR-SAM (AAAI 2024), Eigen-SAM (NeurIPS 2024), and the recent one [b]. Chameleon also aims to improve the first-order curvature approximation of SAM, by learning perturbation scales through normalized surrogate. Although the specific optimization formulations of these methods are different, the issues they focus on have obvious connections. It is recommended to clearly compare the differences in curvature modeling, perturbation construction, and optimization objectives, and further compare performance and computational costs, especially with the direct comparison of [b].

  3. Chameleon uses the normalized surrogate in Eq. (4) and the approximate gradient in Eq. (6b) to update ρ\rho, and Proposition 3.2 links the surrogate with the maximum eigenvalue of the transformed curvature matrix. However, this theoretical connection cannot directly prove that the learned scales of Chameleon align with the Hessian curvature, nor can it guarantee that Eq. (6b) accurately approximates the ideal curvature-aware perturbation. It is recommended to directly analyze the relationship between learned ρi\rho_i and local curvature information, and compare it with high-precision curvature-aware perturbations to verify its approximation accuracy and actual mechanism.

  4. The existing ablation experiments need to be increased. The current content is not sufficient to distinguish the contributions of each core design of Chameleon. Chameleon also incorporates anisotropic scaling, dynamic radius adjustment, and normalized curvature surrogate. However, the existing experiments do not distinguish the individual roles of these mechanisms. It would be better to add fixed anisotropic perturbation, dynamic isotropic radius, alternative scale-learning strategies, and remove the comparison of Eq. (6b) with gradient-difference correction. Different methods should adopt the same perturbation budget and training budget to confirm whether the performance improvement is indeed due to the proposed curvature-guided geometry learning.

  5. The performance improvement of the method has certain value, but its actual efficiency advantage still requires a more complete evaluation. In the ViT-B/32 experiment on ImageNet-1K, Chameleon only improves by 0.18 percentage points compared to CR-SAM, and the advantage on ImageNet-C/R is also smaller. Although the paper reports relative FLOPs and runtime, Chameleon requires additional search for ρmin⁡\rho_{\min}, ρmax⁡\rho_{\max}, ρinit\rho_{\mathrm{init}}, and αρ\alpha_\rho.

  6. Theoretical analysis should ideally establish a correspondence with the actual scale-learning algorithm. The existing content does not directly prove that the scale update in Eq. (6b) can converge to the corresponding curvature-aware geometry. Additionally, Eq. (5) defines ww and ρ\rho as joint minimization variables, but the actual algorithm seems to use gradient ascent for ρ\rho and gradient descent for ww. It would be best to clarify the relationship between the theoretical objective and the actual optimization process, and prove whether the proposed update can optimize the corresponding objective or approach the ideal geometry analyzed.

[a] Sharpness-Aware Minimization Revisited: Weighted Sharpness as a Regularization Term. KDD, 2023.

[b] Fix the Loss, Not the Radius: Rethinking the Adversarial Perturbation of Sharpness-Aware Minimization. ICML, 2026.

Questions for authors

Please check the weakness

1 comment
AuthorsOpenReview ↗

Mechanism Differentiation, Curvature Evidence, and Theoretical Scope

We thank the reviewer and address all six concerns.

W1&W4: Novelty and ablations. We agree anisotropy alone is not novel. ASAM uses parameter-magnitude scaling; Fisher-SAM uses a Fisher-based metric; μ\muP$^2$ uses width-dependent layerwise scaling; WSAM [a] reweights sharpness. Chameleon instead learns time-varying coordinate-wise scales through a normalized curvature surrogate and gradient-difference updates, without explicit Hessian computation. Its contribution is fine-grained adaptive learning, not anisotropy per se.

We isolate its components using fixed anisotropic scaling (no learning), layerwise scaling (coarser granularity), dynamic isotropic scaling (one radius), Frozen@50 (early-stopped adaptation), unnormalized ascent (normalization), and no gradient difference (secant correction). Chameleon outperforms all controls. Gains over LE-SAM [b], no-gradient-difference, and dynamic-isotropic controls are respectively 0.28/0.39/0.62, 0.35/0.43/0.66, and 0.38/0.47/0.81 points on Cutout/CutMix/PACS. Shape-only controls use a common radius schedule; radius-dynamics controls match cumulative squared perturbation norms. Native and normalized comparisons are separate.

Table 1. CIFAR-100/ResNet-18 and PACS accuracy (%).

MethodCutoutCutMixPACS
SAM81.21±0.2482.15±0.0478.02±0.45
WSAM81.46±0.1882.44±0.0778.31±0.41
LE-SAM81.58±0.1682.59±0.0878.64±0.36
Fixed anisotropic81.29±0.2182.22±0.0878.10±0.48
Layerwise81.41±0.1682.39±0.0678.38±0.37
Dynamic isotropic81.48±0.1882.51±0.0778.45±0.39
Frozen@5081.49±0.1882.53±0.0878.42±0.38
Unnormalized ascent81.34±0.2582.30±0.1178.19±0.44
No gradient difference81.51±0.2082.55±0.0878.60±0.36
Chameleon81.86±0.1982.98±0.0779.26±0.34

W2: Related curvature-aware methods. CR-SAM regularizes a finite-difference proxy of normalized Hessian trace but retains an isotropic neighborhood; Eigen-SAM estimates the leading Hessian eigenvector by power iteration; LE-SAM [b] derives a scalar radius from a loss-space budget. Chameleon learns a coordinate-wise ellipsoid, rather than estimating a trace/eigenvector or adapting one radius. As reported in Appendix K.1, Table 11, using ViT-B/32 trained from scratch under the same ImageNet-1K protocol, Chameleon achieves 75.46±0.09%, 48.37±0.13%, and 25.26±0.10% on ImageNet-1K, ImageNet-C, and ImageNet-R, respectively, outperforming CR-SAM (75.28±0.12%, 48.12±0.17%, 25.04±0.14%) and Eigen-SAM (74.05±0.10%, 47.42±0.15%, 24.66±0.12%).We will further clarify these differences.

W3: Curvature relationship. With a Lipschitz Hessian,

g1−g0=H(w)ϵ+O(∥ϵ∥22),g_1-g_0=H(w)\epsilon+O(\|\epsilon\|_2^2),

providing a directional secant signal, not the full Hessian. We compare Eq. (6b) with the exact HVP-based scale gradient and multi-step projected ascent. Table 2 reports cgc_g (cosine with exact scale gradient), rHr_H (Spearman correlation of scales with ∣diag⁡(H)∣|\operatorname{diag}(H)|), cϵc_\epsilon (direction cosine), and q2=s2/s2∗q_2=s_2/s_2^* (quadratic component relative to multi-step ascent, not a certified optimum). Positive rHr_H indicates association with curvature magnitude, not inverse-curvature optimality; we will report inverse-proxy correlation too.

Table 2. CIFAR-100/ResNet-18 diagnostics.

Methodcgc_grHr_Hcϵc_\epsilonq2q_2Acc.FLOPs/SAM
Eq. (6b).942±.011.846±.021.873±.018.864±.02581.861.09
Exact HVP1.000.861±.019.889±.015.881±.02281.911.94

W5: Efficiency and tuning. On ViT-B/32, gains over CR-SAM are modest (+0.18/+0.25/+0.22 points on ImageNet-1K/C/R). Runtime is 1.09× SAM for Chameleon vs. 1.50× for CR-SAM (35.3 vs. 48.6 GPU-hours; 27.3% less). We will separate FLOPs and runtime, and report trial counts/GPU-hours for tuning ρmin⁡,ρmax⁡,ρinit,αρ\rho_{\min},\rho_{\max},\rho_{\rm init},\alpha_\rho; per-run training cost excludes search overhead.

W6: Objective and scale updates. Eq. (5)'s joint minimization over w,ρw,\rho conflicts with Algorithm 1's descent in ww and ascent in ρ\rho; the algorithm is an approximate alternating procedure, not an exact solver of Eq. (5). We will revise the formulation and clarify the implemented updates:

w_{t+1}=w_t-\eta_t d_t.$$ For $J(\rho)=L_1/\|\nabla L_1\|_2$, where $L_1=L(w+\rho\odot v)$, fixing $w,g_0$, let $v=g_0/\|g_0\|_2$, $g_1=\nabla L_1$, $s=\|g_1\|_2$, and $H_1=\nabla^2L(w+\rho\odot v)$. The exact gradient is $$\nabla_\rho J=v\odot\left[\frac{g_1}{s}-\frac{L_1H_1g_1}{s^3}\right].$$ Eq. (6b) uses $g_1-g_0\approx H(w)\epsilon$ as a secant signal, not generally an approximation to $H_1g_1$. Thus, it is a heuristic scale direction, not the exact gradient. We will restrict convergence claims to model stationarity under bounded scales, not scale convergence to the oracle.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.