- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper proposes a zeroth-order optimization framework for LLM fine-tuning that incorporates Langevin dynamics to encourage global exploration and convergence toward flat minima. The authors further introduce a layerwise normalized variant and establish non-asymptotic Wasserstein-1 convergence guarantees. Experiments across several language models and downstream tasks demonstrate competitive accuracy and computational efficiency compared with existing ZO methods.
Critical strengths
- Improving the effectiveness and efficiency of zeroth-order LLM fine-tuning is an important research problem. Exploring optimization strategies beyond conventional ZO-SGD is potentially valuable.
- The paper provides extensive theoretical analysis, including non-asymptotic Wasserstein-1 bounds for ZOLD with respect to a flatness-biased Gibbs measure.
- The experiments cover multiple model families, including OPT models up to 66B parameters, and a variety of downstream tasks. The proposed methods achieve favorable accuracy-efficiency trade-offs compared with several existing ZO baselines, particularly HiZOO and FZOO.
Critical weaknesses
-
The core motivation for introducing Langevin noise is not sufficiently justified. My primary concern is the motivation behind adding Langevin noise to zeroth-order optimization. ZO methods inherently suffer from highly noisy gradient estimates, especially in high-dimensional settings such as LLM fine-tuning. The variance of standard ZO gradient estimators can grow with the parameter dimension, making the optimization process already highly stochastic. Given this inherent stochasticity, it is unclear why introducing additional Gaussian noise is necessary or beneficial. Moreover, the paper argues that conventional ZO-SGD methods lack a mechanism for global exploration. However, this characterization overlooks the substantial stochasticity already present in ZO updates.
-
The algorithmic novelty is limited. The proposed ZOLD algorithm appears to be a straightforward application of Langevin dynamics to standard ZO optimization, essentially replacing the stochastic gradient in SGLD with a conventional two-point ZO gradient estimator. Zeroth-order Langevin methods have already been investigated in previous literature [1,2]. Furthermore, the connection between Gaussian smoothing, Hessian-trace regularization, and flat minima has been established in existing works [3] .
-
A major limitation of the experimental evaluation is the lack of ablation studies directly demonstrating the effectiveness of the injected Langevin noise.
-
There is a substantial gap between the theoretical guarantees and the practical algorithms. The theoretical analysis focuses on vanilla ZOLD with fixed inverse temperature and smoothing radius. However, the main experiments employ an annealing schedule, which is not covered by the theoretical results. Furthermore, the analysis does not directly apply to LN-ZOLD, since layerwise normalization modifies the expected update direction.
-
The experimental evidence is insufficient to support the claimed global exploration and flat-minima benefits. The paper's central claim is that Langevin dynamics enables global exploration and favors global flat minima. However, the experimental evidence supporting this mechanism is largely restricted to a two-dimensional synthetic example. For the LLM experiments, the authors primarily report downstream task performance without directly examining whether ZOLD discovers flatter solutions, escapes unfavorable local minima, or explores the loss landscape more effectively than existing ZO methods. In addition, the inverse-temperature values used in the LLM experiments are extremely large, resulting in a relatively small injected noise magnitude. It would be useful to quantify the magnitude of the Langevin perturbations relative to the inherent noise of ZO gradient estimates and demonstrate that they meaningfully affect the optimization trajectory.
[1] Liu, L., & Wang, Z. (2020). One-Point Gradient Estimators for Zeroth-Order Stochastic Gradient Langevin Dydamics.
[2] Roy, A., Shen, L., Balasubramanian, K., & Ghadimi, S. (2022). Stochastic zeroth-order discretizations of Langevin diffusions for Bayesian inference. Bernoulli, 28(3), 1810-1834.
[3] Bruno, S., Hwang, Y., An, J., Sabanis, S., & Lim, D. Y. (2025). Flatness-Aware Stochastic Gradient Langevin Dynamics. arXiv preprint arXiv:2510.02174.
Questions for authors
I do not have any questions.