- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper empirically investigates why per-sample clipping can improve model performance in zero-th SGD. It first illustrates the limiting behavior of gradients when clipping threshold approaches either 0 or infinity, providing theoretical foundation. Then through extensive experiments on four RoBERTa classification tasks, the authors found that when learning rate and clipping threshold are inversely proportional, then clipping can robustly improve model performance by driving the model converges to a different solution. Label-corruption and data-pruning experiments suggest that the improved performance might originate from downweighting hard-to-fit examples.
Critical strengths
- Proposition 2.1 gives an intuitive explanation of the influence of clipping threshold on per-sample gradient, making later empirical observations easier to understand.
- Extensive experiments and complementary diagnostics demonstrate the performance gains from clipping.
- Label-corruption and data-pruning experiments are well-designed, properly supprting the downweighting claim.
Critical weaknesses
- Lack of clarity of the reported accuracy improvements. The paper currently in mixed-use of train and test accuracy: for example, Figure 1 reports test accuracy while Figure 2 reports train accuracy. Moreover, some statements refer broadly to improvements in “accuracy,” making it unclear whether they concern training “accuracy, test “accuracy, or both. Consistently specifying the metric would help readers understand the scope and implications of the claimed gains.
- The interpretation of convergence to different solutions needs stronger support. The distances from initialization and projected loss landscapes suggest that clipping changes the optimization trajectory. However, these finite-budget observations do not fully distinguish a lasting change in the solutions reached from differences in optimization speed or transient behavior. Stronger evidence for this distinction would substantiate a central interpretation of the paper.
Questions for authors
- Figure 1, right panel. Why is identified as fully clipped regime, when less than half (20-29 out of 64) of gradients are actually clipped? What happens if is set to a small value (say or ) so that all gradients are truly clipped. Would the performance gain remain?
- How consistent are the comparisons in Figure 5 across different random seeds and few-shot training subsets? In particular, does the advantage over Winsorization persist? Reporting means and standard deviations would help assess the reliability of this finding.
4 comments
3. Consistency Across Random Seeds in Figure 5 (Question 2)
To verify the reliability of the comparisons in Figure 5 across random seeds, we ran 5 independent random seeds on SNLI and updated Figure 5 (Section 3.4) to report the mean ± standard error in every cell for per-sample clipping, batch clipping, and the Winsorized mean. The advantage of per-sample clipping is consistent and statistically significant across seeds: per-sample clipping achieves – along (peaking at at and at ), clearly outperforming unclipped ZO-SGD (), batch clipping (peaking at ), and Winsorized mean (peaking at ). We are continuing to run multi-seed repetitions in the background for the remaining single-seed grids; however, generating multi-seed standard errors across every remaining figure in the paper, which spans four datasets (SNLI, MNLI, SST-2, TREC), two model families (RoBERTa and OPT), label corruption sweeps, and first-order grids, each requiring steps per grid cell, requires substantial compute time. We will include complete mean ± standard error bounds for all experiments in the camera-ready version.
4. Consistently Specifying "Train Accuracy" vs. "Test Accuracy" (Weakness 1)
Thank you for catching this. We have audited the entire manuscript (including Sections 3, 3.2, 3.3, 3.4, 3.5, and 4.3, as well as the figure captions) to explicitly specify "train accuracy" or "test accuracy" at every mention, and we have referenced the 2D test loss and test accuracy landscapes (Appendix G.1, Figure 25) directly alongside the training landscapes in Section 3.2 so readers can see that the test surface mirrors the training surface.
We have uploaded the revised manuscript with all updates highlighted in blue, including the expanded small- sweep and per-iteration clip-count trajectories in Appendix F.2, the expanded basin-separation analysis in Section 3.2, the 5-seed evaluations in Figure 5, and consistent train/test metric labels throughout the text. We really appreciate how much your feedback helped sharpen both the experimental evidence and the presentation of the paper, and we are happy to address any additional questions during the discussion phase. Thank you!
I sincerely thank the authors for their responses. My original concerns have been sufficiently addressing. I believe the empirical insights into the role of clipping in zeroth-order optimization in this work are interesting. Therefore, I am updating my score from 2 to 3.
We would like to thank the reviewer for their reply and constructive feedback. We will make sure to include all the revisions in the new version of the manuscript.
We thank you for the constructive review and for noting the clarity of Proposition 2.1 and the design of our label-corruption and data-pruning experiments. We have updated the manuscript (with revisions highlighted in blue) and address each of your questions and concerns below.
1. Why <50% of Gradients Are Clipped at c≤100 & Behavior at Ultra-Small c∈{0.1,0.5,1.0} (Question 1)
Thank you for pointing this out. Our original phrasing was unclear: the optimizer does operate in a nearly fully clipped regime during the active learning phase, but the 20–29 statistic in Figure 1 (right) is averaged over the entire 10,000-step training run. Zeroth-order optimization can be thought of like descending a steep slope blindfolded, where initial steps in random directions are highly volatile before flattening out near the bottom. As shown in our new per-iteration clip-count tracking in Appendix F.2 (Figure 24, left), during the first ∼4,000 steps of training, almost all 64 per-sample directional derivatives in each batch (60–64 out of 64 for c≤5) exceed c and are clipped. This initial volatile phase drives the sharp separation in weight distance ∥wt−w0∥ (Figure 1, middle) and the rapid ascent in accuracy. After this initial ∼4,000-step period, the model settles into the basin and adapts to the clean, easy examples (pushing them into the >0.9 confidence bin, Figure 7), so their gradient projections shrink below c. In the second half of training, primarily the ambiguous or mislabeled examples continue to generate gradients large enough to trigger clipping (Figures 9–10). Averaging this long settling phase with the heavily clipped initial phase pulls the 10,000-step average down to 20–29 clips per batch. We have clarified this in the caption of Figure 1 and updated our terminology throughout the paper to refer to c≤100 as the clipping-dominated regime.
To test what happens when c is set to even smaller values so that gradients continue to be clipped throughout training, we expanded our RoBERTa-SNLI sweep down to c∈{0.1,0.5,1.0} with correspondingly larger learning rates η∈{5×10−3,10−3,5×10−4} along the ηc=5×10−4 line in Appendix F.2 (Figures 22–24). Across all of these ultra-small c settings, both the ηc=5×10−4 scaling relationship and the test accuracy gains (82.7%–83.8% vs. 80.2% at c=∞) remain completely intact, and the weight distance ∥wt−w0∥ follows the exact same high-distance trajectory (Figures 22–23) while the 10,000-step average clip count rises to 33.89 out of 64 at c=0.1 (Figure 24, right).
2. Stronger Evidence of Convergence to Distinct Solutions vs. Optimization Speed (Weakness 2)
You raised an important point regarding whether clipping genuinely drives the model to a different solution basin or simply alters optimization speed along the same trajectory. We have expanded Section 3.2 (lines 218–230) to rule out the optimization-speed explanation through three pieces of evidence. First, if unclipped ZO-SGD (w3) were merely progressing more slowly along the same path toward the same minimum as clipped ZO-SGD (w1,w2), then w3 would have a higher training loss than w1 and w2 at step 10,000. Instead, w3 converges to a strictly lower training and test loss (0.45 unclipped vs. 0.70 at c=10, Figure 4 and Figure 13 right) while achieving lower train and test accuracy, whereas clipped models plateau at higher loss with higher accuracy. Second, our logit ensembling experiments (Figure 3) test functional diversity directly: ensembling two clipped checkpoints from different points on the ηc=constant line (w1 at c=10 and w2 at c=50, trained from the same initialization and random seed) yields zero ensemble gain (Figure 3, middle), confirming that varying (η,c) along the trade-off line reaches the same basin. In contrast, ensembling a clipped model (w2) with an unclipped model (w3) improves both test accuracy and test loss (Figure 3, left), demonstrating that they capture distinct solutions. Third, this matches our theoretical analysis in Proposition 2.1 and Appendix K.2: at any stationary point w∗ of the unclipped objective satisfying n1∑i=1n∇fi(w∗)=0, the expected clipped update n1∑i=1nhc(∥∇fi(w∗)∥)∇fi(w∗) remains non-zero whenever per-sample gradient norms vary across examples, driving clipped ZO-SGD to the minimizer of a distinct surrogate objective Fc(w) (derived in Appendix K.2.1, Eqs. 26–29).
Continued in next comment...