acceptodds
Under review as a conference paper at ICLR 2027

Sparsity without Sweeps: Adaptive Regularization for Mirror Descent

Abstract

Sparse training reduces the memory and computational costs of deep neural networks. However, sparse optimization methods, e.g., those adding an penalty, often control sparsity only indirectly through a regularization parameter , whose mapping to the final sparsity rate is highly non-trivial. This work addresses the selection of in the mirror descent (MD) sparse optimization framework such that a user-specified sparsity rate is achieved. Instead of treating this selection as a meta-optimization problem, which potentially requires expensive trial-and-error sweeps, our approach adapts online during training. We propose two schemes to perform the adaptation: one based on the magnitude of the largest subgradients, and the other based on the difference between the model's current and target sparsity. The first scheme can be interpreted as a dynamic sparse training cycle. A sparsity schedule further extends this control to intermediate targets, gradually sparsifying the model during training. We also provide a convergence analysis for adaptive in MD. Experiments on CIFAR-100, TinyImageNet, and ImageNet-1K show that both schemes reliably reach the final as well as the scheduled intermediate sparsity targets. Beyond controlling sparsity, the adaptive schemes match or exceed the accuracy of a tuned fixed at equal or higher sparsity in every evaluated setting, and clearly surpass it when combined with a sparsity schedule: with ResNet-50 on TinyImageNet, they reach 67.1% top-1 accuracy at 95% sparsity, compared with 64.6% for a fixed at only 90.4% sparsity.

Primary area:
optimization
Keywords:
sparse training, sparsity control, mirror descent, adaptive regularization

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_TZSKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a way to train sparse neural networks toward a user-specified sparsity level without repeatedly running training with different regularization parameters. It introduces two adaptive methods, TopK and SGap, which adjust the regularization parameter \lambda during training, and extends them with a gradual sparsity schedule. Experiments on CIFAR-100, TinyImageNet, and ImageNet-1K show that the proposed methods can reach target sparsity levels and often improve accuracy over fixed-\lambda training, especially when gradual sparsification is used.

Critical strengths
  • The proposed TopK and SGap methods adapt the regularization parameter \lambda online, reducing the need for repeated training runs to find a suitable sparsity level, which is technically sound.
  • Experiments across CIFAR-100, TinyImageNet, and ImageNet-1K show that the proposed methods achieve high sparsity while matching or outperforming fixed-\lambda baselines, particularly with gradual sparsity schedules.
  • This paper combines practical methods with theoretical analysis and provides two complementary adaptation strategies, introduces scheduled sparsification, and analyzes convergence, while connecting TopK-based mirror descent to dynamic sparse training.
Critical weaknesses
  • The novelty is limited. Setting \lambda to the (k*+1)-th largest |p| makes the soft-threshold step a top-k selection on the dual (accumulated-gradient) variable, with uniform shrinkage. This is very close to existing top-k and IHT-style methods on accumulated gradients.
  • Some baselines are missing:
    • The paper does not compare to methods that hit a target sparsity in a single run, such as constrained L0 (Gallego-Posada et al. 2022) and STR (Kusupati et al. 2020), which have been discussed in this paper.
    • Following the novelty, Top-KAST is a similar method that the paper cites but does not compare against.
  • The proposed method keeps a dense dual variable p and computes dense gradients at every step. Training is therefore not memory-sparse, which undercuts the abstract’s claim that sparse training reduces memory and compute.
  • Experiments are limited to image-classification CNNs. Other architectures such as transformers and LLMs, where sparsity control currently matters most, are not tested.
Questions for authors
  • Evaluate on other architectures.
  • Compare with more baselines (see above).
  • Discuss the dense training cost (see above).
4 comments

W1: “The novelty is limited … This is very close to existing top-k and IHT-style methods on accumulated gradients“

We agree that performing top-k selection on an accumulated gradient method is very similar to our TopK adaptation. Although this framing identifies the similarity between the methods, it does not recognize that our proposal is a bridge between sparse MD and these lines of work. We even formalized exactly this similarity with DST in Proposition 2 to show that they are related. More importantly, this framing reduces our contributions to the TopK adaptation only. Our listed contributions include the following: 1) proposing two λ\lambda adaptation schemes to control sparsity for MD, 2) an empirical study showing that adapting λ\lambda improves performance compared to tuning it for the same sparsity budget, 3) providing the convergence analysis for the iterates with adaptive λ\lambda, and 4) interpreting TopK adaptation as a DST method.

The essence of our work is hence to argue for adapting λ\lambda instead of tuning it in MD; TopK is only one possible way to perform the adaptation. In fact, we have shown that SGap adaptation leads to similar improvements. The performance gain following λ\lambda adaptation is highly non-trivial since the adaptations are designed to control sparsity, not to improve accuracy. We attempted to explain the reason based on the training dynamics as discussed in the “Topology exploration” paragraph and demonstrated in Figure 2.

We acknowledge that "contribution (III)" in the paper did not clearly connect these points. Hence, we updated its statement to be more explicit. We have also highlighted the last point in the results.

 

W2 & Q2: “The paper does not compare to methods that hit a target sparsity in a single run …”

The main relevant baseline is the (original) fixed-λ\lambda LinBreg as this comparison isolates the effect of our modification. Our paper already includes four additional sparse baselines, three of which are DST methods, since we interpret our method through the DST framework. Regarding the specific requested baselines:

  • STR [1] does not hit the target sparsity in a single run as claimed. Its authors unambiguously describe sparsity control as a drawback, requiring the tuning of not 1 but 2 hyperparameters: “The major drawback of STR is the tuning of the weight-decay parameter, λ\lambda and finer-tuning with sinits_{\text{init}} to obtain the targeted overall sparsity”. We avoided using STR as a baseline since it controls sparsity via weight-decay, making homogenizing hyperparameters with other methods infeasible.
  • Constrained L0 by Gallego-Posada et al. [2] is based on explicit, not implicit, regularization. Also, their main results are reported for structured sparsity.
  • Top-KAST [3] is a relevant baseline. But its main benefit is arguably in enabling sparse backward passes.

 

W3 & Q3: “Discuss the dense training cost”

The computational costs are briefly discussed in Appendix D: “The training cost per iteration depends on whether a method computes the gradient for the active weights only or for all weights. ... LinBreg and our methods compute it for all weights at every iteration”. Furthermore, Table 2 reports the training FLOPs, which reflect the additional cost of the dense backward pass. So, using a dense dual is stated as a cost incurred by our method, and the higher training FLOPs reflect it.

W4: "Experiments are limited to image-classification CNNs ... Other architectures such as transformers and LLMs ... are not tested."

 

We agree that extending the evaluation to transformers is of great interest. Following the reviewer's suggestion, we trained DeiT-S [4] on ImageNet-1k. Given the cost of training, a complete comparison could not be finished within the rebuttal period. Nevertheless, we report our preliminary results at 90% sparsity and (only) 100 training epochs.

OptimizerMethodTop-1 acc. [%]Best epoch
AdamWDense70.8869 (training ongoing)
AdaBreg + SGap + Ramp66.0898
GraNet52.1098
SGDDense57.75100
LinBreg + SGap + Ramp46.42100
GraNet36.5598

 

Two observations from the table stand out: 1) our adaptive method outperforms GraNet by a large margin with both optimizers, and 2) all SGD runs are considerably worse than their AdamW counterparts. The latter is a known issue [5, 6], which our evaluation likely amplified by not tuning the hyperparameters for DeiT-S. The cause of GraNet’s large degradation is less clear. A possible contributing factor is the issue of cold-start for newly regrown parameters when using Adam-style optimizers in DST [7]. However, the authors in [7] linked this issue to loss spikes after topology updates, which we did not observe in our runs. Hence, we cannot explain GraNet's performance by this phenomenon.

In short, our preliminary results indicate that our adaptive schemes can train sparse transformers and compare favorably with the DST baseline (GraNet). Nevertheless, we opt not to include this analysis in the updated manuscript, since concrete conclusions require further investigation.

 


[1] Kusupati et al. “Soft threshold weight reparameterization for learnable sparsity”. ICML, 2020. https://proceedings.mlr.press/v119/kusupati20a/kusupati20a.pdf

[2] Gallego-Posada et al., “Controlled Sparsity via Constrained optimization or: how I learned to stop tuning penalties and love constraints”. NeurIPS, 2022, https://proceedings.neurips.cc/paper_files/paper/2022/file/089b592cccfafdca8e017

[3] Jayakumar et al. “Top-KAST: Top-K always sparse training”. NeurIPS, 2020, https://proceedings.neurips.cc/paper_files/paper/2020/file/ee76626ee11ada502d5d

[4] Touvron et al. “Training data-efficient image transformers & distillation through attention”. ICML, 2021. http://proceedings.mlr.press/v139/touvron21a/touvron21a.pdf

[5] Zhang et al. "Why are adaptive methods good for attention models?". NeurIPS, 2020. https://proceedings.neurips.cc/paper/2020/file/b05b57f6add810d3b7490866d74c0053-Paper.pdf

[6] Zhang et al. "Why transformers need Adam: a Hessian perspective". NeurIPS, 2024. https://proceedings.neurips.cc/paper_files/paper/2024/file/ee0e45ff4de76cbfdf07015a7839f339-Paper-Conference.pdf

[7] Xiao et al. "Memory-Efficient LLM Training with Dynamic Sparsity: From Stability to Practical Scaling". ICML, 2026. https://proceedings.mlr.press/v306/xiao26t.html

Thank you for the response and the additional discussions and experiments on ImageNet. The rebuttal has addressed my concerns, so I would like to increase my rating.

Thank you for raising your score and for the constructive feedback.

Reviewer_w7UTOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper proposes adaptive regularization to control sparsity in mirror descent without parameter sweeps. The authors introduce two methods: TopK, which adjusts the regularization parameter using the largest subgradient magnitudes, and SGap, which uses the difference between current and target sparsity. They also introduce sparsity scheduling and provide convergence analysis. Experiments on CIFAR-100, TinyImageNet, and ImageNet-1K show that both methods reliably achieve target sparsity and competitive accuracy.

Critical strengths
  1. The paper addresses a practical limitation of mirror descent for sparse training. Finding a suitable regularization parameter usually requires multiple training runs, while the proposed methods can achieve the desired sparsity within a single run.
  2. The proposed methods are simple and well motivated. TopK directly controls the number of active weights, while SGap provides an alternative that avoids computing the TopK threshold. Both can be incorporated into the existing mirror descent framework.
  3. The authors compare against fixed-λ\lambda mirror descent and several dynamic sparse training methods on multiple datasets, including ImageNet-1K. The results show reliable sparsity control and competitive accuracy, particularly when sparsity scheduling is used.
Critical weaknesses
  1. The convergence analysis does not directly cover the proposed algorithms. The analysis assumes a given regularization parameter sequence and full gradients, while the actual methods adapt the parameter based on the current sparsity and use stochastic gradients. The theoretical results therefore provide only partial support for the methods.
  2. The comparison with fixed-λ\lambda mirror descent is not always conducted at the same sparsity level. Since different sparsity levels can affect accuracy, it is difficult to determine how much improvement comes from adaptive regularization itself.
  3. The proposed methods still require dense gradient computation during training. According to Table 2, their training FLOPs are higher than several dynamic sparse training methods. Since the paper is motivated partly by training efficiency, actual training time and memory overhead would help establish the practical benefits.
Questions for authors

Please see the weakness.

2 comments

W1: "The convergence analysis does not directly cover the proposed algorithms."

This is the same answer in response to W2 of reviewer 6KwH

 

Regarding the scope of Proposition 1, we agree with the reviewer that the actual algorithm used for the numerical examples is not covered by these assumptions. The theory analyzes the core conceptual deviation of our method from standard MD algorithms, namely adapting the regularization parameter in the elastic-net functional. To the best of our knowledge, this scheme has not been analyzed in previous works. In particular, we want to highlight that our analysis does not require convexity of the underlying loss function. This is achieved through a non-trivial adaptation of the technique developed in [1], which only assumes a Kurdyka–Łojasiewicz property to hold (satisfied e.g. by semialgebraic or real-analytic losses). The adaptation is non-trivial because changing the regularization parameter breaks the subgradient property on which existing analyses rely: the surrogate (Lyapunov) functional is no longer monotone, but only decreases up to an additive error, which, to the best of our knowledge, is not covered by existing abstract KŁ frameworks. As can be seen from the appendix, establishing these convergence properties already required a significant amount of argumentation.

The usage of full gradients is also remarked upon in the paper. We completely agree that it is a desirable step to perform the analysis with stochastic gradients. However, we want to note that, to the best of our knowledge, convergence of the iterates for MD with a nonsmooth mirror map such as the elastic net has not been established for nonconvex losses with stochastic gradients, even for a fixed regularization parameter. This requires a new technique, which we deem to go well beyond the scope of the current paper.

Regarding the assumption on the parameter sequence, it is true that in the written form the λ\lambda updates might not fulfill the desired decay property. In the revised version we add a plot (Figure 5, Appendix E) indicating that our employed schemes in fact decay faster than needed for the analysis. For Proposition 1 it suffices that the difference sequence ∣λ(t+1)−λ(t)∣|\lambda^{(t+1)} - \lambda^{(t)}| decays faster than tq−1t^{q-1}, where q<−32q < - \frac{3}{2}. The empirical plots we added suggest that this is indeed the case. A possible algorithmic modification would be to enforce ∣λ(t+1)−λ(t)∣≤Cλt−3|\lambda^{(t+1)} - \lambda^{(t)}| \leq C_{\lambda} t^{-3} by restricting the λ\lambda update to a maximum deviation per step. Here, we would choose CλC_{\lambda} large to allow bigger changes in the beginning of the algorithm. Since our analysis only uses the values of λ(t)\lambda^{(t)}, such a modification would allow the parameter sequence to depend on the iterates θ(t)\theta^{(t)} and still fulfill the desired property for the analysis.

Furthermore, it is indeed true that the momentum and dual decay strategies are currently excluded from the analysis. This will be part of future work that can build on our current work.

Overall, we want to summarize that the goal of the convergence analysis is not to capture the full implemented algorithm, but to provide theoretical guarantees for our core algorithmic modification, adapting the regularization parameter in MD. In particular, this was achieved without assuming convexity of the underlying loss function. Further extensions that move closer to the implemented algorithm are desirable, but are beyond the scope of the current work and thus left for future work.

 

W2: “The comparison with fixed λ\lambda ...”

We acknowledge this as an issue in our previous evaluations. We updated our results in Table 1 and Figures 1, 2, and 3 to use λ\lambda values tuned separately for every target, model, and dataset (previously used the same values across datasets and models). The new values result in sparsities much closer to the targets. Please refer to our revised manuscript or the answer to Q2 for reviewer 6KwH to see the updated entries.

 

W3: “The proposed methods still require dense gradient ...”

That is correct; our method inherits the dense backward passes from the original MD formulation. What our method offers is the potential efficiency gain from avoiding the sweeps over different λ\lambda values. To improve the computational efficiency of the backward passes, our method can be combined with the multilevel MD formulation of Lunk et al. (2026) [2]. However, this is currently outside the scope of this work.

 


[1] Benning et al. "Choose your path wisely: gradient descent in a Bregman distance framework." SIAM Journal on Imaging Sciences, 2021. https://epubs.siam.org/doi/epdf/10.1137/20M1357500.

[2] Lunk et al. “Sparse Training of Neural Networks based on Multilevel Mirror Descent” arXiv preprint, 2026. https://arxiv.org/pdf/2602.03535

Thanks the authors for the response. I will keep my current positive score.

Reviewer_6KwHOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces target sparsity control into the linearized Bregman framework through two online adaptation schemes: TopK selects a threshold from dual-variable magnitudes, while SGap adjusts it using the sparsity gap. Both support sparsity scheduling. The paper provides conditional convergence analysis and a DST-like interpretation. Experiments on three image classification benchmarks demonstrate effective sparsity control and accuracy improvements over fixed-parameter LinBreg, with further gains from scheduling.

Critical strengths
  1. Both schemes let users specify the desired sparsity without searching over fixed regularization parameters. TopK guarantees exact sparsity at adaptation steps under the no-tie condition, and the same adaptation settings work across the main experiments.
  2. Results cover three datasets and multiple architectures, with three seeds. At 99% target sparsity on TinyImageNet, SGap + Ramp achieves 66.4% accuracy with ResNet-50, versus 64.0% for GraNet. Additional ablations examine adaptation frequency and applicability to AdaBreg.
Critical weaknesses
  1. At 95% sparsity on ImageNet-1K, TopK + Ramp improves accuracy over GraNet from 71.7% to 72.5%, but approximately doubles training FLOPs (0.43 versus 0.21 relative to dense training). An advantage under matched training budgets remains unestablished.
  2. The convergence guarantees do not directly cover the proposed algorithms. Proposition 1 requires a convergence rate for the regularization sequence that is not established for TopK or SGap. It also excludes the stochastic updates, momentum, and dual-variable weight decay used experimentally, limiting its support for the implemented methods.
Questions for authors
  1. What are the actual sparsities of the evaluated checkpoints, especially for SGap at the 99% target?
  2. How were the fixed LinBreg regularization values selected, including the search range and criterion? Were configurations closer to each target sparsity evaluated? This would clarify whether the reported results represent a well-tuned baseline.
2 comments

 

W1: "Minor improvement at the expense of doubling training FLOPS."

Training efficiency relative to the baselines can be viewed as a shortcoming; however, we believe it can be meaningfully improved by applying our adaptive solution to efficient MD formulations, e.g., the multilevel MD formulation of Lunk et al. (2026) [1]. Such extensions are outside the scope of this work.

 

W2: "The convergence guarantees do not directly cover the proposed algorithms"

Regarding the scope of Proposition 1, we agree with the reviewer that the actual algorithm used for the numerical examples is not covered by these assumptions. The theory analyzes the core conceptual deviation of our method from standard MD algorithms, namely adapting the regularization parameter in the elastic-net functional. To the best of our knowledge, this scheme has not been analyzed in previous works. In particular, we want to highlight that our analysis does not require convexity of the underlying loss function. This is achieved through a non-trivial adaptation of the technique developed in [2], which only assumes a Kurdyka–Łojasiewicz property to hold (satisfied e.g. by semialgebraic or real-analytic losses). The adaptation is non-trivial because changing the regularization parameter breaks the subgradient property on which existing analyses rely: the surrogate (Lyapunov) functional is no longer monotone, but only decreases up to an additive error, which, to the best of our knowledge, is not covered by existing abstract KŁ frameworks. As can be seen from the appendix, establishing these convergence properties already required a significant amount of argumentation.

The usage of full gradients is also remarked upon in the paper. We completely agree that it is a desirable step to perform the analysis with stochastic gradients. However, we want to note that, to the best of our knowledge, convergence of the iterates for MD with a nonsmooth mirror map such as the elastic net has not been established for nonconvex losses with stochastic gradients, even for a fixed regularization parameter. This requires a new technique, which we deem to go well beyond the scope of the current paper.

Regarding the assumption on the parameter sequence, it is true that in the written form the λ\lambda updates might not fulfill the desired decay property. In the revised version we add a plot (Figure 5, Appendix E) indicating that our employed schemes in fact decay faster than needed for the analysis. For Proposition 1 it suffices that the difference sequence ∣λ(t+1)−λ(t)∣|\lambda^{(t+1)} - \lambda^{(t)}| decays faster than tq−1t^{q-1}, where q<−32q < - \frac{3}{2}. The empirical plots we added suggest that this is indeed the case. A possible algorithmic modification would be to enforce ∣λ(t+1)−λ(t)∣≤Cλt−3|\lambda^{(t+1)} - \lambda^{(t)}| \leq C_{\lambda} t^{-3} by restricting the λ\lambda update to a maximum deviation per step. Here, we would choose CλC_{\lambda} large to allow bigger changes in the beginning of the algorithm. Since our analysis only uses the values of λ(t)\lambda^{(t)}, such a modification would allow the parameter sequence to depend on the iterates θ(t)\theta^{(t)} and still fulfill the desired property for the analysis.

Furthermore, it is indeed true that the momentum and dual decay strategies are currently excluded from the analysis. This will be part of future work that can build on our current work.

Overall, we want to summarize that the goal of the convergence analysis is not to capture the full implemented algorithm, but to provide theoretical guarantees for our core algorithmic modification, adapting the regularization parameter in MD. In particular, this was achieved without assuming convexity of the underlying loss function. Further extensions that move closer to the implemented algorithm are desirable, but are beyond the scope of the current work and thus left for future work.

 


[1] Lunk et al. “Sparse Training of Neural Networks based on Multilevel Mirror Descent” arXiv preprint, 2026. https://arxiv.org/pdf/2602.03535

[2] Benning et al. "Choose your path wisely: gradient descent in a Bregman distance framework." SIAM Journal on Imaging Sciences, 2021. https://epubs.siam.org/doi/epdf/10.1137/20M1357500.

Q1: "What are the actual sparsities of the evaluated checkpoints, especially for SGap at the 99% target?"

All checkpoints are within ζ=0.5\zeta=0.5% of the target s∗\mathsf{s}^*. For example, SGap on ResNet-50 and TinyImageNet was evaluated at 98.9999% sparsity.

 

Q2: “How were the fixed LinBreg regularization values selected …”

Originally, we used the same λ\lambda values for WRN-28-10 and ResNet-50 on both CIFAR-100 and TinyImageNet. The choices were manually calibrated based on predefined sweeps. We acknowledge that the setup resulted in missing several targets in Table 1. We now tune the values separately per target, model, and dataset to be more accurate, as shown below:

ArchitectureDataset90%95%99%
ResNet-50CIFAR-10076.9 ± 0.2 (89.8, 0.49)75.3 ± 0.3 (94.8, 0.83)70.4 ± 0.2 (98.8, 3.1)
TinyImageNet64.7 ± 0.3 (89.8, 0.85)62.5 ± 0.4 (94.9, 1.55)58.9 ± 0.5 (98.9, 6.6)
WRN-28-10CIFAR-10077.5 ± 0.3 (89.8, 0.41)76.2 ± 0.3 (94.8, 0.75)73.4 ± 0.4 (98.8, 2.7)
TinyImageNet63.4 ± 0.3 (89.8, 0.74)61.5 ± 0.6 (94.9, 1.41)60.0 ± 0.1 (98.7, 4.9)

where each cell shows accuracy [%] followed by (s(θ∗)\mathsf{s}(\theta^*) [%], λ\lambda).

 

Although it is hard to define a concrete criterion for “well-tuned” LinBreg, we consider the new values to be well-tuned because their best checkpoints coincide with achieving the target sparsity. We use the word coincide because, for fixed-λ\lambda, the best checkpoint is selected without imposing a sparsity constraint. Hence, its sparsity can take any value along the trajectory from the initial to the final sparsity. Please refer to Figure 1 to see the updated sparsity curves.

Comments on the paper

1 comment

General response

We thank the reviewers for their valuable feedback. Below, we address the concerns shared by several reviewers and summarize the changes to the manuscript.

 

Shared concerns

  • Fixed- λ\lambda baseline at matched sparsity (6KwH Q2, w7UT W2). We re-tuned the fixed λ\lambda separately for each target, model, and dataset (previously shared across models and datasets). In Table 1, sparsity of LinBreg with fixed-λ\lambda now lie within 0.30.3% of the targets. The accuracy decreases with increasing sparsity for every setting, as expected.

  • Scope of the convergence analysis (6KwH W2, w7UT W1). Proposition 1 does not cover our full training setup. However, it covers the core algorithmic change in our method, which is adapting λ\lambda in mirror descent (MD). We assume exact gradients and no momentum or weight decay. Under these assumptions, Proposition 1 shows that if the λ\lambda updates decay sufficiently fast, the MD iterates converge for nonconvex losses. We note that, to our knowledge, convergence of MD with an elastic-net mirror map on nonconvex losses has not been established under stochastic gradients, even for fixed λ\lambda. We added Figure 5 (Appendix E), which shows that the λ\lambda updates of both our schemes satisfy the decay assumption of Proposition 1 in our experiments.

  • Training cost (6KwH W1, w7UT W3, TZSK W3). Similar to LinBreg, our adaptive schemes use dense gradients. This cost is reported in Table 2 (training FLOPs) and discussed in Appendix D. The saving our schemes offer comes from avoiding multiple training runs to tune λ\lambda, while also improving accuracy over a tuned fixed λ\lambda. Reducing the iteration cost, e.g., by combining our schemes with multilevel MD, is outside the scope of this work.

 

Summary of changes (highlighted in blue)

  • Re-tuned fixed-λ\lambda baseline: updated abstract, Tables 1 and 4, Figures 1, 2, and 3, and the corresponding text in the results section (6KwH Q2, w7UT W2).
  • Updated contribution (III) (TZSK W1).
  • Added Figure 5 (Appendix E) on the decay of λ\lambda (6KwH W2, w7UT W1).
  • Added a table of contents for the supplementary material.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.