acceptodds
Under review as a conference paper at ICLR 2027

Fully First-Order Lagrangian Methods for Byzantine-Robust Federated Bilevel Learning

Abstract

Federated bilevel optimization (FBO) plays a crucial role in solving hierarchical learning problems over distributed data. However, its distributed nature makes it highly susceptible to faulty or Byzantine clients. Existing Byzantine-robust FBO methods either rely on Hessian or Jacobian vector products or perform repeated robust updates within an inner loop, incurring additional differentiation or synchronization costs. To address these limitations, we propose FOBFBL, the first Byzantine-robust FBO method that is both fully first-order and single-loop. With a finite Lagrangian multiplier , it forms three coupled blocks of stochastic gradients that the server robustly aggregates at their natural scales in one round. For nonconvex upper-level and strongly convex lower-level objectives, we establish a nonasymptotic stationarity bound that separates the decaying optimization error, the Byzantine-induced floor, and the Lagrangian bias, yielding iterations above the residual floor. Under an additional upper-level PL condition, we obtain a last-iterate function-gap bound with finite-horizon dependence. We further develop the Polyak-momentum variant FOBFBL-M, which removes stochastic variance from the persistent Byzantine error and leaves a limiting neighborhood governed by honest heterogeneity and the Lagrangian bias. Experiments on federated hyperparameter optimization and hyper-representation learning validate both methods.

Primary area:
optimization
Keywords:
Federated Bilevel Optimization, Byzantine Robustness, Fully First-Order Optimization, Single-Loop Algorithms, Lagrangian Methods

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_CwpNOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes FOBFBL, a fully first-order, single-loop method for Byzantine-robust federated bilevel optimization, and its momentum variant. By using finite multiplication numerical functions to rephrase and blockwise robust aggregation of three first-order directions, high-order calculations and repetitive inner loop communication are avoided. The paper establishes convergence guarantees for the nonconvex setting and under an additional upper-level PL condition, and proves that appropriately configured momentum eliminates the stochastic-variance term from the asymptotic Byzantine-induced error. Experiments on MNIST and CIFAR-10 evaluate robustness and computational, memory, and communication costs.

Critical strengths

(1) Finite-multiplier reformulation and blockwise robust aggregation enable fully first-order, single-loop updates in one communication round, reducing computation and communication costs.

(2) The paper provides guarantees for nonconvex stationary points under strong convex conditions in the lower layer and for the final iteration under PL conditions in the upper layer. It distinguishes between optimization error, Byzantine error, and multiplier bias, and proves that appropriate momentum can eliminate the random variance term in asymptotic Byzantine error.

(3) The experiment covers hyperparameter optimization and hyper representation learning, comparing various attacks, aggregators, and baselines, balancing predictive performance and resource cost, and comprehensively evaluating robustness and single update efficiency.

Critical weaknesses

(1) The main components have direct precedents, but there is a lack of robust aggregation versions such as MemFBO and BR FedBiM for direct comparison, making it difficult to clarify the independent benefits and importance of the design in this article.

(2) The correspondence between theoretical assumptions and experimental implementation is unclear. The description of the HPO model is inconsistent, and the local update protocol is not clear, which raises doubts about whether the strong convexity and conditional unbiasedness of the lower layer hold, and whether the existing convergence guarantees cover the actual implementation.

(3) The experiment has not yet systematically examined the trade-off between approximate bias, noise, and heterogeneity of finite multipliers, nor has it directly verified the influence of momentum on asymptotic residual error under theoretical parameter settings.

(4) The current efficiency evaluation focuses on the cost of a single update, and in some configurations, the accuracy is lower than the main baseline. Therefore, it is still necessary to compare the total time, communication volume, and Oracle call times under the same accuracy or stagnation error tolerance.

Overall Assessment after Rebuttal Thank you for the detailed revisions and clarifications. The authors have provided detailed responses to my remaining technical concerns, and the additional explanations have improved the clarity and rigor of the manuscript. I appreciate their efforts and acknowledge the value of the theoretical contributions. However, my overall assessment of the manuscript remains unchanged, as the revisions do not substantially alter my evaluation of its novelty and significance. I therefore maintain my original rating.

Questions for authors

(1) What HPO models and configurations are used for Table 2, Appendix B.1 and B.3 respectively? What satisfies the assumption of strong convexity in the lower layer?

(2) What are the advantages of our method compared to MemFBO and BR FedBiM using block robust aggregation under the same experimental setup and comparable parameter tuning budget?

(3) Do local steps repeat sampling at fixed shared parameters or continuously update parameters? If it is the latter, how to define the uploaded message, and does it still satisfy the condition of unbiasedness required by equation (10)?

(4) Can you provide an experiment that satisfies theoretical assumptions and allows for direct calculation of real hypergradients to verify convergence analysis?

(5) Does the total running time, number of communication bytes, and number of calls still demonstrate significant efficiency advantages when achieving the same accuracy or stationary error tolerance?

(6) How is the attack of ALIE/IPM on three message blocks constructed, before or after momentum processing and gradient pruning? If the attacker can observe the current round of honest messages, does the conditional independence required for non-momentum proof still hold?

7 comments
AuthorsOpenReview ↗

Response to Reviewer CwpN (1/4)

W1 & Q2: Independent benefits relative to robustly aggregated MemFBO and BR-FedBiM.

Thank you for your comment. Relative to blockwise robust MemFBO, our contribution is a convergence analysis that controls coupled Byzantine and finite-multiplier errors, together with a momentum extension that removes stochastic variance from the persistent Byzantine error. Relative to BR-FedBiM, we establish this variance-removal property using only ordinary gradients. The BR-FedBiM comparisons have been added to Tables 2–3 in Section 5.1, highlighted in red‾\underline{\text{red}}.

  • Relation to MemFBO. Replacing MemFBO's [1] server averaging with blockwise robust aggregation gives the FOBFBL update structure under matched parameters and protocols. Our Lyapunov analysis controls biased aggregation and the coupled evolution of xx and its two lower-level estimates, establishing Theorem 1 with separate Byzantine and finite-multiplier error terms.

  • Difference from BR-FedBiM. BR-FedBiM [2] removes stochastic variance from the asymptotic Byzantine error using Hessian- and Jacobian-vector products. Theorem 3 achieves this using ordinary gradients, by jointly controlling momentum lag and both lower-level tracking errors under its parameter choices. Heterogeneity and finite-multiplier terms remain.

  • Comparison with BR-FedBiM. Both methods outperform BR-FedBiM in all 16 HPO settings. For hyper-representation learning, FOBFBL and FOBFBL-M lead in worst-attack accuracy with Trimmed Mean and CWMed, respectively. BR-FedBiM leads with Krum and Median (Table R1). Methods use the same five local steps and 1,000-round communication budget, with fixed paper configurations and no additional tuning (Appendix B.1).

Table R1. Worst-attack accuracy (%) in the two MNIST tasks. Minimum across four attacks of the three-seed mean accuracy in Tables 2–3.

TaskAggregatorBR-FedBiMFOBFBLFOBFBL-M
HPOKrum61.8066.1965.94
HPOMedian55.8165.0966.20
HPOTrimmed Mean56.6161.5862.75
HPOCWMed54.1762.9763.75
Hyper-representationKrum88.2565.9272.95
Hyper-representationMedian82.4576.3168.88
Hyper-representationTrimmed Mean77.9384.4776.43
Hyper-representationCWMed77.3884.6185.08

References

[1] Yang et al. First-Order Federated Bilevel Learning. AAAI 2025.

[2] Li et al. Single-Loop Byzantine-Resilient Federated Bilevel Optimization. ICLR 2026.


W2 & Q1: HPO model configurations and lower-level strong convexity.

Thank you for pointing this out. We apologize for the confusion in the model description. The MNIST HPO experiments in Table 2 use an MLP. We have corrected Section 5.1 and clarified the upper- and lower-level HPO objectives, with Appendix B.1 referring to the same setting. These changes are highlighted in red‾\underline{\text{red}}. Strong convexity is used in the theoretical analysis, and our experiments cover both strongly convex and nonconvex lower levels.

  • Our algorithms can apply to both strongly convex and nonconvex lower-level objectives. FOBFBL and FOBFBL-M use only first-order gradients and run the same updates in both settings. In the convergence proof, strong convexity makes the lower-level solution y⋆(x)y^\star(x) unique and Lipschitz in xx, which bounds the lower-level tracking and finite-multiplier approximation errors.

  • Our experiments cover both settings. Hyper-representation learning (Tables 3–4) has a strongly convex lower level: a regularized linear classifier with fixed features. HPO uses a 784–200–100–10 MNIST MLP (Table 2, Appendices B.1/B.3) and a CIFAR-10 CNN with 16/32 convolutional channels and a 2048–100–10 classifier (Appendix B.3). Its upper level tunes 20 classwise VS-loss parameters using class-balanced validation cross-entropy. The nonconvex lower level trains all network weights with VS loss plus 0.01∣y∣2/20.01\\|y\\|^2/2.

AuthorsOpenReview ↗

Response to Reviewer CwpN (2/4)

W2 & Q3: Local updates, uploaded messages, and conditional unbiasedness.

Our theoretical analysis considers one local step per communication round. The neural-network experiments use five sequential local updates per round.

  • Extending the analysis to multiple local steps. Under the stated oracle assumptions, the directions in Eq. (10) are conditionally unbiased at the broadcast point when one local step is used. With multiple steps, directions remain conditionally unbiased at their local query points, but their average can be biased at the broadcast point. Extending the proof requires controlling local drift [1] alongside lower-level tracking, robust aggregation, and momentum errors.

  • The neural-network experiments use sequential local updates. Clients start from (xk,yk,zk)(x^k,y^k,z^k) and update all three variables five times, using fresh minibatches at each step, with some derivative components sharing a minibatch. For directions hv,ik,sh_{v,i}^{k,s} evaluated at the local parameters, FOBFBL uploads the average

\qquad v\in\\{x,y,z\\}.$$ FOBFBL-M maintains a momentum buffer for each client across rounds [2,3] and uploads $m_{v,i}^k=\beta_v m_{v,i}^{k-1}+(1-\beta_v)\widetilde h_{v,i}^k$. The server then robustly aggregates each block and updates the broadcast parameters. **References** [1] Allouah et al. Byzantine-Robust Federated Learning: Impact of Client Subsampling and Local Updates. ICML 2024. [2] Karimireddy et al. Learning from History for Byzantine Robust Optimization. ICML 2021. [3] Farhadkhani et al. Byzantine Machine Learning Made Easy By Resilient Averaging of Momentums. ICML 2022. --- > **Q4: Convergence measured using the exact hypergradient.** Thank you for your suggestion. **FOBFBL reduces the true stationarity residual as the iteration budget increases, both with and without Byzantine attacks.** Our new synthetic experiment measures this residual using the exact hypergradient on a problem satisfying Assumptions 1–5. * **A strongly convex lower level with a closed-form solution.** For each honest client $i$, we define the upper-level objective $f_i$ and lower-level objective $g_i$ as $$\begin{aligned} f_i(x,y)&=0.05\\|x\\|^2+\sum_{j=1}^{20}\left(\sqrt{1+(y_j-t_{i,j})^2}-1\right),\\\\ g_i(x,y)&=\tfrac12 y^\top Ay-y^\top(Bx+h_i). \end{aligned}$$ Here $x,y\in\mathbb R^{20}$, $A=A^\top\succ0$ and $B$ are shared across clients, and $h_i,t_i$ encode client heterogeneity. Both levels use averages over the honest clients $\mathcal H$, with $n_h=|\mathcal H|$. The lower-level Hessian is $A\succ0$, giving the unique minimizer $$y^\star(x)=A^{-1}(Bx+\bar h), \qquad \bar h=\frac1{n_h}\sum_{i\in\mathcal H}h_i.$$ FOBFBL uses independent zero-mean Gaussian oracle noise of bounded variance, one local step, and stepsizes proportional to $K^{-1/2}$ with a pilot-selected coefficient. * **True stationarity error decreases even under attack.** The residual falls by **87.3% with ALIE + CWMed and 90.0% without attacks** over the tested budgets (Table Q4). The exact hypergradient of $\Phi_{\mathcal H}(x)=n_h^{-1}\sum_{i\in\mathcal H}f_i(x,y^\star(x))$ follows from the chain rule: $$\nabla\Phi_{\mathcal H}(x)=0.1x+B^\top A^{-1}\frac1{n_h}\sum_{i\in\mathcal H} \frac{y^\star(x)-t_i}{\sqrt{1+(y^\star(x)-t_i)^2}}.$$ The fraction is evaluated coordinatewise, and this exact hypergradient is used only for evaluation. Table Q4 reports its mean squared norm over the $K$ iterates, $K^{-1}\sum_{k=0}^{K-1}\\|\nabla\Phi_{\mathcal H}(x^k)\\|^2$, corresponding to Theorem 1's stationarity criterion. **Table Q4. Average squared hypergradient across iteration budgets.** FOBFBL, synthetic task, $\lambda=8$, seed 42. Each budget is a separate run. | Updates $K$ | No attack (Mean) | ALIE (CWMed) | | --- | ---: | ---: | | 1,000 | 1.056 | 1.076 | | 4,000 | 0.7138 | 0.7428 | | 16,000 | 0.4058 | 0.4377 | | 64,000 | 0.2098 | 0.2414 | | 256,000 | 0.1055 | 0.1365 |
AuthorsOpenReview ↗

Response to Reviewer CwpN (3/4)

W3: The trade-off between finite-multiplier bias, stochastic noise, and client heterogeneity.

Thank you for your comment. The experiments support task-dependent multiplier selection and identify noise suppression as momentum's clearest benefit in these tests. We measure test accuracy on MNIST HPO and the exact squared hypergradient on a synthetic problem with a strongly convex lower level. All tests below use seed 42.

  • 1. Select λ\lambda to balance bias and perturbations on the target task.

On MNIST HPO, both methods peak at λ=0.5\lambda=0.5, with similar accuracy at smaller multipliers and lower accuracy at larger values under the stated stepsize scaling (Table R2).

Table R2. MNIST HPO accuracy (%) across λ\lambda. ALIE + CWMed, 1,000 updates. ηy=0.05/λ\eta_y=0.05/\lambda, other nominal stepsizes fixed, and β=0.5\beta=0.5 for FOBFBL-M.

Methodλ=0.1\lambda=0.10.250.250.50.51122551010
FOBFBL77.7377.6378.0177.3474.8468.8761.28
FOBFBL-M77.5077.6177.6677.4574.8767.8760.16

On the synthetic problem, the lowest residual occurs at λ=8\lambda=8 for FOBFBL and λ=16\lambda=16 for FOBFBL-M (Table R3). Larger values worsen the residual, consistent with Remark 2's bias–perturbation trade-off.

Table R3. Synthetic squared-hypergradient residual across λ\lambda. Strongly convex lower level, ALIE + CWMed, σ=a=0.003\sigma=a=0.003. Final-10% average over 100,000 updates, with fixed stepsizes and β=0.5\beta=0.5 for FOBFBL-M.

Methodλ=2\lambda=233448816163232
FOBFBL0.006160.002940.001770.0007880.001360.00477
FOBFBL-M0.006120.002880.001690.0005610.0005440.00163
  • 2. Moderate momentum preserves HPO accuracy and is especially effective under high noise.

MNIST HPO accuracy is stable across moderate β\beta and falls at the largest tested value (Table R4).

Table R4. MNIST HPO accuracy (%) across β\beta. ALIE + CWMed, λ=1\lambda=1, 1,000 updates. Here β=0\beta=0 recovers FOBFBL.

β\beta00.250.50.750.90.95
Final test accuracy (%)77.3477.4377.4577.4077.1876.77

Momentum lowers the synthetic residual in all three settings, with a 65.1% reduction under high noise, versus 17.8% under high heterogeneity (Table R5). Varying aa changes the honest objectives, so the heterogeneity comparison measures sensitivity to that change.

Table R5. Synthetic residuals under separate noise and heterogeneity changes. ALIE + CWMed, λ=8\lambda=8, fixed stepsizes, β=0.5\beta=0.5 for FOBFBL-M. Final-10% averages over 100,000 updates. Reductions are relative to FOBFBL.

SettingNoise σ\sigmaHeterogeneity aaFOBFBLFOBFBL-MRelative reduction (%)
Reference0.0030.0030.0007880.00056128.8
Higher noise0.030.0030.03060.010765.1
Higher heterogeneity0.0030.30.001740.0014317.8
  • 3. Coupled momentum reduces the stationarity residual's sensitivity to noise.

Using Theorem 3's coupling form, β(K)=1−400ηK\beta(K)=\sqrt{1-400\eta_K} with ηK∝K−1/2\eta_K\propto K^{-1/2}, we compare noisy and noiseless synthetic runs at zero heterogeneity. All methods use matched stepsizes at each budget, with pilot-selected scaling constants. Coupled momentum gives the lowest average squared true hypergradient at every tested budget (Table R6). Its residual increase over the paired noiseless run falls from 0.00393 to 0.00166 as the budget grows. At the largest budget, this increase is 0.03212 without momentum and 0.01585 with fixed momentum.

Table R6. Synthetic problem: full-run average squared true hypergradient. ALIE + CWMed, λ=8\lambda=8, a=0a=0, σ=0.03\sigma=0.03. Separate runs for each budget.

Updates KKNo momentum (FOBFBL)Fixed momentum (β=0.5\beta=0.5)Coupled momentum (β(K)\beta(K))
4,0001.2006501.1932491.188419
16,0000.8918750.8801440.870067
64,0000.5590410.5443730.530746
256,0000.3130410.2967720.282857
Reviewer_zV6SOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies Byzantine-robust federated bilevel optimization with nonconvex upper-level and strongly convex lower-level objectives. It proposes FOBFBL, a fully first-order, single-loop method that combines a finite-multiplier Lagrangian formulation with robust aggregation of gradient blocks. This paper establishes convergence guarantees, with an additional last-iterate one under an upper-level PL condition. They also introduce a momentum variant, FOBFBL-M, whose asymptotic error bound reduces stochastic variance under the stated assumptions. Experiments on federated hyperparameter optimization and hyper-representation learning evaluate the proposed methods’ robustness and computational and communication efficiency.

Critical strengths

The motivation for avoiding Hessian-vector products and repeated inner-loop updates in federated bilevel optimization is clear.

Critical weaknesses
  1. The implications of the finite-multiplier bias for the complexity comparison remain insufficiently characterized. Theorems 1–4 explicitly include the Lagrangian bias, and Table 1 acknowledges that the complexity of the proposed methods are measured above the residual floor. However, this qualification is central to interpreting the comparison. As Remark 2 explains, increasing λ\lambda reduces the bias but amplifies lower-level noise and heterogeneity contributions and tightens the stepsize conditions. The paper would benefit from a quantitative characterization of this tradeoff and a comparison using a common accuracy criterion. Alternatively, Table 1 should explicitly distinguish the accuracy criteria and residual terms associated with the reported rates.

  2. The empirical results leave the accuracy–efficiency tradeoff unclear, particularly for hyper-representation learning. Table 3 shows substantial accuracy gaps relative to BR-FedBi in many configurations and relative to Rob-FedBOB under Random Noise attacks. For example, with Krum under Random Noise, FOBFBL and FOBFBL-M achieve 78.06% and 81.19%, compared with 96.46% for BR-FedBi and 94.79% for Rob-FedBOB. Moreover, Figure 1 measures costs per update or over a fixed number of updates, rather than costs to reach comparable accuracy. Consequently, it remains unclear when the computational savings compensate for the accuracy losses. In conjunction with the finite-multiplier bias, this limits the evidence for the practical advantage of the proposed methods.

Questions for authors
  1. Could the authors provide a quantitative choice of λ\lambda that balances the Lagrangian bias against the noise and heterogeneity terms, and state the resulting attainable accuracy and complexity, including the relevant λ\lambda-dependence? Remark 2 discusses this tradeoff qualitatively, but a quantitative result would help interpret Table 1.

  2. While Appendix B.1 acknowledges that the MLP-based HPO experiments fall outside the strongly convex lower-level setting of the theory, could the authors explain what insights the theory provides for this nonconvex setting? Alternatively, could the effectiveness of the proposed methods be validated in a controlled setting that satisfies the theoretical assumptions, particularly lower-level strong convexity?

8 comments
AuthorsOpenReview ↗

Response to Reviewer zV6S (1/3)

W1 & Q1: Accuracy criteria, multiplier selection, and residual errors.

Thank you for highlighting the effect of finite-multiplier bias. For W1, we now compare complexity using a common accuracy criterion that includes this bias. In the Byzantine-free case, both methods attain E∣∇Φ(xR)∣2≤ε\mathbb E\\|\nabla\Phi(x^R)\\|^2\leq\varepsilon with O(ε−3.5)\mathcal O(\varepsilon^{-3.5}) stochastic-gradient calls per client, including initialization, matching the ε\varepsilon exponent of MemFBO and basic F²SA under their respective assumptions. For Q1, Corollary 2 gives quantitative multiplier choices that balance bias against noise and heterogeneity, the resulting attainable residual bounds, and the complexity of approaching those bounds. Equation (255) explicitly retains the λ\lambda dependence. These derivations in Appendix E.2 and the Asymptotic Error column in manuscript Table 1 are highlighted in red‾\underline{\text{red}}.

  • 1. Total-accuracy complexity matches MemFBO and basic F²SA. In the Byzantine-free case (κ=0\kappa=0), we use the common criterion E∣∇Φ(xR)∣2≤ε\mathbb E\\|\nabla\Phi(x^R)\\|^2\leq\varepsilon. Choosing λ=Θ(ε−1/2)\lambda=\Theta(\varepsilon^{-1/2}) makes the squared approximation bias O(ε)\mathcal O(\varepsilon). Accounting for the resulting stepsize dependence, Corollary 1, Appendix E.2 (pp. 53–54) gives O(ε−3.5)\mathcal O(\varepsilon^{-3.5}) stochastic-gradient calls per client for both methods, including initialization. This matches the ε\varepsilon exponent of MemFBO [1, Corollary 5.12] and basic F²SA [2, Corollary 4.2], up to logarithmic factors under their respective assumptions. Table R1 compares this exponent with client counts fixed and stochastic gradients at both levels.

  • 2. The additional Byzantine error is reported separately. Under attacks, the residual contains both the finite-multiplier bias O(λ−2)\mathcal O(\lambda^{-2}) and the Byzantine contribution in the last column of Table R1. The same additional-error terms appear in manuscript Table 1. Momentum removes stochastic variance from this persistent Byzantine contribution under Theorem 3's conditions. Here δ\delta is the Byzantine fraction, κ=O(δ)\kappa=\mathcal O(\delta), and σ2=σf2+λ2σg2\sigma^2=\sigma_f^2+\lambda^2\sigma_g^2.

Table R1. Sample complexity and additional Byzantine error. N. A.: no Byzantine guarantee.

MethodSample complexity (κ=0\kappa=0)Additional Byzantine error
Basic F²SA [2]O~(ε−3.5)\widetilde{\mathcal O}(\varepsilon^{-3.5})N. A.
MemFBO [1]O(ε−3.5)\mathcal O(\varepsilon^{-3.5})N. A.
FOBFBLO(ε−3.5)\mathcal O(\varepsilon^{-3.5})O!(δ(σ2+ζf2+λ2ζg2))\mathcal O\\!\left(\delta(\sigma^2+\zeta_f^2+\lambda^2\zeta_g^2)\right)
FOBFBL-MO(ε−3.5)\mathcal O(\varepsilon^{-3.5})O!(δ(ζf2+λ2ζg2))\mathcal O\\!\left(\delta(\zeta_f^2+\lambda^2\zeta_g^2)\right)
  • 3. Under attacks, the multiplier balances bias against lower-level perturbations. The bias decreases as λ−2\lambda^{-2}, while the lower-level Byzantine contribution grows as κλ2(σg2+ζg2)\kappa\lambda^2(\sigma_g^2+\zeta_g^2) for FOBFBL and κλ2ζg2\kappa\lambda^2\zeta_g^2 for FOBFBL-M. Balancing these terms gives, up to fixed problem constants,
\bigl[\kappa(\sigma_g^2+\zeta_g^2)\bigr]^{-1/4}, \qquad \lambda_{\mathrm{FOBFBL-M}}\asymp (\kappa\zeta_g^2)^{-1/4}.$$ **Corollary 2, Appendix E.2 (pp. 54–55)** establishes these choices when the growing perturbation term is positive and the admissible multiplier lower bound is inactive. Thus, **stronger lower-level perturbations favor a smaller balancing multiplier**. For each fixed selected $\lambda$, reducing the squared-gradient error to **at most $\epsilon$ above the residual bound** requires $\mathcal O(\epsilon^{-2})$ stochastic-gradient calls per client. **Equation (255) gives the full $\lambda$ dependence**, with initialization and stepsize conditions in Corollary 2. If the growing perturbation term vanishes, $\lambda$ is instead chosen to control the approximation bias at the desired tolerance. The separate Q1 experiments examine this trade-off across noise, heterogeneity, and Byzantine fractions. **References** [1] Yang, Yifan, Peiyao Xiao, Shiqian Ma, and Kaiyi Ji. "First-Order Federated Bilevel Learning." AAAI, 2025. [2] Kwon, Jeongyeol, Dohyun Kwon, Stephen Wright, and Robert D. Nowak. "A Fully First-Order Method for Stochastic Bilevel Optimization." ICML, 2023.
AuthorsOpenReview ↗

Response to Reviewer zV6S (2/3)

Q1: Additional experiments on multiplier selection.

Thank you for your question. Alongside the theoretical choices in Corollary 2, Appendix E.2, our experiments support selecting λ\lambda to balance approximation bias against perturbation amplification at a fixed training budget.

  • 1. Excessively large λ\lambda increases the stationarity residual under perturbations. With mean aggregation and no attacks, noise, or heterogeneity, increasing λ\lambda reduces the residual throughout the tested grid. Under attacks, both methods instead achieve their lowest measured residual at an intermediate λ\lambda (Table R2). For example, increasing λ\lambda from 8 to 32 raises the FOBFBL residual approximately sixfold in the reference attack setting. This illustrates why reducing the approximation bias alone is insufficient for choosing λ\lambda.

  • 2. Stronger perturbations favor smaller λ\lambda in these experiments. Increasing either noise or heterogeneity shifts the best tested λ\lambda downward for both methods (Table R2). Increasing the Byzantine fraction produces the same trend while keeping the honest clients, their objectives, noise, and stepsizes fixed (Table R2b). These finite-budget sweeps use fixed stepsizes and β=0.5\beta=0.5, so even the momentum residual remains noise-dependent. Removing stochastic variance from the persistent Byzantine term requires the theorem's coupled stepsize and momentum choices. The sweeps therefore guide multiplier selection at a fixed training budget.

Table R2. Best tested λ\lambda and squared-hypergradient residual. Strongly convex synthetic lower level, 100,000 updates, seed 42. Grid: 2,3,4,8,16,32\\{2,3,4,8,16,32\\}, fixed stepsizes, β=0.5\beta=0.5 for FOBFBL-M. Final-10% averages. Attacked settings: ALIE + CWMed.

SettingFOBFBL: best λ\lambdaFOBFBL: residualFOBFBL-M: best λ\lambdaFOBFBL-M: residual
No perturbations320.0000285320.0000285
Reference with attacks (σ=a=0.003\sigma=a=0.003)80.000788160.000544
Higher noise with attacks (σ=0.03\sigma=0.03)30.0079140.00456
Higher heterogeneity with attacks (a=0.3a=0.3)40.0012140.00108

Table R2b. Best tested λ\lambda and residual across Byzantine fractions. Same budget and metric as Table R2. ALIE + CWMed, with 18 fixed honest clients and σ=a=0.003\sigma=a=0.003, including at δ=0\delta=0.

Byzantine fraction δ\deltaFOBFBL: best λ\lambdaFOBFBL: residualFOBFBL-M: best λ\lambdaFOBFBL-M: residual
0.0%320.000129320.0000655
10.0%160.000343160.000195
18.2%80.000619160.000331
28.0%80.000788160.000544
AuthorsOpenReview ↗

Response to Reviewer zV6S (3/3)

W2: Accuracy, efficiency, and cost to a common accuracy.

Thank you for raising this trade-off. In whole-network HPO, FOBFBL-M reaches comparable accuracy with less training time and uses less sampled tensor memory. The original hyper-representation task uses a linear lower-level head, where BR-FedBi and Rob-FedBOB achieve higher accuracy under Krum + Random Noise (Table 3). Our new MNIST benchmark retains this attack and aggregator while optimizing the full CNN at the lower level.

  • 1. Comparable accuracy with less training time. On the larger CNN, FOBFBL-M reaches 94.49% at 900 s, compared with 93.50% for BR-FedBi at 1,800 s. At a common 90% target, mean training time falls from 462.0 to 395.7 s, a 14.3% reduction (Table R3).

  • 2. Lower sampled memory use. On the same CNN, FOBFBL-M reduces sampled tensor allocation from 374.2 to 238.4 MiB, a 36.3% reduction. Sampled driver allocation decreases by 3.7%. These measurements quantify the memory benefit for this whole-network HPO setting.

Both methods use layerwise ℓ2\ell_2 regularization, five local steps, batch size 128, identical data splits, matched validation-tuning budgets, and the same MPS device. Configurations and prediction rules (zz for FOBFBL-M, yy for BR-FedBi) are fixed before test evaluation.

Table R3. MNIST CNN HPO under Krum + Random Noise, mean ± SD over three seeds. Full budgets: 600 s for (32, 64), 1,800 s for (128, 256).

CNN channelsMethodAcc@half (%)Acc@full (%)Time to 90% (s)Time to 95% (s)Sampled tensor peak (MiB)
32, 64BR-FedBi94.57 ± 0.5095.30 ± 0.5545.0 ± 7.0149.2 ± 27.497.0 ± 3.9
32, 64FOBFBL-M95.27 ± 0.9596.96 ± 0.7840.4 ± 8.4110.1 ± 34.357.7 ± 0.4
128, 256BR-FedBi92.19 ± 2.4393.50 ± 1.04462.0 ± 38.2NR (0/3)374.2 ± 42.1
128, 256FOBFBL-M94.49 ± 0.2494.05 ± 1.23395.7 ± 36.91075.3 ± 210.2238.4 ± 0.7

Time includes updates, attack generation, and aggregation, excluding evaluation, warmup, and I/O. Targets are first observed crossings and need not be maintained. NR (0/3): no seed reaches 95% within budget. Memory is the maximum sampled over seven single-client calls per seed, a lower bound on peak allocation.


Q2: Nonconvex lower-level objectives and validation under strong convexity.

Thank you for your suggestion. We have added a strongly convex HPO experiment to complement the nonconvex MLP results.

  • 1. Strong convexity supports the proof, while the updates apply in both settings. Both methods use the same first-order updates for strongly convex and nonconvex lower levels. In the proof, strong convexity makes y⋆(x)y^\star(x) unique and Lipschitz in xx, enabling control of tracking and finite-multiplier approximation errors. The MLP experiments test empirical performance beyond this assumption.

  • 2. The added experiment directly tests lower-level strong convexity. FOBFBL or FOBFBL-M achieves the highest worst-attack accuracy with five of six aggregators (Table R4). BR-FedBi leads with vector Trimmed Mean. The linear-classifier HPO objectives are

\qquad y^\star(x)=\arg\min_y\left\\{ \ell_{\mathrm{train}}(x,y)+\frac{0.01}{2}\\|y\\|^2 \right\\}.$$ Here $x$ contains 20 classwise training-loss parameters, and $y$ contains all classifier weights and biases. Both losses average over honest clients, with class-balanced validation cross-entropy at the upper level. For fixed $x$, the training loss is convex in $y$, so the regularizer makes the lower objective **$0.01$-strongly convex**. **Table R4. Strongly convex MNIST HPO: worst-attack accuracy (%) across four attacks.** 1,000 exchanges, seed 42. All methods use the column's aggregator. Trimmed Mean: vector trimming. GeoMed: geometric median. | Method | Krum | Median | Trimmed Mean | CWMed | CWTrimMean | GeoMed | | --- | ---: | ---: | ---: | ---: | ---: | ---: | | FedNest | 52.33 | 37.71 | 56.64 | 58.85 | 57.44 | 59.96 | | AggITD | 49.61 | 23.72 | 59.48 | 23.58 | 55.04 | 51.99 | | BR-FedBi | 60.32 | 62.85 | **65.54** | 60.30 | 64.86 | 65.71 | | Rob-FedBOB | 65.16 | 63.83 | 59.46 | 64.25 | 64.45 | 65.37 | | FOBFBL | 65.15 | **66.49** | 52.13 | 68.49 | 68.96 | 69.13 | | FOBFBL-M | **66.70** | 66.45 | 43.02 | **69.01** | **69.68** | **70.51** |
Reviewer_V8ayOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

FOBFBL robustly aggregates three first-order Lagrangian gradient blocks in one server exchange, and FOBFBL-M adds local momentum for federated bilevel learning with Byzantine clients. The analysis separates optimization error, Byzantine error, and finite-multiplier bias, while experiments report attack robustness and lower update costs on MNIST and CIFAR-10 tasks.

Critical strengths
  1. The method protects both lower-level trackers and the upper update, rather than averaging client hypergradients that generally do not equal the population hypergradient (Equations 6-12). Its three-block protocol needs ordinary gradients and one communication exchange per iteration.
  2. The theory clearly separates errors that vanish from those that remain. It explains the multiplier tradeoff, states the restricted honest-client speedup of the momentum variant, and limits the lower-bound comparison to the Byzantine heterogeneity term (Theorems 1-4 and Remarks 1-3). The momentum aggregation argument also avoids assuming that aggregation errors are unbiased (Lemma 9 proof).
  3. The empirical study includes four attacks, four aggregators, multiple datasets, and three-seed means with variation. The efficiency appendix gives unusually explicit accounting for time, memory, message sizes, and synchronization, and states what these measurements exclude.
Critical weaknesses
  1. The HPO model is described inconsistently (major but fixable). Section 5.1 says multinomial logistic regression, while Appendix B.1 calls the same accuracy experiments a nonconvex MLP problem and links them to Table 2. This affects whether lower-level strong convexity is plausible. Identify the exact model and objective behind every accuracy table and curve, and separate theorem-compatible tests from neural-network demonstrations.
  2. Equal communication rounds do not give an equal local computation budget here (major but fixable). The accuracy experiment uses five local stochastic steps for most methods but one for BR-FedBi, whereas the cost probe uses five for all methods. These are different protocols, so their results cannot directly establish end-to-end efficiency. Add matched-gradient-budget comparisons and time/bytes to a common accuracy, including robust server aggregation; the current timing probe uses one client and mean aggregation.
  3. The momentum comparison misses a particularly relevant baseline (fixable). The paper explicitly describes BR-FedBiM as an earlier robust bilevel method with the same variance-removal property, but the empirical comparison lists BR-FedBi rather than its momentum variant. Compare with BR-FedBiM under matched tuning and local work, and report paired FOBFBL versus FOBFBL-M differences; Tables 2-3 show that momentum is not uniformly better.
  4. The main practical accuracy tradeoff is not tested directly (fixable evidence gap). A larger multiplier reduces approximation bias but increases lower-level noise and heterogeneity effects, while the variance-removal theorem uses a horizon-dependent momentum schedule (Remark 2 and Theorem 3). The listed ablations vary attacks, client count, and data imbalance, not these choices; the cost probe fixes the multiplier and momentum. Add multiplier/momentum sweeps and controlled noise-versus-heterogeneity tests, and explain how users choose these values without knowing the theoretical constants.
Questions for authors

Please refer to the weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer V8ay (1/3)

W1: Inconsistent HPO model description and the applicability of lower-level strong convexity.

Thank you for pointing this out. We apologize for the confusion in the model description. The MNIST HPO experiments in Table 2 use an MLP. We have corrected Section 5.1 and clarified the upper- and lower-level HPO objectives, with Appendix B.1 referring to the same setting. These changes are highlighted in red‾\underline{\text{red}}. Strong convexity is used in the theoretical analysis, and our experiments cover both strongly convex and nonconvex lower levels.

  • Our algorithms can apply to both strongly convex and nonconvex lower-level objectives. FOBFBL and FOBFBL-M use only first-order gradients and run the same updates in both settings. In the convergence proof, strong convexity makes the lower-level solution y⋆(x)y^\star(x) unique and Lipschitz in xx, which bounds the lower-level tracking and finite-multiplier approximation errors. The algorithms remain effective on nonconvex lower levels, as the neural-network HPO results below show.

  • The model and lower-level objective are specified for each experiment. Table 2 and the curves in Appendix B.1 use a 784–200–100–10 MNIST MLP for HPO. The upper level tunes classwise VS-loss parameters using class-balanced validation cross-entropy, while the nonconvex lower level trains all MLP weights with regularized VS loss. Tables 3–4 and Appendix B.2 use CNNs for hyper-representation learning. With the feature extractor fixed at each upper-level point, the lower level optimizes a regularized linear classifier and is strongly convex. Figure 1 and Appendix B.3 use the same MNIST MLP and a CIFAR-10 CNN for HPO, both with nonconvex lower levels. These neural-network HPO results provide empirical evidence beyond the strong-convexity setting of our analysis.


W2: Comparable local computation budgets and the cost of reaching a common accuracy.

Thank you for pointing this out. We apologize for the confusion in the local-step description. All methods, including BR-FedBi, use five local steps in Tables 2–3, as clarified in Section 5.1 and Appendix B.2.1. We have added comparisons under a common gradient budget and measured time and communication to common accuracy targets, including client computation and robust aggregation in the timing.

  • The accuracy advantage persists under a common gradient budget. Both proposed methods exceed the strongest baseline by more than 13 percentage points under the same gradient-call cap (Table R1a). In particular, both outperform Rob-FedBOB at exactly the same gradient count. The advantage therefore persists when local computation is controlled. Our methods also avoid the HVP/JVP evaluations required by BR-FedBi, FedNest, and AggITD, which are reported separately.

Table R1a. Accuracy under a 36,000-gradient budget. MNIST HPO, ALIE + CWMed, seed 42. Last complete update within the cap.

MethodGradient callsHVP callsJVP callsAccuracy (%)
FOBFBL36,0000075.88
FOBFBL-M36,0000075.80
BR-FedBi36,00012,00012,00044.42
Rob-FedBOB36,0000061.91
FedNest35,17597590053.17
AggITD35,4251,22590051.46
  • Higher accuracy within the time budget, with lower communication at a shared target. Only FOBFBL and FOBFBL-M reach 75% accuracy within 300 s, and both finish with higher accuracy than every baseline (Table R1b). At the shared 60% target, they use less communication than every baseline that reaches it and are faster than Rob-FedBOB and AggITD. FedNest reaches this lower target sooner, while our methods use less than half its communication payload. The comparison therefore establishes gains in attainable accuracy and communication efficiency, with a time advantage that depends on the target and baseline.

Table R1b. Time and communication at the first recorded target crossings. Same setting as Table R1a, with 300 s on the same GPU. Time excludes transmission. Payload includes uploads and downloads. NR: not reached.

MethodAccuracy at budget (%)Time to 20% (s)Time to 40% (s)Time to 60% (s)Time to 75% (s)Payload to 60% (GB)
FOBFBL77.4913.430.773.4231.11.99
FOBFBL-M76.3018.939.280.1226.02.14
BR-FedBi44.4229.2105.7NRNRNR
Rob-FedBOB68.0533.168.8151.8NR8.80
FedNest48.145.421.361.5NR5.02
AggITD57.575.419.6154.4NR13.25
AuthorsOpenReview ↗

Response to Reviewer V8ay (2/3)

W3: Comparison with BR-FedBiM and the empirical contribution of momentum.

Thank you for your suggestion. We have added BR-FedBiM to Tables 2 and 3 in Section 5.1, highlighted in red‾\underline{\text{red}}, covering both MNIST tasks, four aggregators, four attacks, and three seeds per configuration. We also report a separate paired ablation below to isolate the effect of momentum.

  • Comparison with BR-FedBiM using only first-order gradients. Both methods exceed BR-FedBiM in all 16 HPO settings. For hyper-representation learning, FOBFBL and FOBFBL-M lead in worst-attack accuracy with Trimmed Mean and CWMed, respectively, while BR-FedBiM leads with Krum and Median.

  • Paired gains from momentum. All four methods use the same data split, seed, batch size, five local steps, and 1,000-round budget, with fixed configurations and no additional tuning. Momentum improves FOBFBL in five of eight settings, including all IPM settings (Table R2). Under ALIE, it improves CWMed but slightly lowers accuracy with the other aggregators.

Table R2. Paired momentum ablation: accuracy (%). MNIST HPO, 1,000 rounds, seed 42. Paired runs differ only in β\beta (0 versus 0.5). Δ Ours: FOBFBL-M minus FOBFBL. Trimmed Mean: vector trimming.

AttackAggregatorBR-FedBiBR-FedBiMFOBFBLFOBFBL-MΔ Ours (pp)
IPMKrum54.5060.0661.9663.33+1.37
IPMMedian53.1960.9164.3065.04+0.73
IPMTrimmed Mean53.6652.2765.5966.85+1.26
IPMCWMed58.9059.6060.9962.10+1.11
ALIEKrum62.0261.5964.9163.94-0.97
ALIEMedian37.6655.8468.8767.90-0.97
ALIETrimmed Mean63.9063.6671.9071.74-0.16
ALIECWMed26.2460.9077.0277.23+0.21

W4(a): Multiplier selection and the trade-off between bias and perturbations.

Thank you for your suggestion. The multiplier sweeps support choosing λ\lambda for the target task. We report test accuracy on MNIST HPO and the exact squared-hypergradient residual on a synthetic problem with a strongly convex lower level.

  • Select λ\lambda to balance bias and perturbations on the target task.

On MNIST HPO, both methods maintain similar accuracy across the smaller tested multipliers and peak at λ=0.5\lambda=0.5, while larger values reduce accuracy under the stated stepsize scaling (Table R3). This gives a useful starting range for selection on MNIST HPO.

For practical λ\lambda selection on MNIST HPO, start with λ∈0.5,1\lambda\in\\{0.5,1\\} at a fixed β\beta, retain the ηy\eta_y scaling, and compare held-out upper-level loss under equal budgets.

Table R3. MNIST HPO accuracy (%) across λ\lambda. ALIE + CWMed, 1,000 updates, seed 42. ηy=0.05/λ\eta_y=0.05/\lambda, other nominal stepsizes fixed, and β=0.5\beta=0.5 for FOBFBL-M.

Methodλ=0.1\lambda=0.10.250.250.50.51122551010
FOBFBL77.7377.6378.0177.3474.8468.8761.28
FOBFBL-M77.5077.6177.6677.4574.8767.8760.16

On the synthetic problem, both methods achieve their lowest measured residual at an intermediate multiplier, λ=8\lambda=8 for FOBFBL and λ=16\lambda=16 for FOBFBL-M (Table R4). Larger values worsen the residual, illustrating Remark 2's trade-off between reduced approximation bias and amplified lower-level perturbations. The different preferred multipliers across the two tasks reflect their different objectives, metrics, and stepsize settings.

Table R4. Synthetic problem: squared-hypergradient residual across λ\lambda. Strongly convex lower level, ALIE + CWMed, σ=a=0.003\sigma=a=0.003, seed 42. Final-10% average over 100,000 updates, with fixed stepsizes and β=0.5\beta=0.5 for FOBFBL-M.

Methodλ=2\lambda=233448816163232
FOBFBL0.006160.002940.001770.0007880.001360.00477
FOBFBL-M0.006120.002880.001690.0005610.0005440.00163
AuthorsOpenReview ↗

Response to Reviewer V8ay (3/3)

W4(b): Momentum selection, noise, and heterogeneity.

Thank you for your suggestion. Moderate momentum preserves HPO accuracy, while its clearest benefit in the synthetic tests is reduced sensitivity to noise. We use MNIST HPO to examine practical β\beta selection and a synthetic problem with a strongly convex lower level to separate noise and heterogeneity effects.

  • 1. Moderate momentum preserves HPO accuracy and is especially effective under high noise.

On MNIST HPO, accuracy changes little across moderate momentum values, while the largest tested β\beta reduces accuracy (Table R5). After selecting λ\lambda, compare β∈0.25,0.5,0.75\beta\in\\{0.25,0.5,0.75\\} using held-out upper-level loss and freeze the configuration before testing. This practical procedure does not require the unknown theoretical constants. The formal guarantees additionally require the parameter conditions in Theorems 1 and 3.

Table R5. MNIST HPO accuracy (%) across β\beta. ALIE + CWMed, λ=1\lambda=1, 1,000 updates, seed 42. Here β=0\beta=0 recovers FOBFBL.

β\beta00.250.50.750.90.95
Final test accuracy (%)77.3477.4377.4577.4077.1876.77

On the synthetic problem, we vary noise and heterogeneity separately and measure the exact squared-hypergradient residual (Table R6). Momentum reduces the residual in all three settings, with the largest relative reduction of 65.1% under high noise, compared with 17.8% under high heterogeneity. This identifies noise suppression as a particularly pronounced benefit in these tests. Here, varying aa changes the honest objectives, so the heterogeneity comparison measures sensitivity to that change.

Table R6. Synthetic residuals under separate noise and heterogeneity changes. ALIE + CWMed, λ=8\lambda=8, fixed stepsizes, β=0.5\beta=0.5 for FOBFBL-M, seed 42. Final-10% averages over 100,000 updates. Reductions are relative to FOBFBL.

SettingNoise σ\sigmaHeterogeneity aaFOBFBLFOBFBL-MRelative reduction (%)
Reference0.0030.0030.0007880.00056128.8
Higher noise0.030.0030.03060.010765.1
Higher heterogeneity0.0030.30.001740.0014317.8
  • 2. Coupled momentum reduces the stationarity residual's sensitivity to noise.

On the same synthetic problem, we use Theorem 3's coupling form, β(K)=1−400ηK\beta(K)=\sqrt{1-400\eta_K} with ηK∝K−1/2\eta_K\propto K^{-1/2}, and compare noisy and noiseless runs at zero heterogeneity. All methods use the same stepsizes at each budget, with pilot-selected scaling constants. Coupled momentum gives the lowest average squared true hypergradient at every tested budget (Table R7). Its residual increase relative to the paired noiseless run falls from 0.00393 to 0.00166 as the budget grows. At the largest budget, this increase is 0.03212 without momentum and 0.01585 with fixed momentum. These results support reduced noise sensitivity of the stationarity residual under the tested coupling.

Table R7. Synthetic problem: full-run average squared true hypergradient. ALIE + CWMed, λ=8\lambda=8, a=0a=0, σ=0.03\sigma=0.03, seed 42. Separate runs for each budget.

Updates KKNo momentum (FOBFBL)Fixed momentum (β=0.5\beta=0.5)Coupled momentum (β(K)\beta(K))
4,0001.2006501.1932491.188419
16,0000.8918750.8801440.870067
64,0000.5590410.5443730.530746
256,0000.3130410.2967720.282857
AuthorsOpenReview ↗

A Kind Reminder Regarding Our Author Response

Dear Reviewer V8ay,

Thank you for your thoughtful feedback on our submission. We have posted our responses and revised the manuscript, with particular attention to the following points:

  • Corrections to the HPO model description and clarification of the strong-convexity assumption.
  • New BR-FedBiM comparisons, plus evaluations under a common gradient-call budget and cost measurements at shared accuracy targets.

If you have a chance to review our responses, we would be grateful for any further feedback and would be happy to clarify any remaining questions or concerns.

We sincerely appreciate the time and care you have devoted to reviewing our work. Thank you again for considering our responses.

Warm regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Author Responses and Manuscript Revisions

Dear reviewers and AC,

Thank you for your thoughtful comments, questions, and suggestions. We have addressed each reviewer’s comments in our detailed responses. We summarize the main experiments and revisions addressing these questions below.

New experiments

To address the reviewers’ questions, we added the following experiments and comparisons. Changes in the revised manuscript are highlighted in red‾\underline{\text{red}}.

  • Added BR-FedBiM comparisons on both MNIST tasks (Section 5.1, Tables 2–3).
  • Tested CNN and strongly convex models for MNIST hyperparameter optimization. On the larger CNN, FOBFBL-M reached 90% accuracy faster than BR-FedBi (Reviewer zV6S).
  • Compared methods at matched accuracy and under a common gradient budget (Reviewers CwpN and V8ay).
  • Tested multiplier and momentum choices across noise and heterogeneity levels (Reviewers V8ay and zV6S).
  • Measured convergence with the exact hypergradient and verified the multiplier condition (Appendix B.6, Table 8).
  • Nearest-neighbor mixing before Krum improved both first-order methods under Random Noise in the follow-up experiment (Reviewer zV6S).

Other manuscript updates and clarifications

  • Updated the complexity comparison and clarified how to choose the multiplier (Table 1, Appendix E.2).
  • Corrected the model description and documented the five-step experiments and one-step guarantee (Sections 4.2 and 5.1, Appendix B.5).
  • Clarified the attack assumptions in the non-momentum analysis (Section 3.2, Lemma 4).

Thank you again for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.