- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes an automated parameter-selection strategy for the ADMM algorithm applied to linear-quadratic problems, eliminating the need for exhaustive parameter searching and thereby enhancing computational efficiency. The authors introduce a theoretically grounded "upper-envelope" framework along with analytical parameter-selection rules, and demonstrate the effectiveness of the proposed approach through applications in PDEs and diffeomorphic image registration.
Critical strengths
- Designing automated selection strategies for numerical algorithms that require hyperparameters is highly practical;
- The paper presents plausible theorems and proofs, and demonstrates fast convergence in practical applications;
- The paper has a relatively clear structure.
Critical weaknesses
-
ADMM is proposed to solve problems of the form , where f and g are not necessarily quadratic functions, and can even be non-differentiable functions, such as . The paper solves a specific case where and are quadratic functions, which is a relatively simple problem that can be solved directly using least squares without ADMM. Is the proposed method applicable to general problems that require ADMM for solution?
-
According to Algorithm 1, the update of depends on w and b. Is the iteration matrix in Eq. (2) correct? How can and be eliminated to obtain the form Q in Eq.(2)? It has not been derived in detail, and the formula seems to have some omissions.
-
Is the selection of ADMM parameters important? Are different parameters sensitive to the results? It is suggested to add a graph showing the decrease in the objective function value or ground-truth error of the ADMM algorithm with different parameters as the number of iterations decreases, such as in Figure 7, to more fully illustrate the importance of automated parameter selection.
Questions for authors
-
Discuss the applicability of the proposed method to more general problems solved by ADMM;
-
Supplement the sensitivity analysis of the ADMM algorithm to parameters;
-
Is Eq. (2) correct?
2 comments
Thank you for the detailed clarifications and additional experimental results. The authors have provided further explanations regarding the scope of the proposed method, supplemented the mathematical derivations, and conducted additional parameter sensitivity experiments, which have helped address several of my concerns.
Nevertheless, the applicability of the proposed method remains relatively limited, as it is specifically designed for structured linear-quadratic problems rather than general nonlinear or nonsmooth formulations. Additionally, I am unable to fully verify the correctness of all the mathematical derivations.
We thank the reviewer for the constructive comments and positive assessment of our work. Below, we address the three critical weaknesses (CW1–CW3) and their corresponding questions (Q1–Q3).
Critical Weaknesses:
CW1 / Q1: Applicability to general ADMM problems and motivation for solving quadratic problems with ADMM.
As discussed in the Introduction, although the first-order optimality condition of Eq. (1) yields the normal equation
(μA⊤A+L⊤L)u=μA⊤f,
directly solving this system can be computationally challenging. The system matrix may be singular, and even when it is positive definite, direct methods can be computationally expensive for large-scale problems.
The key advantage of ADMM in our setting is that it exploits the structure of the problem through operator splitting. Specifically, the resulting subproblems involve the shifted operators μA⊤A+θI and L⊤L+θI, which are positive definite for θ>0. More importantly, these subproblems often admit efficient specialized solvers, including FFT/DCT diagonalization, multigrid methods, and Sherman–Morrison–Woodbury identities, as explained in the Introduction and demonstrated in our PDE and image-registration applications.
We emphasize that our contribution is an analytical parameter-selection framework specifically designed for structured linear-quadratic problems. We do not claim that the proposed spectral upper-envelope analysis directly extends to general nonlinear or nonsmooth ADMM formulations. Such extensions are beyond the scope of the present work.
CW2 / Q3: Correctness and derivation of the iteration matrix in Eq. (2).
We appreciate the reviewer pointing out that the elimination of the auxiliary and dual variables was not explicitly derived. Below, we provide the derivation.
Let H=μA⊤A, G=L⊤L, and d=μA⊤f. From Algorithm 1, the ADMM updates are
wk+1=θ(G+θI)−1(uk+bk),
uk+1=(H+θI)−1(d+θwk+1−θbk),
bk+1=bk+uk+1−wk+1.
Combining the second and third equations gives
θbk+1=d−Huk+1.
Therefore, for k≥1, the dual variable can be eliminated as
bk=θ−1(d−Huk).
Substituting this identity into the w-update yields
wk+1=(G+θI)−1[(θI−H)uk+d].
Using the same identity in the u-update, we obtain
uk+1=(H+θI)−1[Huk+θwk+1].
Consequently, the iteration can be expressed as
uk+1=M1uk+c,
where
M1=I+θ(H+θI)−1[(G+θI)−1(θI−H)−I],
and
c=θ(H+θI)−1(G+θI)−1d.
This recovers exactly the iteration matrix given in Eq. (2), thereby confirming its correctness. We will include the intermediate elimination steps in the revised manuscript to make the derivation more explicit.
CW3 / Q2: Sensitivity to penalty and relaxation parameters.
We conducted a fixed-parameter sensitivity study on a 128×128 image-registration quadratic subproblem. All runs use identical problem data and zero initialization. Convergence is measured by the relative normal-equation residual Rk=∥Hvk−b∥2/∥b∥2, where Hv=b is the optimality condition of the quadratic problem. Each run terminates when Rk≤10−6.
The analytically selected penalties for ADMM and oADMM are both θ^≈0.0036384, with the selected oADMM relaxation β^≈1.752. For ADMM, we vary θ over {0.1,0.5,1,2,10}θ^ with β=1. For oADMM, we fix θ=θ^ and vary β.
The following table reports the number of iterations required to reach the common residual tolerance.
These results illustrate the sensitivity of ADMM and oADMM convergence to the penalty and relaxation parameters. The proposed ADMM penalty requires 149 iterations, compared with 189–1110 iterations for the tested alternatives. For oADMM, the proposed relaxation reduces the iteration count from 149 at β=1 to 82, while a larger relaxation of β=1.9 requires 159 iterations.
We will include the corresponding residual convergence curves in the revised Appendix.