- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper learns diagonal preconditioners for proximal gradient (PG) with a term and for primal-dual hybrid gradient (PDHG). A neural network predicts a small number of preconditioners once per problems. Their inverses are interpolated using hat functions, producing a sequence of preconditioners for each iterate. Spectral rescaling enforces sufficient conditions for convergence. Experiments with wavelet and total-variation priors for deblurring and inpainting show improvements in iteration counts over several baselines.
Critical strengths
- The construction is simple and well motivated, with some useful properties: interpolating inverse metrics makes the spectral inequalities affine, and the finite collection of representative metrics makes it possible to control an infinite sequence.
- The separation between training and convergence is nice: under the stated spectral assumptions, convergence does not depend on the quality of the learned generator.
Critical weaknesses
-
The scope of the method is restricted. The experiments are limited to classical wavelet-sparse and TV restoration models that are favorable to the chosen diagonal preconditioning. Applicability to more recent image restoration approaches using nonconvex deep priors is not established, as these settings fall outside the current convergence guarantees. These limitations should be stated more clearly.
-
A separate network is trained for each algorithm/operator pair: the experiments do not provide evidence of transfer across degradation operators or noise levels.
-
The targets used both for training and for evaluation are the solutions of a single method. However, the convex problem is not assumed to have a unique minimizer. For PDHG, the accuracy of all methods is evaluated by distance to a selected numerical reference, which may change with the algorithm. Moreover, the chosen loss forces to agree with one particular minimizer.
-
Figure 2 does not establish, as in the table, a consistent improvement over FISTA.
-
The benefit beyond classical or recent learned metrics is not fully established. An important baseline comparison is to learn only a scalar stepsize for each iteration instead of the full diagonal. The question is whether learning this simpler scalar sequence already provides much of the acceleration. The experiments should also compare with more evolved metrics (not only diagonal preconditoners) such as (Tan et al., 2023) .
-
Ten images with one noise realization each and no uncertainty estimates provide limited evidence of robustness.
Questions for authors
- Are the PDHG reference solutions unique in the tested instances?
- Have you considered learning the metrics by unrolling ?
4 comments
Response to Reviewer 2vFY (Part 2/2)
3. On the transfer across noise levels and observation operators (W2)
We added experiments in which the generators are used without retraining at other noise levels and with other observation operators.
Noise levels. The generators trained at are tested from a quarter of this noise level to four times of it. For PG, the order of the methods in changes little in deblurring, and LVM attains the final PSNR earlier than the other methods in inpainting over this whole range. For PDHG, LVM attains the smallest over most of the time range up to four times the noise level of the training. At half and a quarter of it, some baselines attain smaller .
Observation operators. For deblurring, the results depend on the type of the kernel. With the other motion blur kernels, the decrease of by LVM stays close to that with the kernel of the training. With the Gaussian and square kernels, it slows down, and there is a baseline faster than LVM for each of PG and PDHG. For inpainting, the acceleration by LVM is kept with other masks, and the relation between LVM and the baselines is the same as that with the mask of the training.
Thanks to the comment of the reviewer, we believe that these results clarified the range in which the proposed method can be used without retraining. Importantly, LVM converges in every setting without retraining, since the theorems do not depend on the training. Indeed, even in the settings where the acceleration weakened, LVM never diverged, which is consistent with Theorems 3.3 and 3.4.
To address these points, we made the following revisions:
- Appendix I.3: We added the experiments at other noise levels (Figures 5 and 6).
- Appendix I.4: We added the experiments with other blur kernels and masks (Figures 7 to 10).
4. On the comparison with other learned metrics (W5, Q2)
Learned scalar stepsizes. Following the suggestion, we added "Learned stepsizes" to the ablation study of PG, which learns one scalar stepsize for each iteration. The following table briefly summarizes the results (the mean number of iterations until first falls below each tolerance).
| Task | Method | ||||
|---|---|---|---|---|---|
| Deblurring | LVM | 3 | 10 | 29 | 62 |
| Deblurring | Learned stepsizes | 4 | 11 | 46 | 84 |
| Inpainting | LVM | 3 | 8 | 29 | 67 |
| Inpainting | Learned stepsizes | 27 | 48 | 96 | 142 |
Learned stepsizes needs more iterations than LVM at every tolerance in both tasks. As for PG, we also compared LVM for PDHG with a variant that learns scalar stepsizes, and it needs more iterations than LVM at every tolerance (see Table 9 for details).
Learning by unrolling. The training of LVM is similar to unrolling in that it is truncated at the horizon . However, LVM differs from unrolling in that (i) the algorithm converges to a solution for any parameter of the generator, (ii) the learning does not alter the problem to be solved, and (iii) the metrics are defined for all the iterations, including those after . To clarify these points, we added this discussion to Section 3.3 (lines 365–371).
Comparison with non-diagonal metrics. We apologize that the intention of our comparison was not sufficiently conveyed. The experiments mainly investigate how LVM yields effective metrics for a given algorithm while keeping its updates efficiently computable. In this paper, we compare LVM with the methods that keep the proximity operators efficiently computable, and for this reason, we did not compare it with methods outside this scope, e.g., Tan et al. (2023). To convey this intention precisely, we clarified the scope of the experiments in Section 4.
To address these points, we made the following revisions:
- Appendix I.5: We added Learned stepsizes and the variants without the variation over the coordinates to the ablation study (Tables 8 and 9).
- Section 4 (lines 388–392): We clarified the scope of the compared methods.
5. On the scope of the method (W1)
As the reviewer correctly mentioned, the convergence guarantees of Theorems 3.3 and 3.4 rely on the convexity of and and do not directly cover weakly convex regularizers learned by neural networks.
To state the scope more clearly, we made the following revisions:
- Section 3.2 (lines 344–346): We stated that the guarantees rely on the convexity of and , and do not directly cover weakly convex regularizers learned by neural networks.
- Section 5 (lines 538–539): We added the extension to weakly convex or learned regularizers as future work.
We are grateful to the reviewer for the careful review and the comments, which improved the manuscript.
Thanks for the detailed responses and the additional experiments. I have two additional comments based on your responses.
-
The revised evaluation only briefly comments the dependence on the choice of the minimizer. Did you identify any nonunique instances in the experiments, and did you check whether the learned acceleration is sensitive to the choice of training reference in those cases ?
-
Table 8 answers whether full LVM beats a learned scalar schedule shared across images. It leaves open how much of the gain comes from the diagonal metric itself, because LVM also adapts to each image.
Response to Reviewer 2vFY
We appreciate the reviewer's continued engagement in the discussion. We address the two additional comments below.
1. On the uniqueness of the solutions and the reference solutions for the training
Uniqueness of the solutions. In deblurring, we confirmed that the solutions of both the problem for PG and the problem for PDHG are unique. Specifically, we numerically confirmed that the smallest singular value of the observation operator is positive for all the blur kernels and image sizes in our experiments (at least for the kernel in Section 4). In inpainting, the observation operator is not injective. For five test images, we computed two reference solutions, in the same setting as in the training and by replacing only its initial point with the true image. For PG, is on average and did not change with iterations, and the relative difference between their objective values is at most . For PDHG, it is on average, which is smaller than that for PG, but even this does not imply that the solution is unique.
Sensitivity to the reference solutions for the training. Based on the above, we compared for the iterates of each method in inpainting. For PG (Table 11), the ratios of LVM are comparable to those of the other methods (except FISTA, which is the solver used to compute ), and we observe no evident sign that the iterates of LVM are particularly drawn to . For PDHG, the same ratio for LVM is at most . We did not train LVM with other reference solutions, and we left the effect of their choice on the acceleration as future work.
Based on these discussions, we made the following revisions:
- Appendix I.7: We added this subsection, which contains the discussion on the uniqueness of the solutions and the comparison with the two reference solutions (Table 11).
- Section 5: We added the sensitivity to the reference solutions as future work.
2. On the contribution of the diagonal metric (Table 8)
Following the comment, we added " spatial variation" to the ablation study of PG. This variant generates one scalar stepsize at each representative time for each image, and its metric does not vary over the coordinates. Note that it does not use the rescaling, since the rescaling for PG maps every scalar candidate to the same fixed stepsize. It needs more iterations than LVM at every tolerance in both tasks, as shown in the table below, which reports the mean number of iterations until first falls below each tolerance.
| Task | Method | ||||
|---|---|---|---|---|---|
| Deblurring | LVM (ours) | 3 | 10 | 29 | 62 |
| Deblurring | spatial variation | 4 | 21 | 41 | 90 |
| Inpainting | LVM (ours) | 3 | 8 | 29 | 67 |
| Inpainting | spatial variation | 26 | 49 | 82 |
Since this variant still generates the stepsizes for each image, this result suggests that the variation over the coordinates contributes to the acceleration of LVM, although the variant also differs from LVM in the rescaling and eventually diverges in inpainting.
To address these points, we added " spatial variation" to Table 8 and to the discussion in Appendix I.5.
We hope that these additions resolve the remaining concerns of the reviewer.
Response to Reviewer 2vFY (Part 1/2)
We sincerely thank the reviewer for the careful review and for the constructive feedback. Our responses are given below.
1. On the robustness of the evaluation (W4, W6)
Number of images and uncertainty. In Section 4, we replaced the test set by BSD68 (68 images), and Tables 2 and 3 report the standard deviations over the images. The relation between LVM and the compared methods is unchanged from the original submission, and the standard deviations of LVM are at the same level as those of the compared methods without an inertial term. We also added an experiment that repeats the benchmark with five independent realizations of the noise. The standard deviation of the number of iterations over the realizations is at most 1.1% of the mean for PG and at most 3.6% for PDHG.
Comparison with FISTA. As the reviewer correctly mentioned, the relation between LVM and FISTA depends on the experimental setting. In the updated results, FISTA is faster than LVM at every tolerance in deblurring, whereas in inpainting LVM is faster down to Gn≤10−5 (31 ms and 47 ms at this tolerance). Let us clarify that the experiments mainly investigate how LVM yields effective metrics for a given algorithm. FISTA accelerates the iterations by an inertial term and not by the metric, and this term can be combined with changes of the metric, including LVM.
To address these points, we made the following revisions:
2. On the reference solutions and the training loss (W3, Q1)
Uniqueness and evaluation. In our experimental setting, the solution of PDHG is not necessarily unique. Following the comment, we replaced the distance to the reference solution by the measure Gn defined for PDHG as
Gn:=max{∥Dx⋆∥2,1∣∥Dxn∥2,1−∥Dx⋆∥2,1∣,εmax{0,∥Φxn−b∥−ε}},
where ∥Dx∥2,1 is the value of the total variation of x and x⋆ is the reference solution. Since all the solutions share the same value of the total variation, Gn does not depend on the choice of the reference solution. The overall tendency is unchanged with this new measure, and LVM still reaches every tolerance in the fewest iterations (Table 3 and Figure 3).
Training loss. As the reviewer pointed out, when an instance has multiple solutions, the loss (13) does not necessarily take all of them into account. Let us clarify that this affects only which metrics are learned. The convergence to a solution holds for any parameter of the generator (Theorems 3.3 and 3.4).
To address these points, we made the following revisions: