acceptodds
Under review as a conference paper at ICLR 2027

Learning Variable Metrics for Convergent Proximal Splitting Algorithms

Abstract

We propose learned variable metric (LVM), a framework that learns metrics of proximal splitting algorithms adapted to each problem instance, coordinate, and iteration with a convergence guarantee. Proximal splitting algorithms can solve a wide variety of convex optimization problems but often require many iterations to converge. Choosing a suitable metric is an effective remedy, and variable metric theory naturally extends convergence guarantees to metrics that vary over the iterations. Learning is a promising way to design such adaptive metrics beyond the reach of manual design, but a naive implementation leads to (i) an expensive evaluation of a network at every iteration and (ii) difficulty in satisfying the conditions that the theory imposes on the whole infinite sequence of metrics. To resolve these difficulties, we first train a neural network to generate a few representative metrics once per problem instance, and then obtain the metric of every iteration by combining them with basis functions of the normalized iteration index. We present LVM for the proximal gradient method with a quadratic term and the primal-dual hybrid gradient method, and prove their convergence under a mild spectral condition, which is ensured by a rescaling of the representative metrics. In experiments on deblurring and inpainting, LVM reduces the number of iterations to reach a target accuracy in most tested settings compared with several baselines, and the computation time at high accuracies.

Primary area:
optimization
Keywords:
Learning to optimize, preconditioning, variable-metric, proximal splitting algorithm, convergence guarantee, inverse problem.

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_2vFYOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper learns diagonal preconditioners for proximal gradient (PG) with a L2L^2 term and for primal-dual hybrid gradient (PDHG). A neural network predicts a small number of preconditioners once per problems. Their inverses are interpolated using hat functions, producing a sequence of preconditioners for each iterate. Spectral rescaling enforces sufficient conditions for convergence. Experiments with wavelet and total-variation priors for deblurring and inpainting show improvements in iteration counts over several baselines.

Critical strengths
  • The construction is simple and well motivated, with some useful properties: interpolating inverse metrics makes the spectral inequalities affine, and the finite collection of representative metrics makes it possible to control an infinite sequence.
  • The separation between training and convergence is nice: under the stated spectral assumptions, convergence does not depend on the quality of the learned generator.
Critical weaknesses
  1. The scope of the method is restricted. The experiments are limited to classical wavelet-sparse and TV restoration models that are favorable to the chosen diagonal preconditioning. Applicability to more recent image restoration approaches using nonconvex deep priors is not established, as these settings fall outside the current convergence guarantees. These limitations should be stated more clearly.

  2. A separate network is trained for each algorithm/operator pair: the experiments do not provide evidence of transfer across degradation operators or noise levels.

  3. The targets x∗x^* used both for training and for evaluation are the solutions of a single method. However, the convex problem is not assumed to have a unique minimizer. For PDHG, the accuracy of all methods is evaluated by distance to a selected numerical reference, which may change with the algorithm. Moreover, the chosen loss forces to agree with one particular minimizer.

  4. Figure 2 does not establish, as in the table, a consistent improvement over FISTA.

  5. The benefit beyond classical or recent learned metrics is not fully established. An important baseline comparison is to learn only a scalar stepsize for each iteration instead of the full diagonal. The question is whether learning this simpler scalar sequence already provides much of the acceleration. The experiments should also compare with more evolved metrics (not only diagonal preconditoners) such as (Tan et al., 2023) .

  6. Ten images with one noise realization each and no uncertainty estimates provide limited evidence of robustness.

Questions for authors
  1. Are the PDHG reference solutions unique in the tested instances?
  2. Have you considered learning the metrics by unrolling ?
4 comments
AuthorsOpenReview ↗

Response to Reviewer 2vFY (Part 1/2)

We sincerely thank the reviewer for the careful review and for the constructive feedback. Our responses are given below.

1. On the robustness of the evaluation (W4, W6)

Number of images and uncertainty. In Section 4, we replaced the test set by BSD68 (68 images), and Tables 2 and 3 report the standard deviations over the images. The relation between LVM and the compared methods is unchanged from the original submission, and the standard deviations of LVM are at the same level as those of the compared methods without an inertial term. We also added an experiment that repeats the benchmark with five independent realizations of the noise. The standard deviation of the number of iterations over the realizations is at most 1.1% of the mean for PG and at most 3.6% for PDHG.

Comparison with FISTA. As the reviewer correctly mentioned, the relation between LVM and FISTA depends on the experimental setting. In the updated results, FISTA is faster than LVM at every tolerance in deblurring, whereas in inpainting LVM is faster down to Gn≤10−5\mathcal{G}_ n\le10^{-5} (31 ms and 47 ms at this tolerance). Let us clarify that the experiments mainly investigate how LVM yields effective metrics for a given algorithm. FISTA accelerates the iterations by an inertial term and not by the metric, and this term can be combined with changes of the metric, including LVM.

To address these points, we made the following revisions:

  • Section 4: We replaced Set10 by BSD68, added the standard deviations to Tables 2 and 3, clarified the scope of the experiments, and revised the discussion on FISTA according to the new results.
  • Appendix I.2: We added the results for other realizations of the noise (Tables 6 and 7).

2. On the reference solutions and the training loss (W3, Q1)

Uniqueness and evaluation. In our experimental setting, the solution of PDHG is not necessarily unique. Following the comment, we replaced the distance to the reference solution by the measure Gn\mathcal{G}_ n defined for PDHG as

Gn:=max⁡{∣∥Dxn∥2,1−∥Dx⋆∥2,1∣∥Dx⋆∥2,1,max⁡{0,∥Φxn−b∥−ε}ε},\mathcal{G}_ n:=\max\left\lbrace\frac{\left|\Vert D\boldsymbol{x}_ n\Vert_{2,1}-\Vert D\boldsymbol{x}^{\star}\Vert_{2,1}\right|}{\Vert D\boldsymbol{x}^{\star}\Vert_{2,1}},\quad\frac{\max\lbrace 0,\Vert\Phi\boldsymbol{x}_ n-\boldsymbol{b}\Vert-\varepsilon\rbrace}{\varepsilon}\right\rbrace,

where ∥Dx∥2,1\Vert D\boldsymbol{x}\Vert_{2,1} is the value of the total variation of x\boldsymbol{x} and x⋆\boldsymbol{x}^{\star} is the reference solution. Since all the solutions share the same value of the total variation, Gn\mathcal{G}_ n does not depend on the choice of the reference solution. The overall tendency is unchanged with this new measure, and LVM still reaches every tolerance in the fewest iterations (Table 3 and Figure 3).

Training loss. As the reviewer pointed out, when an instance has multiple solutions, the loss (13) does not necessarily take all of them into account. Let us clarify that this affects only which metrics are learned. The convergence to a solution holds for any parameter of the generator (Theorems 3.3 and 3.4).

To address these points, we made the following revisions:

  • Section 4.2 and Appendix G.5: We replaced the measure for PDHG by the new Gn\mathcal{G}_ n and updated Table 3 and Figure 3. We also added its definition.
  • Section 3.3: We added a footnote to the main text on the training for instances with multiple solutions.
AuthorsOpenReview ↗

Response to Reviewer 2vFY (Part 2/2)

3. On the transfer across noise levels and observation operators (W2)

We added experiments in which the generators are used without retraining at other noise levels and with other observation operators.

Noise levels. The generators trained at σ=0.01\sigma=0.01 are tested from a quarter of this noise level to four times of it. For PG, the order of the methods in Gn\mathcal{G}_ n changes little in deblurring, and LVM attains the final PSNR earlier than the other methods in inpainting over this whole range. For PDHG, LVM attains the smallest Gn\mathcal{G}_ n over most of the time range up to four times the noise level of the training. At half and a quarter of it, some baselines attain smaller Gn\mathcal{G}_ n.

Observation operators. For deblurring, the results depend on the type of the kernel. With the other motion blur kernels, the decrease of Gn\mathcal{G}_ n by LVM stays close to that with the kernel of the training. With the Gaussian and square kernels, it slows down, and there is a baseline faster than LVM for each of PG and PDHG. For inpainting, the acceleration by LVM is kept with other masks, and the relation between LVM and the baselines is the same as that with the mask of the training.

Thanks to the comment of the reviewer, we believe that these results clarified the range in which the proposed method can be used without retraining. Importantly, LVM converges in every setting without retraining, since the theorems do not depend on the training. Indeed, even in the settings where the acceleration weakened, LVM never diverged, which is consistent with Theorems 3.3 and 3.4.

To address these points, we made the following revisions:

  • Appendix I.3: We added the experiments at other noise levels (Figures 5 and 6).
  • Appendix I.4: We added the experiments with other blur kernels and masks (Figures 7 to 10).

4. On the comparison with other learned metrics (W5, Q2)

Learned scalar stepsizes. Following the suggestion, we added "Learned stepsizes" to the ablation study of PG, which learns one scalar stepsize for each iteration. The following table briefly summarizes the results (the mean number of iterations until Gn\mathcal{G}_ n first falls below each tolerance).

TaskMethod≤10−3\le10^{-3}≤10−4\le10^{-4}≤10−5\le10^{-5}≤10−6\le10^{-6}
DeblurringLVM3102962
DeblurringLearned stepsizes4114684
InpaintingLVM382967
InpaintingLearned stepsizes274896142

Learned stepsizes needs more iterations than LVM at every tolerance in both tasks. As for PG, we also compared LVM for PDHG with a variant that learns scalar stepsizes, and it needs more iterations than LVM at every tolerance (see Table 9 for details).

Learning by unrolling. The training of LVM is similar to unrolling in that it is truncated at the horizon TT. However, LVM differs from unrolling in that (i) the algorithm converges to a solution for any parameter of the generator, (ii) the learning does not alter the problem to be solved, and (iii) the metrics are defined for all the iterations, including those after TT. To clarify these points, we added this discussion to Section 3.3 (lines 365–371).

Comparison with non-diagonal metrics. We apologize that the intention of our comparison was not sufficiently conveyed. The experiments mainly investigate how LVM yields effective metrics for a given algorithm while keeping its updates efficiently computable. In this paper, we compare LVM with the methods that keep the proximity operators efficiently computable, and for this reason, we did not compare it with methods outside this scope, e.g., Tan et al. (2023). To convey this intention precisely, we clarified the scope of the experiments in Section 4.

To address these points, we made the following revisions:

  • Appendix I.5: We added Learned stepsizes and the variants without the variation over the coordinates to the ablation study (Tables 8 and 9).
  • Section 4 (lines 388–392): We clarified the scope of the compared methods.

5. On the scope of the method (W1)

As the reviewer correctly mentioned, the convergence guarantees of Theorems 3.3 and 3.4 rely on the convexity of ff and gg and do not directly cover weakly convex regularizers learned by neural networks.

To state the scope more clearly, we made the following revisions:

  • Section 3.2 (lines 344–346): We stated that the guarantees rely on the convexity of ff and gg, and do not directly cover weakly convex regularizers learned by neural networks.
  • Section 5 (lines 538–539): We added the extension to weakly convex or learned regularizers as future work.

We are grateful to the reviewer for the careful review and the comments, which improved the manuscript.

Thanks for the detailed responses and the additional experiments. I have two additional comments based on your responses.

  • The revised evaluation only briefly comments the dependence on the choice of the minimizer. Did you identify any nonunique instances in the experiments, and did you check whether the learned acceleration is sensitive to the choice of training reference in those cases ?

  • Table 8 answers whether full LVM beats a learned scalar schedule shared across images. It leaves open how much of the gain comes from the diagonal metric itself, because LVM also adapts to each image.

AuthorsOpenReview ↗

Response to Reviewer 2vFY

We appreciate the reviewer's continued engagement in the discussion. We address the two additional comments below.

1. On the uniqueness of the solutions and the reference solutions for the training

Uniqueness of the solutions. In deblurring, we confirmed that the solutions of both the problem for PG and the problem for PDHG are unique. Specifically, we numerically confirmed that the smallest singular value of the observation operator is positive for all the blur kernels and image sizes in our experiments (at least 3.0×10−43.0\times10^{-4} for the kernel in Section 4). In inpainting, the observation operator is not injective. For five test images, we computed two reference solutions, xref\boldsymbol{x}_ {\mathrm{ref}} in the same setting as in the training and xorc\boldsymbol{x}_ {\mathrm{orc}} by replacing only its initial point with the true image. For PG, ∥xref−xorc∥/∥xref∥\Vert\boldsymbol{x}_ {\mathrm{ref}}-\boldsymbol{x}_ {\mathrm{orc}}\Vert/\Vert\boldsymbol{x}_ {\mathrm{ref}}\Vert is 2.97×10−22.97\times10^{-2} on average and did not change with 3000030000 iterations, and the relative difference between their objective values is at most 3×10−123\times10^{-12}. For PDHG, it is 1.75×10−51.75\times10^{-5} on average, which is smaller than that for PG, but even this does not imply that the solution is unique.

Sensitivity to the reference solutions for the training. Based on the above, we compared ∥xn−xref∥/∥xn−xorc∥\Vert\boldsymbol{x}_ {n}-\boldsymbol{x}_ {\mathrm{ref}}\Vert/\Vert\boldsymbol{x}_ {n}-\boldsymbol{x}_ {\mathrm{orc}}\Vert for the iterates xn\boldsymbol{x}_ {n} of each method in inpainting. For PG (Table 11), the ratios of LVM are comparable to those of the other methods (except FISTA, which is the solver used to compute xref\boldsymbol{x}_ {\mathrm{ref}}), and we observe no evident sign that the iterates of LVM are particularly drawn to xref\boldsymbol{x}_ {\mathrm{ref}}. For PDHG, the same ratio for LVM is at most 1.0061.006. We did not train LVM with other reference solutions, and we left the effect of their choice on the acceleration as future work.

Based on these discussions, we made the following revisions:

  • Appendix I.7: We added this subsection, which contains the discussion on the uniqueness of the solutions and the comparison with the two reference solutions (Table 11).
  • Section 5: We added the sensitivity to the reference solutions as future work.

2. On the contribution of the diagonal metric (Table 8)

Following the comment, we added "−- spatial variation" to the ablation study of PG. This variant generates one scalar stepsize at each representative time for each image, and its metric does not vary over the coordinates. Note that it does not use the rescaling, since the rescaling for PG maps every scalar candidate to the same fixed stepsize. It needs more iterations than LVM at every tolerance in both tasks, as shown in the table below, which reports the mean number of iterations until Gn\mathcal{G}_n first falls below each tolerance.

TaskMethod≤10−3\le10^{-3}≤10−4\le10^{-4}≤10−5\le10^{-5}≤10−6\le10^{-6}
DeblurringLVM (ours)3102962
Deblurring−- spatial variation4214190
InpaintingLVM (ours)382967
Inpainting−- spatial variation264982>500>500

Since this variant still generates the stepsizes for each image, this result suggests that the variation over the coordinates contributes to the acceleration of LVM, although the variant also differs from LVM in the rescaling and eventually diverges in inpainting.

To address these points, we added "−- spatial variation" to Table 8 and to the discussion in Appendix I.5.

We hope that these additions resolve the remaining concerns of the reviewer.

Reviewer_LSGaOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The idea is well motivated and the theoretical construction is clean and correct. However, (i) the guarantee does not strictly cover the implemented algorithm, (ii) the practical advantage is limited: it appears mainly at very high accuracies, against weak non-inertial baselines, while non-learned FISTA remains competitive, and (iii) the experimental evaluation is narrow. In its current form, the evidence does not sufficiently support the claimed practical benefits.

Critical strengths

A clean, provably convergent construction for learned iteration-varying metrics. The key idea is to form each iteration’s metric as the inverse of a convex combination of inverse representative metrics. This reduces the conditions that variable metric theory imposes on an infinite metric sequence to conditions on only mm representative metrics, and a spectral rescaling then enforces them. The generator is evaluated once per instance, so no network evaluation is needed per iteration.

Critical weaknesses
  1. The claimed practical benefit is weakly supported by the experiments. Figures 2 and 3 show no evident advantage over most of the range. Untuned FISTA needs fewer iterations than LVM at 10−610^{-6} in both PG tasks (35 vs. 54, 70 vs. 95), and AdaPGM ties LVM in deblurring. The best variant, LVM+FISTA, is not covered by the theory. The offline training cost (4.8–37.8 GPU hours) is not included in the cost analysis.
  2. The guarantee does not strictly cover the implemented algorithm. Theorems 3.3/3.4 require the exact βj\beta_j. The implementation instead uses 1.05β^j1.05\hat\beta_j, where β^j\hat\beta_j is obtained from 20–50 power-iteration steps, which underestimate the largest eigenvalue. The assumptions are verified only a posteriori on Set10 (Table 5), and the largest value is ξj=0.981\xi_j = 0.981, a margin below 2%. For unseen instances, there is no a priori guarantee. This matters because “convergence for any generator output” is the paper’s selling point.
Questions for authors
  1. At which accuracy does LVM become faster in wall-clock time than the best baseline in each setting, in particular FISTA and AdaPGM? Is that accuracy relevant for image restoration, given that PSNR saturates much earlier?
  2. Can the rescaling use a certified upper bound on βj\beta_j (e.g., ∥A∥2max⁡i(P~(j))ii\|A\|^2\max_i(\tilde P_{(j)})_{ii}), or include a cheap online safeguard, so that the guarantee holds for unseen instances without the Lanczos check? If the Lanczos check is part of the pipeline, what does it cost?
  3. How were the hyperparameters in Table 4 (cc, TT, T\mathcal T, ωt\omega_t, training-set size) selected, and was a validation set separate from Set10 used? Can you report standard deviations for Tables 2–3?
  4. In Table 2, GnG_n is used before its definition, and F⋆F^\star is defined as “the smallest value observed”. Is this taken across all methods?
4 comments
AuthorsOpenReview ↗

Response to Reviewer LSGa (Part 1/2)

We sincerely thank the reviewer for the careful review and for the concrete suggestions. Our responses are given below.

1. On the convergence guarantee of the implemented algorithm (W2, Q2)

We agree that the guarantee should cover the implemented algorithm, and we revised the implementation and the experiments following the suggestion of the reviewer. The revised implementation computes an upper bound on βj\beta_j before the iterations and does not use the power iteration when solving the test instances. For PG in inpainting, the upper bound is the one suggested by the reviewer, ∥A∥2max⁡i(P~⟨j⟩)ii\Vert A\Vert^{2}\max_i (\widetilde{P}_ {\langle j\rangle})_ {ii}. For PG in deblurring, it is derived from the block circulant structure of A∗AA^{\ast}A, and for PDHG, it is the Collatz–Wielandt bound on the spectral radius.

Thanks to this modification, the guarantee holds a priori for unseen instances, and the Lanczos check is no longer part of the pipeline. This revision not only gives mathematical upper bounds but also shortens the setup of LVM, as stated in the next point. The cost of the upper bounds for each image is included in the reported computation time as a part of the setup. For a fair comparison, we also revised the compared methods whose setup used the power iteration to use these upper bounds.

To address these points, we replaced the results in Section 4 by those of the revised implementation and added the details of the new upper bounds to Appendix H.

2. On the practical benefit (W1, Q1)

Following the revision of the upper bounds above, we updated Tables 2 and 3 and Figures 2 and 3. Please note that all the results below are those after the setup with the power iteration was replaced by the new one.

Comparison with AdaPGM. LVM is faster than AdaPGM at every tolerance in both tasks. For example, at Gn≤10−5\mathcal{G}_n\le10^{-5}, LVM takes 55 ms and AdaPGM 70 ms in deblurring, and 31 ms and 101 ms in inpainting. In deblurring, the two methods need almost the same numbers of iterations, but AdaPGM takes longer in both the setup and each iteration.

Comparison with FISTA. FISTA is faster than LVM at every tolerance in deblurring. In inpainting, LVM is faster down to Gn≤10−5\mathcal{G}_n\le10^{-5} (31 ms and 47 ms at this tolerance), and FISTA is faster at 10−610^{-6}. As the reviewer noted, the PSNR settles early, and in inpainting it has already settled at 10−510^{-5}. Let us clarify that the experiments mainly investigate how LVM yields effective metrics for a given algorithm while keeping its updates efficiently computable. FISTA accelerates the iterations by an inertial term and not by the metric, and this term can be combined with changes of the metric, including LVM. We agree that our theorems do not cover LVM + FISTA. We added the convergence guarantee of this combination to the conclusion as future work.

PDHG. In both tasks, LVM reaches every tolerance in the fewest iterations, and it reaches every tolerance except Gn≤10−1\mathcal{G}_n\le10^{-1} in the least computation time.

Cost of the training. Following the comment, we revised the main text to state the time of the training, which was reported in the appendix.

To address these points, we made the following revisions:

  • Section 4: We clarified the aim of the experiments, added the standard deviations and the computation time including the setup to Tables 2 and 3, updated Figures 2 and 3, and revised the discussion according to the new results.
  • Section 3.3 (lines 371–373): We stated the time of the training and the number of instances after which the saved time exceeds it.
AuthorsOpenReview ↗

Response to Reviewer LSGa (Part 2/2)

3. On the standard deviations and the selection of the hyperparameters (Q3)

Standard deviations. Following the comment, we reported the standard deviations over the 68 test images in Tables 2 and 3. For both PG and PDHG, the standard deviations of LVM are at the same level as those of the compared methods without an inertial term.

Selection of the hyperparameters. We tuned the settings of the training manually without using the test images, and we did not use a separate validation set. The horizon TT and the number of training instances were chosen from the accuracy reached on the training instances and from the memory and the time of the training. The checkpoints in T\mathcal{T} roughly double up to TT. The constant cc was set through the normalized time sT=T/(T+c)s_T=T/(T+c) at the end of the training run, and the weights ωt\omega_t for PG emphasize the late checkpoints.

To address these points, we made the following revisions:

  • Section 4: Following the comment, we added the standard deviations to Tables 2 and 3.
  • Appendix G.4: For readers who train LVM for their own problems, we added the above description of the selection of the hyperparameters.

4. On the definitions of Gn\mathcal{G}_n and F⋆F^{\star} (Q4)

Following the comment, we defined Gn\mathcal{G}_n before Table 2. In addition, F⋆F^{\star} is taken across all the methods. It is the minimum of FF over the reference solution and the last iterates of all the methods. We stated this in Section 4.1 (lines 400–402).

We are grateful to the reviewer for the careful review and the comments, which improved the manuscript.

The authors resolved my concerns on practical benefits, training cost, and the gap between the theoretical convergence. Though the authors failed addressing the applicability to more recent image restoration approaches given by the other reviewer, I would like to increase my rating to WA as the paper presents a promising approach to learning iteration-varying metrics that guarantee convergence of proximal splitting iterations.

AuthorsOpenReview ↗

Response to Reviewer LSGa

Thank you very much for taking the time to read our responses and for raising your rating. On the applicability to more recent image restoration approaches, following the comment of Reviewer 2vFY, it is stated in the conclusion of the revised manuscript as future work.

Reviewer_crTfOpenReview ↗
Rating
4
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

A new framework is proposed for accelerating proximal splitting algorithms: the Learned Variable Metric (LVM) framework. It is aimed at minimizing the sum of two convex functions in a Hilbert space.

The paper defines a problem-specific, coordinate-wise and iteration-varying scheme without extra evaluations. In Theorem 2.1, a convergence result in the LVM framework is provided for proximal gradient (PG) applied to the quadratic case. In Theorem 2.2, a convergence result in the LVM framework is provided for primal-dual hybrid gradient (PDHG) methods for the general case.

The LVM scheme is coordinate-wise and iteration-varying as it generates a sequence of metrics that are diagonal for each iteration. Each metric of the sequence is a combination of what is called "representative'' metrics that were precomputed by a convolutional network. Thus it does not necessitate extra evaluations and the sequence can be problem dependent according to the training set of the convolutional network.

Experiments have been conducted on the Flickr2K and Set10 datasets to demonstrate the LVM acceleration of PG and PDHG on image deblurring and inpainting tasks.

Critical strengths

LVM is the first framework without extra evaluations being at the same time problem-specific, coordinate-wise and iteration-varying.

The proposed method is independent of the generation of the representative metrics which isolates its problem-specificity from its other strengths.

Critical weaknesses

The influence of the training of the metric generator was not explored enough. It is not clear how problem specific the method can be.

The paper is also lacking clear statements of the PG and PDHG algorithms for readers less familiar with proximal splitting methods.

Questions for authors
  • How are solutions of (5) and (1) related? A discussion is needed to compare the convergence results stated in Theorem 3.3 and Theorem 3.4. Indeed Theorem 3.3 looks more restricted as it only applies to a quadratic first term.

  • The training of the metric generator looks like an unrolling approach. Indeed, you train your generator on a truncated run of a proximal splitting algorithm. If I have understood correctly, the problem-specific nature comes from the training of the generator. So does the problem-specifity suffers from this truncation? As you pointed out in the introduction, unrolling lacks asymptotic properties.

  • Do you have experiments backing the problem-specifity of your method? How would you measure it?

  • PG and PDHG algorithms should be recalled in the preliminaries.

  • Please define PSNR in Figure 2 and 3.

  • For the experimental details, a table in appendix G1 which summarizes the differences between deblurring and inpainting would be a good addition.

Minor suggestions.

  • A definition of ``metric'' is missing in the Preliminaries.
  • In line 231, define the softplus function.
  • Line 250, only two representative metrics are computed because of the definition of ψj\psi_j and not because of the diagonal nature of the representative metrics.
2 comments
AuthorsOpenReview ↗

Response to Reviewer crTf

We sincerely thank the reviewer for the positive evaluation and for the constructive feedback. Our responses are given below.

1. On the problem specificity of the learned metrics

Let us first clarify that, in our experimental setting, problem specificity means the adaptation to each observed image. To measure this adaptation, we investigated the decrease of the optimality measure Gn\mathcal{G}_n when the algorithm runs with the metrics generated from the observation of another image and when it runs with the metrics generated for the image at hand. We briefly summarize this additional experiment in the following table.

AlgorithmTasknnLVMMetrics of another image
PGDeblurring2002.1×10−92.1\times10^{-9}3.5×10−93.5\times10^{-9}
PGInpainting1002.8×10−72.8\times10^{-7}3.7×10−73.7\times10^{-7}
PDHGDeblurring5008.4×10−68.4\times10^{-6}9.6×10−59.6\times10^{-5}
PDHGInpainting5002.7×10−62.7\times10^{-6}8.9×10−58.9\times10^{-5}

The values are the mean of Gn\mathcal{G}_n over five images. In every setting, the performance degraded when the metrics generated for a different image were used, and this tendency was more pronounced for PDHG.

Effect of the truncation. As the reviewer correctly mentioned, the training of LVM is similar to unrolling in that it is truncated at the horizon TT. However, LVM differs from unrolling. The construction of LVM defines the metrics for all nn, including n>Tn>T, and PnP_n tends to the last representative metric, which is generated for each instance. Hence, the metrics remain adapted to the instance in the iterations after the horizon TT (for PDHG in the table above, the horizon is T=250T=250, and Gn\mathcal{G}_n of LVM is still smaller than that of the metrics of another image at n=500n=500). In addition, the convergence of unrolling is not guaranteed in general, whereas LVM converges to a solution for any parameter of the generator (Theorems 3.3 and 3.4).

To address these points, we made the following revisions:

  • Appendix I.6: We added this subsection and Table 10 to report more detailed results of the above experiment.
  • Section 4 (lines 386–387): We clarified that the metrics adapt to the observed image of each instance.
  • Section 3.3 (lines 365–371): We stated the relation of the training to unrolling.

2. On the relation between (1) and (6) (formerly (5)) and between Theorems 3.3 and 3.4

Under a mild qualification condition, x^\hat{\boldsymbol{x}} solves (1) if and only if (x^,y^)(\hat{\boldsymbol{x}},\hat{\boldsymbol{y}}) solves (6) for some y^\hat{\boldsymbol{y}}, which then solves the dual problem of (1) [Condat 2013, Section 2].

As the reviewer pointed out, Theorem 3.3 is restricted to L=IdL=\mathrm{Id} and a quadratic ff, and Theorem 3.4 allows any LL and any proximable ff and gg. However, let us clarify that Theorem 3.4 does not contain Theorem 3.3. PDHG has no gradient step, and the proximity operator of a quadratic ff generally requires solving a linear system, which is inefficient [Condat et al. 2023, Section 2.1]. Hence, PG remains the suitable algorithm for a quadratic ff, and Theorem 3.3 covers this case.

To address these points, we made the following revisions:

  • Section 2.1 (lines 200–201): We added the relation between the solutions of (1) and (6).
  • Section 3.2 (lines 334–337): We added the relation between Theorems 3.3 and 3.4.

3. On the presentation

We revised the manuscript according to all of the remaining comments:

  • Section 2 (line 161): We added the definition of a metric.
  • Section 2.1 (lines 175–184 and 201–213): We stated the updates of PG and PDHG with fixed stepsizes before their variable metric extensions.
  • Section 3 (line 244): We defined the softplus function.
  • Section 3.1 (lines 265–266): We corrected the reason why only two representative metrics are needed at each iteration, which is the definition of the basis functions, as the reviewer pointed out.
  • Section 4 (lines 392–393): We defined PSNR.
  • Appendix G.1: We added Table 4, which lists the settings of deblurring and inpainting.

We are grateful to the reviewer for the careful review and the comments, which improved the manuscript.

References

[Condat 2013] L. Condat, "A primal-dual splitting method for convex optimization involving Lipschitzian, proximable and linear composite terms," J. Optim. Theory Appl., vol. 158, no. 2, pp. 460–479, 2013.

[Condat et al. 2023] L. Condat, D. Kitahara, A. Contreras, and A. Hirabayashi, "Proximal splitting algorithms for convex optimization: A tour of recent advances, with new twists," SIAM Rev., vol. 65, no. 2, pp. 375–435, 2023.

I thank the authors for their clear answers and thorough explanations. I keep my rating of 4.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to All Reviewers

We sincerely thank all the reviewers for the careful reviews and the constructive feedback. We revised the manuscript according to the comments, and all the changes are marked in blue. The numbers of sections, tables, figures, and equations in our responses refer to the revised manuscript. The main revisions are as follows:

1. Convergence guarantee of the implemented algorithm. We updated the spectral rescaling, which used the power iteration in the original submission, to use upper bounds on βj\beta_j computed before the iterations. Hence, the guarantee holds a priori for unseen instances (Section 3.2 and Appendix H). Thanks to this new rescaling, the setup time of LVM is also reduced.

2. Evaluation. For the experiments in Section 4, we increased the number of test images from 10 (Set10) to 68 (BSD68). Tables 2 and 3 report the standard deviations and the computation time including the setup of LVM, and Figures 2 and 3 are updated. For PDHG, we replaced the distance to the reference solution by a measure independent of the choice of the solution (Section 4.2 and Appendix G.5). We also added the results for other realizations of the noise (Appendix I.2).

3. Additional experiments. We added the transfer to other noise levels and observation operators without retraining (Appendices I.3 and I.4), an ablation study including learned scalar stepsizes (Appendix I.5), and an experiment on the adaptation of the metrics to each instance (Appendix I.6).

4. Clarifications. We stated the scope of the convergence guarantees and of the compared methods (Sections 3.2, 4, and 5), the relation of the training to unrolling (Section 3.3), and the updates of PG and PDHG (Section 2.1).

We hope that these revisions address your concerns.

AuthorsOpenReview ↗

Update on the Second Revision

We thank all the reviewers for the continued discussion. We uploaded a second revision of the manuscript, in which the changes made after the first revision are marked in blue and the changes of the first revision are now shown in black. The revisions are as follows:

  • Appendix I.7: We added a discussion on the uniqueness of the solutions and a comparison with two reference solutions computed from different initial points (Table 11).
  • Appendix I.5: We added a variant of PG with scalar stepsizes generated for each image to the ablation study (Table 8).
  • Section 5: We added the sensitivity to the reference solutions as future work.

The numbers of sections, tables, figures, and equations are unchanged from the first revision, except for the added Appendix I.7 and Table 11.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.