acceptodds
Under review as a conference paper at ICLR 2027

Operator-Adaptive Mean-Target Guidance for Linear Inverse Problems

Abstract

Training-free samplers for inverse problems construct guidance from a residual evaluated at an evaluation target, a point along the generative trajectory. Since the measurement is defined from the clean image through a known operator, most existing methods form the residual at a clean-image estimate. Some methods instead map the measurement to the current sampling step and form the residual at the current state. Although existing methods construct guidance in different ways, they typically treat the evaluation target as given rather than derive it. We introduce Mean-Target Guidance (MTG), which derives operator-adaptive evaluation targets. A single frozen model evaluation produces a continuum of virtual targets. The gradient of a loss that aggregates their weighted squared residuals reduces exactly to one residual at their weighted mean, the mean-target. Inverse-variance weighting over the virtual targets yields a mean-target for each singular direction. While the mean-target determines where the residual is evaluated, a direction-specific damping factor controls how strongly that residual updates the current state. The construction applies to both diffusion and flow-matching backbones. MTG outperforms strong training-free baselines across standard linear inverse problems. An ablation study separates the effects of the mean-target and damping factor. We train the evaluation targets and update strengths, highlighting the importance of where the residual is evaluated and how strongly it acts on the state. Code is available at https://anonymous.4open.science/r/MTG2027.

Primary area:
generative models
Keywords:
Diffusion Models, Flow Matching, Linear Inverse Problems

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_kKXPOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes Mean-Target Guidance (MTG), a training-free guidance rule for linear inverse problems. Rather than forming the measurement residual at the clean-image estimate or at the current state, it considers residuals at a continuum of "virtual targets" along the sampling path, weights them, and shows that the gradient reduces to a single residual at their weighted mean (the "mean-target"). With inverse-variance weights, each singular direction of the operator gets its own target and damping factor, both precomputed. MTG works with diffusion (EDM) and flow-matching models and outperforms strong training-free baselines on standard linear tasks.

Critical strengths
  1. The paper is well written and easy to follow. The method appears cheap (no backpropagation through the network, with per-direction quantities precomputed) and applies to both diffusion and flow-matching backbones, which would make it practically useful.
  2. MTG outperforms strong training-free baselines on most tasks and datasets, often by clear margins.
Critical weaknesses

W1. The proposed mechanism does not clearly drive the gains.

  • The appendix notes that on the benchmark operators, the direction-specific target and damping are either constant across directions (masks share a single singular value) or saturated over most of the spectrum (blur/SR spectra lie mostly far below the noise level). The per-direction adaptivity is therefore mainly exercised on the synthetic operator, and the benchmark gains cannot be clearly attributed to it.
  • In the ablation on the EDM backbone, the mean-target adds little over a fixed current-state target once the derived damping is used, whereas removing the damping causes a large drop. The damping itself resembles standard per-singular-value (Wiener-type) filter factors, as in DDRM and ΠGDM, and this connection is not discussed.
  • The support (TlfT_{lf} vs. TfullT_{full}) directly determines the target, but its choice is not motivated and varies across tasks and ablation rows.

W2. The "derived" target is largely a modeling choice.

  • The main theorem follows from the residual being affine in τ\tau, so the target is more so determined by the chosen weighting rather than by the theorem itself.
  • The inverse-variance weighting relies on simplifying assumptions, and its form differs between the EDM and OT backbones. Alternative continuous weightings are not explored, and directly trained targets and step sizes (TTG) outperform the derived pair (MTG). Together, this suggests the derivation is more heuristic rather than a principled choice.
Questions for authors
  1. What set of images were used to tune the hyperparameters? Please also report runtime compared with the baselines.
  2. Results with both supports for all tasks, with some guidance on when to use each.
  3. A comparison with alternative weightings (e.g., uniform, or using the mode instead of the mean).
10 comments
AuthorsOpenReview ↗

Response to Reviewer kKXP (1/8)

We sincerely thank the reviewer for the thoughtful review, and for noting the clarity of the paper and the practicality of the method. Below we address each point. New results report PSNR ↑ / SSIM ↑ / LPIPS ↓ on 100 test images, and the tables are collected at the end of this response.

W1 (first point) · Direction-specific adaptivity on the benchmarks

Operator adaptivity does not come from the per-direction form. Our weight is the inverse of the residual covariance C(τ)=(1−τ)2AA⊤+τ2σy2IC(\tau)=(1-\tau)^2AA^\top+\tau^2\sigma_y^2 I (Eq. 3.4), which already contains the operator. The weight, the mean-target and the damping therefore change with the operator even without the diagonalization from Eq. 3.4 to Eq. 3.5. Our new ablation confirms this: when the spectrum is summarized by a single average σ2‾=∥A∥F2/m\overline{\sigma^2}=\|A\|_F^2/m, the target and the damping still change from operator to operator.
What the diagonalization adds is that each measured direction receives its own target and its own update strength, which is possible because the target is a variable. One rule can then cover operators whose directions are measured very differently. When the directions are measured alike, as with masks, the per-direction rule reduces to the shared one at no cost, and when they differ strongly, the per-direction rule is what keeps the sampler accurate (Table R1).

On the benchmarks, operator adaptivity shows first as a difference between operators. Each operator receives its own pair of curves over the sampling steps, one for the mean-target and one for the damping factor (Appendix F.1.3, Figs. 6–8, top and bottom rows), and masks, super-resolution and the blurs each receive a different pair. We kindly ask the reviewer to compare these curves across operators.
Within one benchmark operator, the mean-targets differ little, and this follows from the benchmark operators themselves. Only 8 of the 20 benchmark settings have more than one singular value.

  • Masks, denoising and OT super-resolution measure every direction with one singular value, so all directions share one target by construction.
  • On EDM super-resolution, the whole spectrum lies above the noise level, and the targets coincide (noise levels 2.88–3.01 when the current noise level is σ=81\sigma=81).
  • On the EDM blurs, most directions lie below the noise level. Under look-forward support, the targets of the measured directions stay in a narrow band (noise levels 2.84–3.70 at σ=81\sigma=81).
  • OT Gaussian deblurring, run with full support, is the exception. Its targets range from τˉ=0.50\bar\tau=0.50 to 0.900.90, so in mid-sampling they lie on both sides of the current state.

The damping factor, by contrast, differs strongly across directions on every benchmark operator with a spectrum. This is where per-direction processing acts most visibly on the benchmarks, and we would particularly ask the reviewer to look at the bottom rows of Figs. 6–8.

  • EDM super-resolution: the targets coincide, yet the damping spans 13.813.8 to 40.240.2, a factor of 2.92.9, for most of the run.
  • EDM Gaussian and motion blur: the damping spans a factor of 3.63.6 and 3.03.0 at σ=81\sigma=81, and a factor of more than 120120 at σ=0.025\sigma=0.025.
  • OT Gaussian deblurring: the damping spans a factor of 135135 at t=0.51t=0.51.

The target and the damping come from the same per-direction variance Cs(τ)C_s(\tau). Where a benchmark operator leaves the targets close together, the same construction still gives each direction its own damping.
The spread of the targets also depends on the support. Look-forward support, the usual choice on EDM, confines every mean-target between σmin⁡\sigma_{\min} and the current noise level, so the spread in noise level is bounded by the current noise level and shrinks as sampling proceeds. With full support, targets can fall on both sides of the current state. This happens in OT Gaussian deblurring above and in the OT setting of §4.2, where the targets span τˉ=0.41\bar\tau=0.41 to 0.880.88 at t=0.51t=0.51.

AuthorsOpenReview ↗

Response to Reviewer kKXP (2/8)

W1 (first point) · Direction-specific adaptivity on the benchmarks (continued)

§4.2 separates the two derived components, the mean-target and the damping factor, by replacing each with fixed alternatives, with mixing and κ\kappa turned off (m=κ=0m=\kappa=0). It is not an ablation of per-direction processing. The two components can be separated only where both vary across directions, which requires singular values spread across the noise level with no direction falling far below it. Each benchmark setting misses one of these conditions: it has a single singular value, keeps its whole spectrum above the noise level, or places a large share of its directions far below it. We therefore constructed the spatial exposure operator for §4.2. We acknowledge that the manuscript did not convey this intent clearly, and that per-direction processing itself was not ablated in the submission. To avoid this confusion, we have also revised the main text. §3.2 now states that the mean-target differs between operators and, within one operator, across directions with different singular values. §4.1 refers to the direction-sharing ablation, and the opening of §4.2 states that the experiment measures what each component contributes rather than testing per-direction processing.

We have now run that ablation (Table R1) and added it to the revision in Appendix F.1.5, with the §4.2 rows in Appendix F.2.3. The target and the damping both come from the same per-direction variance Cs(τ)C_s(\tau), so turning per-direction processing off means sharing both. We replace each σs2\sigma_s^2 by its average in both rules, keep the actual AA and A⊤A^\top in the update, and retune cc. Sharing lowers all three metrics in all eight benchmark settings that have a spectrum (on average −0.66-0.66 dB / −0.017-0.017 / +0.024+0.024), and the loss grows with how differently the directions are measured.

OperatorDirectionsPSNR change when shared
Box, Rand, Dn, OT SROne singular value0 (identical by construction)
EDM SRSpectrum entirely above the noise level−0.09-0.09 / −0.09-0.09 (FFHQ / ImageNet)
EDM GB, EDM MB, OT GDSpans the noise level, mostly below it−0.30-0.30 to −1.24-1.24
§4.2 graded operatorSpans the noise level−8.31-8.31 (EDM) / −6.71-6.71 (OT)

The benchmark blurs are realistic operators whose spectrum crosses the noise level, with singular values both above and below it, and sharing lowers all three metrics on each of them. Sharing matters even where the targets coincide: on EDM super-resolution only the damping differs across directions, yet sharing lowers all three metrics on both datasets.
Sharing fails because it summarizes the spectrum by its average energy. On EDM Gaussian blur, the shared target stays inside the per-direction range (noise level 3.16 against 2.84–3.70 at σ=81\sigma=81), so sharing mainly changes the damping, and one value replaces the whole range: 2.632.63 against 0.730.73–2.642.64 at σ=81\sigma=81, and 0.0850.085 against 0.00120.0012–0.1520.152 at σ=0.025\sigma=0.025. The strongly measured directions then take much larger steps than their own damping sets, and one global cc cannot correct this without also shrinking the steps of the weakly measured directions. On OT Gaussian deblurring, sharing collapses both components: at t=0.51t=0.51, the shared target is τˉ=0.61\bar\tau=0.61 against 0.500.50–0.900.90, and the shared damping is 6161 against 0.980.98–133133.

W1 (second point) · EDM ablation and Wiener-type filters

We agree that the mean-target adds little over the current-state target once derived damping is used on EDM in §4.2. On that operator, EDM's directions differ mainly in the damping, which both rows share. With the damping removed (g=1g=1, m=0m=0), the mean-target improves all three metrics on both backbones: 21.83 / 0.641 / 0.296 vs. 20.34 / 0.374 / 0.446 on EDM, and 22.14 / 0.699 / 0.196 vs. 18.78 / 0.458 / 0.400 on OT (submitted Table 2). The EDM mean-target rows with and without damping were selected with different supports, so their difference also includes a support change.

The similarity to spectral filters is real. The damping factor gs=(σs2+γs)−1g_s=(\sigma_s^2+\gamma_s)^{-1} has the Wiener/Tikhonov form and shares its spectral principle with ΠGDM and DDRM, and the revision now discusses this connection in Appendices C.4 and D.3. The coefficient γs\gamma_s is a modeling choice made for a specific goal (Appendix C.4). We wanted the damping factor to depend on both the mean-target and the current sampling time, and adopting ΠGDM's weighting directly would not place the mean-target inside the factor. We therefore defined the step as the minimizer of a regularized displacement (Eq. 3.9) and set γs\gamma_s to the ratio of the measurement-noise variance at the mean-target to the conditional variance at the current step. The resulting form coincides with these filters.

AuthorsOpenReview ↗

Response to Reviewer kKXP (3/8)

W1 (third point) and Q2 · Choice of support

We ran all 20 settings under both supports (Table R2; Appendix F.1.5 of the revision), keeping the other hyperparameters fixed and selecting cc again for the alternate support. On EDM, switching SR, Gaussian blur, motion blur and random inpainting from look-forward to full support degrades all three metrics even after retuning, by 1.77–12.93 dB PSNR. For weakly observed directions, the weight is nearly flat, so the mean-target moves to the midpoint of the support. On EDM's full support, [0.01,100][0.01,100], this is a noise level of about 50 regardless of the current step, which late in sampling extrapolates far toward the noisy side, whereas the look-forward midpoint follows the current noise level. EDM box inpainting is insensitive in PSNR (within 0.14 dB), while SSIM clearly favors the submitted full support (0.872 vs. 0.781 on FFHQ, 0.773 vs. 0.716 on ImageNet). On OT, the alternate support stays within −0.77-0.77 to +0.17+0.17 dB after retuning, but perceptual metrics can still differ: on AFHQ-Cat box inpainting, it gains 0.17 dB while LPIPS worsens from 0.055 to 0.072. These results show that the support is not an incidental detail.
The integration interval must be fixed to define the objective, and the space of all intervals cannot be searched for an optimum. We therefore use one interpretable criterion, whether to include the noisier side of the current state: the look-forward support excludes it and the full support includes it. We treat this choice like other hyperparameters, selected per operator on validation, rather than as a fixed prescription, and other intervals, such as a window of fixed width around the current time, are equally possible. In practice, we recommend look-forward support on EDM (box inpainting can use either) and either support on OT. That we cannot identify an optimal support is a limitation, which the revision states in Appendix F.1.5.

Reviewer_PaFxOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper studies where the residual y−Axy-Ax should be computed when guiding a training-free sampler for linear inverse problems. Existing methods use either the clean-image estimate or the current state, and the authors refer to this choice as the evaluation target. By freezing the model output, the state can be extrapolated along a line to any time (τ)(\tau), allowing the residual to be computed at each τ\tau. The authors show that the gradient of an inverse-variance weighted loss over these residuals reduces to that of a single residual evaluated at the weighted mean of τ\tau, termed mean-target. Since the variance depends on the singular values of AA, each singular direction has its own mean-target and damping factor. Both quantities are precomputed before sampling and apply to diffusion and flow-matching backbones. Experiments cover five tasks across four datasets, a synthetic diagonal operator for ablation studies, and a variant with targets learned using CMA-ES.

Critical strengths
  • The paper unifies data-endpoint and current-state residuals along a single line parameterized by τ\tau. Existing approaches such as DPS-style and FIG-style guidance then appear as two endpoints of one family. The resulting update requires only one model evaluation per step, without backpropagation through the network. All per-direction quantities depend solely on the schedule and the spectrum, allowing them to be precomputed before sampling. The same construction carries over to diffusion and flow-matching with minor changes. This perspective provides a clear dimension along which future guidance schemes can be compared.
  • MTG performs well across two backbones, four datasets, and five tasks. Experiments with a graded diagonal operator show that both the mean-target and the damping factor improve reconstruction. The gain is largest in weakly measured directions, where the other targets degrade quickly.
Critical weaknesses
  • The central claim of operator adaptivity is largely inactive on the benchmarks. Appendix E.2 states that none of the ten benchmark settings allows the mean-target to vary meaningfully across directions. Since inpainting and denoising involve only a single singular value, each step uses a single target and dampling value. For deblurring and super-resolution, most singular values lie far below the noise level, which causes both quantities to saturate. Therefore, a synthetic operator was constructed to demonstrate any directional effect. This implies that the benchmark improvements cannot be attributed to the operator-adaptive feature highlighted in the manuscript. Experiments with a realistic operator whose spectrum crosses the noise level would help address this concern.
  • Since the extrapolated residual is affine in τ\tau, the residual at the mean-target is simply a linear combination of the current-state and date-endpoint residuals. Therefore, the continuum of virtual targets reduces to selecting a single mixing coefficient per direction. The update magnitude implied by the loss is then discarded and replaced with a separately designed dampling factor. This factor has a Wiener-type form similar to the spectral weighting used in Π\PiGDM and DDRM. The variance model also differs between backbones, as the OT assumption would collapse the EDM weight to one endpoint. Furthermore, residuals at different values of τ\tau are correlated but weighted independently, while γs\gamma_s and κ\kappa introduced as modeling choices.
  • Several components beyond the derivation contribute to the final sampler. A DDNM-style null-space mixing step is introduced, with coefficients as high as 1.0. However, its effect is not ablated on the benchmarks. The support, guidance scale, mixing coefficient, and κ\kappa are grid-searched separately for each task. The data split used for the grid search is not specified, and runtime is not reported. Thus, it remains unclear how much of the observed improvement can be attributed to the mean-target itself.
Questions for authors
  • What are the Table 1 results with null-space mixing disabled and all other settings unchanged?
  • On the benchmarks, how does MTG compare with a simpler variant that blends the two residuals using one tuned scalar per step, shared across directions?
9 comments
AuthorsOpenReview ↗

Response to Reviewer PaFx (1/6)

We sincerely thank the reviewer for the detailed and constructive review, and for recognizing the unified view of evaluation targets and the results on the graded operator. Below we address each point. New results report PSNR ↑ / SSIM ↑ / LPIPS ↓ on 100 test images, and the tables are collected at the end of this response.

W1 and Q2 · Operator adaptivity on the benchmarks

Operator adaptivity does not come from the per-direction form. Our weight is the inverse of the residual covariance C(τ)=(1−τ)2AA⊤+τ2σy2IC(\tau)=(1-\tau)^2AA^\top+\tau^2\sigma_y^2 I (Eq. 3.4), which already contains the operator. The weight, the mean-target and the damping therefore change with the operator even without the diagonalization from Eq. 3.4 to Eq. 3.5. Our new ablation confirms this: when the spectrum is summarized by a single average σ2‾=∥A∥F2/m\overline{\sigma^2}=\|A\|_F^2/m, the target and the damping still change from operator to operator.
What the diagonalization adds is that each measured direction receives its own target and its own update strength, which is possible because the target is a variable. One rule can then cover operators whose directions are measured very differently. When the directions are measured alike, as with masks, the per-direction rule reduces to the shared one at no cost, and when they differ strongly, the per-direction rule is what keeps the sampler accurate (Table R1).

On the benchmarks, operator adaptivity shows first as a difference between operators. Each operator receives its own pair of curves over the sampling steps, one for the mean-target and one for the damping factor (Appendix F.1.3, Figs. 6–8, top and bottom rows), and masks, super-resolution and the blurs each receive a different pair. We kindly ask the reviewer to compare these curves across operators.
Within one benchmark operator, the mean-targets differ little, and this follows from the benchmark operators themselves. Only 8 of the 20 benchmark settings have more than one singular value.

  • Masks, denoising and OT super-resolution measure every direction with one singular value, so all directions share one target by construction.
  • On EDM super-resolution, the whole spectrum lies above the noise level, and the targets coincide (noise levels 2.88–3.01 when the current noise level is σ=81\sigma=81).
  • On the EDM blurs, most directions lie below the noise level. Under look-forward support, the targets of the measured directions stay in a narrow band (noise levels 2.84–3.70 at σ=81\sigma=81).
  • OT Gaussian deblurring, run with full support, is the exception. Its targets range from τˉ=0.50\bar\tau=0.50 to 0.900.90, so in mid-sampling they lie on both sides of the current state.

The damping factor, by contrast, differs strongly across directions on every benchmark operator with a spectrum. This is where per-direction processing acts most visibly on the benchmarks, and we would particularly ask the reviewer to look at the bottom rows of Figs. 6–8.

  • EDM super-resolution: the targets coincide, yet the damping spans 13.813.8 to 40.240.2, a factor of 2.92.9, for most of the run.
  • EDM Gaussian and motion blur: the damping spans a factor of 3.63.6 and 3.03.0 at σ=81\sigma=81, and a factor of more than 120120 at σ=0.025\sigma=0.025.
  • OT Gaussian deblurring: the damping spans a factor of 135135 at t=0.51t=0.51.

The target and the damping come from the same per-direction variance Cs(τ)C_s(\tau). Where a benchmark operator leaves the targets close together, the same construction still gives each direction its own damping.
The spread of the targets also depends on the support. Look-forward support, the usual choice on EDM, confines every mean-target between σmin⁡\sigma_{\min} and the current noise level, so the spread in noise level is bounded by the current noise level and shrinks as sampling proceeds. With full support, targets can fall on both sides of the current state. This happens in OT Gaussian deblurring above and in the OT setting of §4.2, where the targets span τˉ=0.41\bar\tau=0.41 to 0.880.88 at t=0.51t=0.51.

AuthorsOpenReview ↗

Response to Reviewer PaFx (2/6)

W1 and Q2 · Operator adaptivity on the benchmarks (continued)

§4.2 separates the two derived components, the mean-target and the damping factor, by replacing each with fixed alternatives, with mixing and κ\kappa turned off (m=κ=0m=\kappa=0). It is not an ablation of per-direction processing. The two components can be separated only where both vary across directions, which requires singular values spread across the noise level with no direction falling far below it. Each benchmark setting misses one of these conditions: it has a single singular value, keeps its whole spectrum above the noise level, or places a large share of its directions far below it. We therefore constructed the spatial exposure operator for §4.2. We acknowledge that the manuscript did not convey this intent clearly, and that per-direction processing itself was not ablated in the submission. To avoid this confusion, we have also revised the main text. §3.2 now states that the mean-target differs between operators and, within one operator, across directions with different singular values. §4.1 refers to the direction-sharing ablation, and the opening of §4.2 states that the experiment measures what each component contributes rather than testing per-direction processing.

We have now run that ablation (Table R1) and added it to the revision in Appendix F.1.5, with the §4.2 rows in Appendix F.2.3. The target and the damping both come from the same per-direction variance Cs(τ)C_s(\tau), so turning per-direction processing off means sharing both. We replace each σs2\sigma_s^2 by its average in both rules, keep the actual AA and A⊤A^\top in the update, and retune cc. Sharing lowers all three metrics in all eight benchmark settings that have a spectrum (on average −0.66-0.66 dB / −0.017-0.017 / +0.024+0.024), and the loss grows with how differently the directions are measured.

OperatorDirectionsPSNR change when shared
Box, Rand, Dn, OT SROne singular value0 (identical by construction)
EDM SRSpectrum entirely above the noise level−0.09-0.09 / −0.09-0.09 (FFHQ / ImageNet)
EDM GB, EDM MB, OT GDSpans the noise level, mostly below it−0.30-0.30 to −1.24-1.24
§4.2 graded operatorSpans the noise level−8.31-8.31 (EDM) / −6.71-6.71 (OT)

The benchmark blurs are realistic operators whose spectrum crosses the noise level, with singular values both above and below it, and sharing lowers all three metrics on each of them. Sharing matters even where the targets coincide: on EDM super-resolution only the damping differs across directions, yet sharing lowers all three metrics on both datasets.
Sharing fails because it summarizes the spectrum by its average energy. On EDM Gaussian blur, the shared target stays inside the per-direction range (noise level 3.16 against 2.84–3.70 at σ=81\sigma=81), so sharing mainly changes the damping, and one value replaces the whole range: 2.632.63 against 0.730.73–2.642.64 at σ=81\sigma=81, and 0.0850.085 against 0.00120.0012–0.1520.152 at σ=0.025\sigma=0.025. The strongly measured directions then take much larger steps than their own damping sets, and one global cc cannot correct this without also shrinking the steps of the weakly measured directions. On OT Gaussian deblurring, sharing collapses both components: at t=0.51t=0.51, the shared target is τˉ=0.61\bar\tau=0.61 against 0.500.50–0.900.90, and the shared damping is 6161 against 0.980.98–133133.

AuthorsOpenReview ↗

Response to Reviewer PaFx (3/6)

W2 · Linear combination, damping and variance model

The evaluation point is indeed a linear combination. Because the frozen residual is affine, it combines the two endpoint residuals with a coefficient for each direction and step. Treating the target as a variable is what makes this possible: fixed-target methods use one endpoint, and our contribution is a rule, derived from an objective, that sets this coefficient. The capability also matters on its own. Even setting the derivation aside, assigning a different coefficient to each direction within one update is something fixed-target guidance cannot do, and Table R1 shows that this capability matters.

The damping does not discard the magnitude the loss implies. It sets the step size, which the loss leaves undetermined. The loss depends on the displacement dsd_s only through (r^s+σsds)2(\hat r_s+\sigma_s d_s)^2, and minimizing it alone would take ds=−r^s/σsd_s=-\hat r_s/\sigma_s, which diverges as σs→0\sigma_s\to0 and copies the noise in r^s\hat r_s into the state (§3.3). The regularized solve of Eq. 3.9 is what keeps the step finite. The resulting update still descends the same loss. Since ∇Lλ=∑sZsσsr^svs\nabla\mathcal L_\lambda=\sum_s Z_s\sigma_s\hat r_s v_s (Proposition 3.3), the update −c∑sσsgsr^svs-c\sum_s\sigma_s g_s\hat r_s v_s equals −c P∇Lλ-c\,P\nabla\mathcal L_\lambda with P=∑s(gs/Zs) vsvs⊤P=\sum_s (g_s/Z_s)\,v_sv_s^\top, which is positive on the measured directions. The update is therefore a preconditioned descent step on Lλ\mathcal L_\lambda, with gs/Zsg_s/Z_s as the per-direction preconditioner.

The similarity to spectral filters is real. The damping factor gs=(σs2+γs)−1g_s=(\sigma_s^2+\gamma_s)^{-1} has the Wiener/Tikhonov form and shares its spectral principle with ΠGDM and DDRM, and the revision now discusses this connection in Appendices C.4 and D.3. The coefficient γs\gamma_s is a modeling choice made for a specific goal (Appendix C.4). We wanted the damping factor to depend on both the mean-target and the current sampling time, and adopting ΠGDM's weighting directly would not place the mean-target inside the factor. We therefore defined the step as the minimizer of a regularized displacement (Eq. 3.9) and set γs\gamma_s to the ratio of the measurement-noise variance at the mean-target to the conditional variance at the current step. The resulting form coincides with these filters.

The variance model differs between the backbones for a structural reason. EDM observes yy without rescaling, so if the endpoint estimation error is dropped, the EDM variance increases with the noise coordinate and the precision decreases. In every measured direction the precision is then largest at σmin⁡\sigma_{\min}, and the weighted mean lies below the uniform-weight mean, by an amount that depends on the singular value and the support. The estimated endpoint-error term adds a cost at low noise, so the variance can attain an interior minimum in the chosen interval. EDM therefore keeps this term (Appendix D.2, Eq. D.9). On EDM, the mean-target also does not enter the damping through the same derivation as on OT. We therefore introduced κ\kappa as a modeling choice tuned on validation (Appendix D.3), and we treat it as a hyperparameter rather than a derived quantity.

The comment on correlations combines two different points. Residuals at different τ\tau share the same latent noise and measurement noise, so they are correlated, as we noted at the end of Appendix C.2. Our weights are per-coordinate precisions, and the loss averages per-coordinate costs. Read as the likelihood of a mixture that draws one coordinate (Jensen's inequality bounds its negative log-likelihood by the average cost), the loss does not assume independence across coordinates (Appendix E.1 of the revision), and modeling the joint covariance across coordinates would define a different objective. The independence we do assume is within a single residual, between the forward noise and the endpoint error in EDM Eq. D.9. This is a separate approximation, which the revision now states explicitly in Appendix D.2.

Reviewer_khJgOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

Flow and diffusion-based approaches for inverse problems use guidance to incorporate measurement information, typically by forming a “clean” estimate and applying a likelihood there. This work instead suggests applying measurement guidance at intermediate points along the generative trajectory. Under strong assumptions, the work shows that combining the measurement errors at these points is equivalent to evaluating a single error at their weighted average. They then use estimates of uncertainty to choose this point and adjust the strength of guidance in different directions of the measurement operator. They test the approach on standard inverse problems using diffusion and flow-matching models.

Critical strengths

The method reports strong empirical results across a range of standard inverse problems, consistently matching or outperforming established baselines. The evaluation covers several datasets and both diffusion and flow-matching models.

Critical weaknesses
  1. The central motivation for evaluating measurement consistency at intermediate virtual targets is not sufficiently justified. The measurement likelihood is defined on the clean image, and although the authors construct time-dependent measurement residuals, it is unclear why minimising their weighted sum should provide appropriate guidance for the original inverse problem. The main theoretical result establishes an algebraic identity, but does not demonstrate a connection to the posterior likelihood or justify the choice of weighting. As a result, the mean-target appears to be a heuristic choice of guidance evaluation point rather than a theoretically justified one.

  2. The method requires access to the SVD of the forward operator A, which limits its applicability to large-scale or real-world problems, where computing or applying the SVD is often impractical. Furthermore, the derivation assumes a linear forward operator with additive Gaussian noise, restricting its applicability to general inverse problems involving nonlinear operators or non-Gaussian measurement models.

Questions for authors
  1. [Clarification] As the measurement likelihood (together with the observation model) is defined on the clean image, what is the theoretical justification for evaluating measurement consistency at intermediate points along the generative trajectory? Is there a connection to the marginal likelihood p(y | x_t)?
  2. [Clarification] The mean-target derivation follows from the linearity of the residual, but why should inverse-variance weighting produce a better guidance target? How is this approach connected to the posterior likelihood?
  3. [Clarification] As mentioned in Section 3.2, Appendix C.2 assumes that the clean-image estimate equals the ground truth when deriving the variance weights. How reasonable is this assumption, given that the clean-image estimate is generally imperfect (particularly at early timepoints)?
  4. [Future work] Could the authors compare against simpler interpolation schedules between the current state and the clean-image estimate, while keeping the spectral damping factors fixed?
  5. [Clarification] How practical is the requirement for the SVD of the forward operator in large-scale inverse problems, and could the method be extended to nonlinear operators or non-Gaussian noise?
9 comments
AuthorsOpenReview ↗

Response to Reviewer khJg (1/4)

We sincerely thank the reviewer for the careful reading and for recognizing the empirical strength of the method across backbones and datasets. Below we address each point. New results report PSNR ↑ / SSIM ↑ / LPIPS ↓ on 100 test images, and the tables are collected at the end of this response.

Summary · How guidance is applied

We would like to clarify one point in the summary and briefly restate the idea behind the method.
A training-free sampler must decide where the measurement residual is evaluated. Existing methods fix this choice by convention: DPS-style guidance uses the clean-image estimate, and FIG-style guidance uses the current state. We instead treat the evaluation target as a variable derived from a guidance objective. The mean-target is the value our objective yields, and, as noted in our conclusion, it need not be the only reasonable one.

MTG does not apply guidance at intermediate points along the generative trajectory. At each step:

  1. One model output is frozen at the current state xtx_t. This defines virtual evaluation points zτz_\tau on the line through xtx_t (τ=t\tau=t) and the clean-image estimate (τ=1\tau=1). Full support also includes the noisier side (τ<t\tau<t).
  2. The squared residuals at these points are weighted and integrated into one loss (Eq. 3.2). Because the frozen residual is affine in τ\tau, the gradient of this loss is exactly A⊤A^\top applied to the residual at one coordinate, the weighted mean of τ\tau, up to a scalar factor (Theorem 3.2). This holds for any admissible weight.
  3. The weight is the inverse residual variance (Eqs. 3.4–3.6).
  4. A separately designed damping factor sets how strongly the residual moves the state (Eq. 3.9).

The update is applied to the current state xtx_t. The virtual points are coordinates used to define the objective, and the sampler never moves to them.

AuthorsOpenReview ↗

Response to Reviewer khJg (2/4)

W1, Q1–Q3 · Justification of the objective and the inverse-variance weight

We agree that Theorem 3.2 is an exact algebraic identity and does not by itself establish a connection to the posterior. This reflects a deliberate choice of presentation rather than an oversight. Our goal was a new formulation in which the evaluation target is a variable: we first define a loss over a family of evaluation targets and then derive the target from it, without committing to a particular distributional form. To keep this objective-first structure, we intentionally did not build the main text on a distributional justification. The objective nevertheless admits a likelihood reading, which we summarize below and have added to the revision as Appendix E. In short, the weighted aggregate is an upper bound on the negative log-likelihood of an auxiliary mixture model, its inverse-variance weights are the Gaussian precisions of that model, and the damped MTG update is a preconditioned descent step on the resulting surrogate posterior.
The aggregate arises as follows. Let each virtual point zτz_\tau define a normalized likelihood surrogate p~τ(y∣zτ)\tilde p_\tau(y\mid z_\tau), and consider an auxiliary model that draws one coordinate τ\tau uniformly from the support T\mathcal T and then generates the observation from that component. Its likelihood is the mixture p~mix(y∣xt)=Eτ[p~τ(y∣zτ)]\tilde p_{\rm mix}(y\mid x_t)=\mathbb E_{\tau}[\tilde p_\tau(y\mid z_\tau)], and Jensen's inequality gives
−log⁡p~mix(y∣xt) ≤ Eτ∼Unif(T)[−log⁡p~τ(y∣zτ)].-\log \tilde p_{\rm mix}(y\mid x_t)\ \le\ \mathbb E_{\tau\sim{\rm Unif}(\mathcal T)}\big[-\log \tilde p_\tau(y\mid z_\tau)\big].
Averaging the per-coordinate costs therefore minimizes an upper bound on the mixture negative log-likelihood. The uniform mixture over T\mathcal T is a modeling choice of the auxiliary model, and the weights come from the local likelihoods.
On the OT path, we model each virtual point as zτ∣u∼N(τu,(1−τ)2I)z_\tau\mid u\sim\mathcal N(\tau u,(1-\tau)^2I) and keep the original p(y∣u)=N(Au,σy2I)p(y\mid u)=\mathcal N(Au,\sigma_y^2 I), with the two noises independent given the clean image uu. With a flat working prior, u∣zτ∼N(zτ/τ,((1−τ)/τ)2I)u\mid z_\tau\sim\mathcal N(z_\tau/\tau,((1-\tau)/\tau)^2 I). Marginalizing gives τy∣zτ∼N(Azτ,Cτ)\tau y\mid z_\tau\sim\mathcal N(Az_\tau,C_\tau) with Cτ=(1−τ)2AA⊤+τ2σy2IC_\tau=(1-\tau)^2AA^\top+\tau^2\sigma_y^2I, which is exactly Eq. 3.4. The state-dependent part of the local negative log-likelihood is 12∑srs(τ)2/Cs(τ)\tfrac12\sum_s r_s(\tau)^2/C_s(\tau) with Cs(τ)=(1−τ)2σs2+τ2σy2C_s(\tau)=(1-\tau)^2\sigma_s^2+\tau^2\sigma_y^2, so the average cost in the Jensen bound equals our loss (Eq. 3.6) divided by ∣T∣|\mathcal T|, plus a term that does not depend on the state. The inverse variance is thus the Gaussian precision of each local likelihood. At τ=t\tau=t, the component coincides with the flat-prior approximation of p(y∣xt)p(y\mid x_t) used by DMPS (Meng & Kabashima, Eqs. 14–17).
The whole update then fits one objective. Multiplying the mixture likelihood by the sampler's marginal pt(xt)p_t(x_t) defines an auxiliary posterior, whose negative log is bounded above by −log⁡pt(xt)-\log p_t(x_t) plus the averaged cost. As in other training-free samplers, the model step follows the prior score and the guidance step addresses the likelihood part. Theorem 3.2 makes the gradient of this bound computable exactly at one coordinate per direction. The damped update −c∑sσsgsr^svs-c\sum_s\sigma_s g_s\hat r_s v_s equals −c P∇Lλ-c\,P\nabla\mathcal L_\lambda with P=∑s(gs/Zs) vsvs⊤P=\sum_s (g_s/Z_s)\,v_sv_s^\top, which is positive on the measured directions, so it is a preconditioned descent step on the same bound. The aggregation, the weight, the target and the damping are therefore parts of one surrogate objective.
This reading gives a model-based account of why we use this objective and this weight. It does not claim that the mean-target is optimal in general, and Appendix E states this scope.

The weight also holds up in practice. Under look-forward support, replacing the inverse-variance mean-target with the uniform-weight mean or the precision mode lowers PSNR by 0.46 dB and 0.66 dB on average over the 20 benchmark settings, and each alternative fails badly on some operator (see our response to Reviewer kKXP, Q3).

Regarding Q3, Appendix C.2 reaches the OT variance by dropping the endpoint estimation error, and we agree that this can be inaccurate, particularly early in sampling. The likelihood reading above does not require this assumption, because the variance follows from the auxiliary Gaussian conditional. It does not, however, show that this variance equals the covariance of the actual frozen residuals. On EDM, we retain an estimated endpoint-error term (Eq. D.9) as a backbone-specific variance model. The algebraic reduction of Theorem 3.2 holds under either variance model.

AuthorsOpenReview ↗

Response to Reviewer khJg (3/4)

Q4 · Comparison with interpolation between the two endpoints

The interpolation view gives a useful way to test whether our target rule offers more than a simple fixed blend. Because the frozen residual is affine, every evaluation point can be written as r=(1−α) rcurrent+α rdatar=(1-\alpha)\,r_{\rm current}+\alpha\,r_{\rm data}. MTG determines this coefficient through the weighted objective, so it differs per direction and per step and can lie outside the segment under full support (α<0\alpha<0, on the noisier side). The added comparison replaces that rule with constant coefficients while keeping the damping fixed.
Controlling the damping requires the setting of §4.2, where mixing and κ\kappa are turned off (m=κ=0m=\kappa=0). We added constant coefficients α∈{0.25,0.5,0.75}\alpha\in\{0.25,0.5,0.75\}, shared by all directions and steps, to the existing endpoints α=0\alpha=0 (current state) and α=1\alpha=1 (clean estimate), and selected cc on validation for every row. We used two damping settings. Under the derived rule, κ=0\kappa=0 makes the damping values identical across targets on EDM, while on OT the same formula is evaluated at each target because it contains the noise variance at the target. Under the constant g=1g=1, the damping is identical for every target, which is the strictest form of fixed damping.
With damping fixed at g=1g=1, MTG improves all three metrics over every constant coefficient on both backbones (Table R1):

  • EDM: 21.83 / 0.641 / 0.296 vs. 20.38 / 0.374 / 0.444 at the validation-selected α=0.25\alpha=0.25.
  • OT: 22.14 / 0.699 / 0.196 vs. 21.17 / 0.500 / 0.293 at α=0.75\alpha=0.75.

With derived damping, MTG also improves all three metrics on OT (28.32 / 0.833 / 0.088 vs. 21.77 / 0.621 / 0.264 at the selected α=0.75\alpha=0.75). These OT runs use different batch groupings for MTG and the constant coefficients, and hence different measurement-noise draws. On EDM, MTG leads on PSNR and SSIM (30.96 / 0.900 vs. 30.63 / 0.895 at the selected α=0\alpha=0), while LPIPS is comparable (0.119 vs. 0.118). On this operator, EDM's directions differ mainly in the damping, which every row keeps, and once the damping is removed (g=1g=1) the target matters clearly. On OT, the derived per-direction coefficients span a wide range within a single step, extending to both sides of the current state, so no single constant can match every direction. The damping factor is also effective in this setting: for every target row on both backbones, derived damping improves PSNR and SSIM over g=1g=1.

W2 and Q5 · SVD requirement and scope

The direction-resolved method needs the measured singular values and the matching basis operations of AA. The standard operators in this literature (masks, super-resolution, blur) admit inexpensive spectral implementations, and SVD- or pseudo-inverse-based methods such as DDRM, ΠGDM and DDNM rely on the same access. Where an operator is not exactly circulant, as for the EDM blur and super-resolution operators, we already use the FFT spectrum together with a measured mismatch correction δA\delta_A (Appendix F.1.1, Eqs. F.2–F.3). These quantities are computed once per operator and reused across images, which takes 0.78 s on EDM and 0.12 s on OT in our runtime measurements. If not even such an FFT-type approximation is available, the direction-resolved form does not apply directly, and the revision states this as a limitation in Appendix F.1.1. A cheaper variant that summarizes the spectrum by its average needs only ∥A∥F2\|A\|_F^2, but it lowers all three metrics on every benchmark operator with a spectrum (by 0.66 dB PSNR on average over eight settings).

We agree that the derivation relies on a linear operator with additive Gaussian noise. This assumption makes the residual on the extrapolated line affine in τ\tau, which Theorem 3.2 requires, and it gives the target and the damping of every direction in closed form. For a nonlinear forward map, the residual is no longer affine in τ\tau, so Theorem 3.2 does not apply as stated. One possible extension is to linearize the forward map around the current clean estimate at each step and apply the same construction to the linearized operator. We have not evaluated this extension and leave it to future work. For non-Gaussian noise, Theorem 3.2 itself does not use Gaussianity, and the inverse-variance weight needs only the second moments of the noise, so the construction extends directly to additive noise with finite variance. What changes is the likelihood reading, which assumes Gaussian noise, and our experiments do not cover this case. The revision states this scope in Appendix F.1.1.

We hope these clarifications and new results address the reviewer's concerns, and we would be glad to answer any further questions.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank the reviewers for their constructive feedback. We are encouraged that the reviewers highlighted the strong reconstruction results across standard inverse problems, datasets, and both diffusion and flow-matching backbones (Reviewers khJg, PaFx, kKXP), the practicality of a method that needs one model evaluation per step without backpropagation and precomputes its per-direction quantities (Reviewers PaFx, kKXP), and the clarity of the writing (Reviewer kKXP). Reviewer PaFx also noted that placing the data-endpoint and current-state residuals on one family gives a clear dimension along which guidance schemes can be compared, and that on the graded diagonal operator both the mean-target and the damping factor improve reconstruction, with the largest gain in weakly measured directions.

We added a likelihood interpretation in Appendix E and expanded the experimental evaluation in Appendix F. We also revised the main text to distinguish adaptation across operators from adaptation across directions within one operator. §3.2 now states both, §4.1 refers to the new direction-sharing results, and the opening of §4.2 clarifies that its ablation measures what the mean-target and the damping factor each contribute rather than testing per-direction processing. The main text also refers to the new appendices where relevant. New and revised material is marked in blue.

Objective and likelihood interpretation (Appendix E). Theorem 3.2 gives the mean-target for any admissible weighting of the residual loss. For OT, averaging local Gaussian negative log-likelihoods upper-bounds the negative log-likelihood of an auxiliary mixture. The local Gaussian precisions supply the inverse-variance weights, and damping preconditions the gradient of the frozen loss. EDM uses the same algebraic reduction with a separate variance model that accounts for endpoint estimation error.

Operator adaptivity (Appendices F.1.5 and F.2.3). The target and damping schedules depend on the operator. Damping can vary substantially across directions even when their targets are close. Sharing both across directions worsens all three metrics in the eight benchmark settings with distinct measured singular values, with an average PSNR drop of 0.66 dB. On the graded operator of §4.2, PSNR drops by 8.31 dB on EDM and 6.71 dB on OT.

Additional ablations (Appendix F.1.5). Comparing both supports across 20 settings shows that support choice matters especially on EDM. With common look-forward support and the same damping, MTG leads the uniform-weight mean and precision mode on all three metrics on average. Disabling null-space mixing lowers PSNR and SSIM in all 14 settings that used it, even after retuning. Measurement guidance comes from the MTG update.

Validation and runtime (Appendix F). We document the validation splits and tuning protocol and report a 100-step runtime and memory comparison with FIG and DPS. MTG has comparable runtime to FIG at small inner iteration counts.

The individual responses provide the full results.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.