acceptodds
Under review as a conference paper at ICLR 2027

Plug-in Image Quality Control for Few-Step Posterior Diffusion Super-Resolution

Abstract

Diffusion-based super-resolution (SR) has made remarkable progress, mainly through diffusion-prior approaches that rely on assumed degradation likelihoods or generative priors. Posterior diffusion SR instead learns directly from LR–HR pairs, but plug-in mechanisms for controlling the image quality of pretrained models remain less explored. To the best of our knowledge, we propose the first plug-in module for few-step posterior diffusion SR that enables image quality control without retraining the core model. Motivated by high-order diffusion solvers, we formulate a derivative-based correction targeting the per-step posterior-mean discrepancy, which may include the effects of accumulated discretization error. This discrepancy is related to a covariance-weighted KL gradient, motivating an LR-based image-guidance surrogate whose effective scale is calibrated for each denoising step. The sign of the guidance selects between a fidelity-oriented mode and a sharpness-oriented mode. In blind SR, the fidelity-oriented mode improves fidelity metrics, while the sharpness-oriented mode achieves higher scores on non-reference perceptual metrics such as MANIQA and CLIPIQA surpassing Stable Diffusion-based methods on RealSR. Our module provides continuous access to a range of image-quality operating points within a single pretrained posterior model.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
diffusion model, super resolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_wt8UOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

For few-step posterior diffusion SR, a plug-in guidance method that does not require retraining the backbone is proposed. The stepwise guidance scale is obtained through LR–HR calibration, and the positive and negative directions of the guidance are used to switch between fidelity-oriented and sharpness-oriented operating points.

Critical strengths

The paper introduces a practically appealing plug-in mechanism that turns pretrained few-step posterior diffusion SR models into controllable systems without retraining the backbone. The solver-inspired perspective is interesting, and the experiments on ResShift and UPSR show a consistent ability to move between fidelity-oriented and sharpness-oriented operating points.

Critical weaknesses

The central theoretical narrative is stronger than what is ultimately justified by the inference-time algorithm: the LR-based surrogate is calibrated only in magnitude and is not shown to align directionally with the oracle correction, while the derivation also relies on a nontrivial Jacobian approximation. Moreover, the claimed continuous operating range and upper/lower bounds are insufficiently validated because the quantitative evaluation focuses mainly on two endpoint settings rather than dense control sweeps.

Questions for authors
  1. The connection between the proposed guidance and a genuine second-order discretization correction is not sufficiently established. The paper itself acknowledges that \delta_t mixes prediction error, conditional uncertainty, and accumulated discretization error, while Eq. (5) additionally relies on the strong approximation J_t\approx-a_tI. Please empirically quantify the Jacobian residual and clarify which claims are exact, approximate, or purely motivational.

  2. Eq. (13) calibrates only the magnitude of the LR-based gradient against the oracle discrepancy, but does not ensure directional alignment between them. Since the effectiveness of a mean-shift correction critically depends on direction, this is a missing validation. Please report per-timestep cosine similarity between the proposed LR guidance and the oracle correction, together with its variance across images and datasets.

  3. Continuous-control claim. Continuous image-quality control is presented as a main contribution, yet Table 2 primarily reports the two endpoint settings LR+ and LR−. Please provide dense sweeps over s in [-1,1] and plot fidelity/perceptual metrics against s for both backbones and datasets.

  4. The manuscript refers to LR+ and LR− as upper and lower bounds of attainable image quality, but the current analysis does not establish that these endpoints are extrema for the reported metrics. Different metrics may be non-monotonic with respect to the guidance strength, especially under sharpening artifacts. I suggest either providing empirical/theoretical justification for the bound interpretation or using a more conservative term such as “operating endpoints.”

  5. The experiments use ResShift and UPSR, but these models share closely related posterior/residual-shifting formulations, so the evidence for generality is limited. And the sharpness-oriented mode is mainly supported by MANIQA and CLIPIQA, a broader set of perceptual metrics would substantially strengthen this claim.

1 comment

We thank the reviewer for the constructive review. We apologize for the late response, which leaves little time for discussion.

Q1. Second-order connection; what is exact, approximate, or motivational. We no longer claim that the correction is equivalent to a second-order discretization correction. We claim that it targets a discrepancy that contains the discretization error, as summarized below.

  • Exact:

    • δt\delta_t is the preconditioned mean gradient of the reverse-kernel KL (Eq. 4; App. D.7, Eq. 103).
    • δt\delta_t is the sum of the Euler local error and a modular component (Eq. 104).
    • δt=−αtηtJˉtvtH\delta_t=-\tfrac{\alpha_t}{\eta_t}\bar J_t v^{\rm H}_t (Eq. 105).
    • Substituting the LR reference for the HR reference gives an identity with a residual and a reference term (Eq. 6).
    • The covariance cancels under matched covariance (App. D.2, Eqs. 49–50).
  • Approximate, measured:

    • Jˉt≈ctI\bar J_t\approx c_tI (Eq. 106; Table 21).
    • The LR-based direction replaces the HR direction (Eq. 6; see Q2).
  • Motivational:

    • The multistep correspondence with DPM-Solver++ (App. D.1).
    • The unsharp-mask interpretation of LR⁻ (App. D.2).
  • What remains of the discretization link.

    • The target contains the Euler local error (Eq. 104). Under the Gaussian model, its share of the target grows with the step size (ratio after Eq. 108).
    • The measurements follow this trend. Under +1×, the per-step error falls by 5.5% and 10.5% at the last two guided steps of 4-step ResShift, but by at most 3.7% at any of the last 13 steps of 15-step ResShift (Tables 10–11). With the HR-referenced oracle, PSNR rises by 2.42 dB for 4 steps and by 1.40 dB for 15 steps (Tables 5–6).
    • The correction therefore acts more strongly where the discretization is coarser. Eq. 104 does not, however, let us attribute this effect to the Euler component alone.
  • Jacobian sign. The approximation is now Jˉt≈ctI\bar J_t\approx c_tI with ct>0c_t>0. The negative slope comes only from the minus sign in Eq. 105, and the approximation is exact for the Gaussian predictor (Eq. 107).

  • Jacobian residual (Table 21). From the second step on, one scalar explains 35–98% of the target energy, and the band-wise slopes are positive and within 1.8× of each other. At the first step it explains about 10%; the correction there is weak but keeps the correct sign, and the applied LR correction is positively aligned for all images (Table 20).

Q2. Magnitude-only calibration and directional alignment.

  • The objective now depends on direction. We replaced the magnitude objective. Eq. 13 now fits the vector residual ∣Δi,t+αΣθ,i,tGi,t∣1|\Delta_{i,t}+\alpha\Sigma_{\theta,i,t}G_{i,t}|_1 on a grid that contains zero. The fitted αt∗\alpha^*_t is therefore a projection coefficient: its sign follows the alignment, and it is near zero when the directions are unrelated.
  • Partial alignment is enough. By the partial-correction bound (App. D.7), any positive alignment combined with a scale below the optimum reduces the discrepancy.
  • Evidence.
    • Fig. 4 shows the cosine between the HR- and LR-based KL gradients along the trajectory.
    • The per-step error decreases at most steps under +1× and increases under −1× (Tables 10–12).
    • Per-timestep cosine (Table 20). Except the three steps, the cosine is positive for at least 92% of the images. The calibrated scale falls inside the error-reducing range (0,2)(0,2) of the bound above for 76–100% of the images.

Q3. Dense sweeps over ss.

  • On RealSR, Table 4 (UPSR) evaluates nine values of s∈[−1,1]s\in[-1,1] in steps of 0.25, extended to ±3×; Tables 5–6 (ResShift) use steps of 0.5; Fig. 10 shows the corresponding outputs.
  • For UPSR and 4-step ResShift, PSNR and MANIQA change monotonically in ss, and CLIPIQA does too, except for a small dip at s=−0.25s=-0.25 for UPSR. LPIPS has an interior optimum near s=0.5s=0.5, and NIQE is not monotonic.

Q4. "Upper and lower bounds." We agree and adopted the reviewer's term. Sec. 4 (¶"Operating range") now calls LR⁺ and LR⁻ operating endpoints, and the Conclusion describes the calibrated scale as a practical end point of the control range. Tables 4–6 confirm that they are not extrema: larger scales move some metrics further, for example CLIPIQA 0.539 at 3× vs. 0.492 at 1× for UPSR, at the cost of artifacts (Fig. 10).

Q5. Generality and perceptual metrics.

  • Other backbones. Table 17 applies the module to GOUB and IR-SDE. These are 100-step mean-reverting/GOU formulations, not residual shifting. On both, LR⁻ increases MANIQA and CLIPIQA. LR⁺ improves LPIPS and DISTS on IR-SDE, but its full-reference gain is small at 100 steps.
  • Perceptual metrics. Tables 13–16 add MUSIQ, TOPIQ, and LIQE to MANIQA and CLIPIQA. For UPSR and 4-step ResShift, all three increase under LR⁻ and decrease under LR⁺, consistent with MANIQA and CLIPIQA.
Reviewer_YwrYOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a training-free inference-time guidance module for few-step posterior diffusion models for image super-resolution. The method uses an LR-based surrogate for a theoretically motivated KL-gradient correction and calibrates its scale at each diffusion step. By changing the sign of the guidance, the method aims to control the fidelity–perceptual quality trade-off without modifying the pretrained model. Experiments on ResShift and UPSR report improvements in distortion metrics under fidelity-oriented guidance and no-reference perceptual metrics under sharpness-oriented guidance.

Critical strengths
  1. Practical control: The proposed method addresses a relevant practical problem: adjusting the fidelity–perception trade-off (long standing problem in the SR community) of pretrained few-step diffusion models without retraining. Because the correction requires only a simple modification of the reverse mean, it introduces negligible additional inference cost.

  2. Evaluation: The paper evaluates both directions of the proposed guidance rather than reporting improvements at only one operating point. This is useful because image SR inherently involves a trade-off between reconstruction fidelity and perceptual quality. The discussion of NIQE and other potentially conflicting perceptual metrics also demonstrates awareness of the limitations of current evaluation protocols.

Critical weaknesses
  1. Insufficient justification for the Jacobian approximation: Proposition C.1 assumes that the denoiser Jacobian satisfies ∂fθ/∂xt≈−(ηt−1/αt)I\partial f_\theta/\partial x_t \approx -(\eta_{t-1}/\alpha_t)I, implying a negative-definite Jacobian. However, for a Bayes-optimal x0x_0-predictor under the assumed Gaussian forward process, the Jacobian is proportional to the conditional covariance, which is positive semidefinite, raising concerns about the validity of the proposed approximation.

  2. Independence: The inference-time method replaces the oracle gradient with Γ−1(xt−c)\Gamma^{-1}(x_t-c), which is independent of the denoiser Jacobian. This makes the connection between Proposition C.1 and the actual guidance mechanism unclear. The authors should empirically examine the Jacobian's sign and spectrum, as well as the approximation residual rtr_t, to better justify the theoretical formulation.

  3. Unclear whether fidelity improvements: Although LR+ improves several full-reference metrics, the results suggest that these gains may primarily come from suppressing high-frequency details rather than correcting reconstruction errors. For example, on RealSR, UPSR with LR+ achieves a MANIQA nearly identical to Bicubic's, while its CLIPIQA score also shifts toward the Bicubic baseline. Moreover, LR+ consistently degrades FID for both UPSR and ResShift on RealSR.

  4. Continuous control: Although the method introduces a continuous guidance parameter ss, the main quantitative evaluations report only the two endpoints, s=±1s=\pm1. Without metric-versus-ss curves, it is difficult to verify whether the method provides smooth, monotonic, and stable control over the fidelity–perception trade-off.

Questions for authors
  1. Can the authors empirically evaluate the Jacobian approximation, including its spectrum and the relative magnitude of the residual?

  2. How sensitive are the coefficients to the dataset? (e.g., scale)

  3. Can the authors provide human preference evaluations or more robust perceptual comparisons (e.g, MOS)?

1 comment

We thank the reviewer for the constructive review. We apologize for the late response, which leaves little time for discussion. We rewrote the theory: Proposition C.1 is replaced by Sec. 3.3 and App. D.7. All references are to the revised PDF.

W1 / Q1. Jacobian sign, spectrum, and residual.

  • The approximation is now Jˉt≈ctI\bar J_t\approx c_tI with ct>0c_t>0 (Eqs. 105–106). The negative slope A^t=−αtηtct\hat A_t=-\tfrac{\alpha_t}{\eta_t}c_t comes only from the explicit minus sign. As the reviewer notes, this is consistent with the Jacobian of the Bayes-optimal predictor, which is proportional to the conditional covariance. It is exact for the Gaussian predictor (Eq. 107).
  • Table 21 reports the band-wise slopes c^t,ℓ\hat c_{t,\ell}, which are Rayleigh quotients of the Jacobian on the low/mid/high bands.
    • Sign: all band-wise slopes are positive.
    • Spectrum: they differ by at most 1.8× across bands.
    • Residual: 1−cos⁡21-\cos^2, i.e., 2–65% of the target energy from the second step on, and about 90% at the first step.
  • These quantities are measured on the noise-matched displacement, which separates the noise carried by xtx_t (App. D.7, ¶"Noise-matched HR reference").

W2. Independence of the inference direction from the Jacobian.

  • The Jacobian enters only through the per-step scalar ctc_t, and the calibrated αt∗\alpha^*_t estimates it (Eq. 13 vs. Eq. 106). For this reason the inference direction needs no Jacobian.
  • Eq. 6 is an identity relating the LR-based and HR-reference directions, with an HR fitting residual and a reference term. A per-step rescaling is absorbed by the slope (Eq. 115).
  • Empirically, the per-step error decreases at most steps under +1× and increases under −1× (Tables 10–12).
  • Direct measurement (Table 20, 50 calibration tiles). Except at the three steps with the smallest effective coefficients, the cosine between δt\delta_t and the applied LR correction is positive for at least 92% of the images, with means of 0.20–0.31 (UPSR) and 0.19–0.61 (ResShift). The LR direction thus points toward the target without any Jacobian and covers part of it: the attainable per-step reduction is 2–6% for UPSR and up to 22% for ResShift.

W3. LR⁺ fidelity vs. high-frequency suppression.

  • We partly agree. LR⁺ is the fidelity end of the trade-off and suppresses high-frequency content (Fig. 10). FID increases in most settings, which we now state in Sec. 4.
  • LR⁺ is not equivalent to low-pass filtering. At comparable PSNR, Gaussian smoothing raises NIQE to 8.0–8.5 and lowers CLIPIQA to 0.29–0.32, whereas LR⁺ keeps them at 6.3–6.4 and 0.32–0.39 (Tables 3–4). The per-step error toward the HR reference also decreases (Tables 10–12).

W4. Continuous control.

  • Table 4 (UPSR) evaluates nine values of s∈[−1,1]s\in[-1,1] in steps of 0.25, extended to ±3×; Tables 5–6 (ResShift) use steps of 0.5. The multiplier in these tables equals ss for ∣s∣≤1|s|\le1.
  • Within this range, PSNR and MANIQA change monotonically for UPSR and 4-step ResShift (Tables 4–5). CLIPIQA does too, except for a small dip at s=−0.25s=-0.25 for UPSR. LPIPS has an interior optimum near s=0.5s=0.5, and NIQE is not monotonic.
  • Beyond the calibrated scale, outputs over-smooth or over-sharpen (Fig. 10).

Q2. Sensitivity to the dataset.

  • Fig. 5 compares image-specific with dataset-level calibration.
  • Table 21 gives the image-to-image variation of the slope (CV 0.02–0.45 across steps).
  • Tables 4–6 show that the metrics change smoothly with a global multiplier around 1×.

Q3. Human evaluation. We agree that no-reference metrics are imperfect proxies, which is why we report where they disagree (NIQE vs. MANIQA/CLIPIQA). A controlled MOS study requires a sufficient number of raters and a controlled protocol, which we could not arrange within the rebuttal period. As more robust perceptual comparisons, we add MUSIQ, TOPIQ, and LIQE (Tables 13–16) to LPIPS and DISTS, and the trends of the two modes agree across them. These metrics can score sharpened outputs above the HR reference (Tables 13, 16), so we present them as descriptors of operating points rather than as a ranking of perceived quality.

Reviewer_YW8nOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a plug-in quality-control method for few-step posterior diffusion super-resolution. It calibrates per-step guidance scales using paired LR–HR data and uses only LR conditioning at inference, controlling fidelity and sharpness through the sign of the guidance. Experiments show that the method can adjust the image-quality trade-offs of ResShift and UPSR, but its connection to second-order discretization-error correction has not been adequately established.

Critical strengths
  1. The method allows output quality to be adjusted without retraining the backbone, offering practical value.
  2. The main trends are consistent across two backbones and two datasets: LR⁺ improves full-reference metrics, while LR⁻ improves some no-reference metrics and FID.
  3. The authors demonstrate artifacts caused by excessive guidance and discuss disagreements among evaluation metrics.
Critical weaknesses
  1. The main text claims formal equivalence to second-order error correction, but the appendix explicitly acknowledges that no convergence-order guarantee is established. It also does not establish equivalence between the auxiliary KL gradient and the reverse-kernel KL gradient. The HR-referenced mean discrepancy includes prediction error and conditional uncertainty and therefore cannot be directly identified with discretization error.

  2. Equation (74) approximates the Jacobian by a negative scalar multiple of the identity matrix, whereas the ideal MSE predictor under Equation (27) has a positive semidefinite Jacobian. Although the actual network need not satisfy this ideal property, the authors neither explain why the approximation is reasonable nor assess the magnitude of the neglected residual.

  3. Equation (13) matches ∥Δt∥\|\Delta_t\| to αt∥Gt∥\alpha_t\|G_t\|, whereas the actual displacement in Equation (14) involves Σθ,tGt\Sigma_{\theta,t}G_t. The paper does not sufficiently explain how the covariance enters calibration or under what conditions it can cancel out or be absorbed into the scale.

  4. The experiments do not isolate the contributions of per-step calibration, the calibration criterion, and guidance during sampling. In particular, comparisons with a shared guidance scale, standard parameter search, and properly tuned smoothing/sharpening baselines are missing. Consequently, it remains unclear whether the gains primarily arise from simple residual processing.

  5. The main text lacks a workflow diagram that clearly distinguishes offline calibration from online inference, as well as the HR-reference gradient used in the theoretical analysis from the LR-based auxiliary gradient used in practice. This makes the method difficult to follow.

Questions for authors

Please refer to the weaknesses above.

1 comment

We thank the reviewer for the careful review. We agree with the main concern that the connection to second-order discretization-error correction was overstated. We rewrote Section 3.3 and Appendix D, and revised the corresponding statements in the Introduction, Experiments, and Conclusion. Every claim is now either an exact identity or a stated approximation whose error we measure. All references are to the revised PDF.

W1. Equivalence claim; auxiliary vs. reverse-kernel gradient; discrepancy vs. discretization error. We agree on all three points.

  • We removed the equivalence claim. App. D.1 now serves only as motivation.
  • What remains is a qualified link. The target contains the Euler local error (Eq. 104), and its share grows with the step size (ratio after Eq. 108). Consistently, the correction acts more strongly with coarser steps: under +1×, the per-step error falls by up to 10.5% for 4-step ResShift vs. at most 3.7% at the last 13 steps of 15-step ResShift (Tables 10–11), and the HR oracle raises PSNR by 2.42 dB vs. 1.40 dB (Tables 5–6).
  • The target δt=μtnet−μθ\delta_t=\mu^{\rm net}_t-\mu_\theta is the preconditioned mean gradient of the reverse-kernel KL (Eq. 4; App. D.7, Eq. 103). It splits into the Euler local error and a modular component (Eq. 104), so we no longer identify it with discretization error (Sec. 3.3, ¶"Posterior-mean discrepancy").
  • The two gradients are linked by δt=−αtηtJˉtvtH\delta_t=-\tfrac{\alpha_t}{\eta_t}\bar J_t v^{\rm H}_t (Eq. 105). The only approximation is Jˉt≈ctI\bar J_t\approx c_tI (Eq. 106).

W2. Jacobian sign and residual.

  • The approximation is now Jˉt≈ctI\bar J_t\approx c_tI with ct>0c_t>0. The slope is negative only because of the minus sign in Eq. 105. This is consistent with the positive semidefinite Jacobian of the ideal predictor, and the approximation is exact for the Gaussian predictor (Eq. 107).
  • App. D.7, ¶"Where the uncertainty weighting enters", states when a scalar is justified.
  • Table 21 measures the residual. From the second step on, the scalar explains 35–98% of the target energy, and the band-wise slopes differ by at most 1.8×. At the first step it explains about 10%; the slope keeps the correct sign there, and the applied LR correction is positively aligned for all images (Table 20), so by the partial-correction bound in App. D.7 the correction there is weak rather than harmful.

W3. Covariance in calibration. Eq. 13 now fits the same vector Σθ,tGt\Sigma_{\theta,t}G_t that is applied at inference (Eq. 14). The conditions under which the covariance cancels are given in Sec. 3.3 (¶"LR-based image guidance", last paragraph), App. D.2 (Eqs. 49–50), and App. D.7. For ResShift, any isotropic Γt\Gamma_t cancels; for UPSR, cancellation requires Γt∝Wu2\Gamma_t\propto W_u^2.

W4. Ablations and simple residual processing.

  • Smoothing and unsharp masking are compared in Tables 3 and 15 and Fig. 11. At comparable PSNR, smoothing raises NIQE to 8.0–8.5, whereas LR⁺ keeps it at 6.3–6.4 (Sec. 4, ¶"Quantitative analysis").
  • Sensitivity to the scale is shown in Tables 4–6 and Fig. 10, and image-specific vs. dataset-level calibration in Fig. 5. Fig. 9 shows that changing the per-step combination of scales (sign, magnitude, and ratio across steps) produces distinct artifacts. The calibration data are described in Sec. 4, ¶"Testing environment".
  • Per-step vs. shared scale.
    • The calibrated coefficients differ across steps more than across images. The effective per-step coefficients αt∗vt\alpha_t^*v_t span 6.0–54.7 (9.1×) for UPSR and 5.1–69.8 (14×) for 15-step ResShift, excluding the two steps whose coefficients are near zero (Table 8). Across images, the per-image optimal slope varies with a coefficient of variation of at most 0.45 (Table 21), and the applied scale lies inside the factor-2 band below for 76–100% of the images at all but the three steps with the smallest coefficients (Table 20).
    • By the partial-correction bound (App. D.7), a scale above a step's optimum increases that step's error, and a scale far below it removes little.
    • With a 9.1× spread, any coefficient shared across the UPSR steps either exceeds twice the optimum at t=3t=3, producing artifacts, or stays below 23% of the optimum at t=4t=4 and t=1t=1, recovering at most 40% of the attainable reduction there. Sharing across images costs far less, which is why we calibrate per step and share across images.
    • Table 19 (UPSR) tests this directly. It fixes αt∗vt\alpha_t^*v_t at every step to its first-step value 53.3, which is 8.9× the optimum at t=3t=3.
      • At s=+1s=+1, PSNR is unchanged (29.06 vs. 29.09), but LPIPS worsens by 14% (0.230 → 0.262), and DISTS and NIQE worsen as well.
      • At s=−1s=-1, PSNR drops by 2.2 dB, NIQE rises from 6.52 to 8.60, and FID rises from 130.0 to 159.6.

W5. Workflow.

  • Fig. 1 (Sec. 3.3) block diagram is revised.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.