acceptodds
Under review as a conference paper at ICLR 2027

Unlocking All-in-one Restoration From Pre-trained Diffusion Models

Abstract

Pre-trained text-to-image and text-to-video diffusion models (PTDMs) have enabled significant advancements in restoration by serving as powerful clean-image priors. Existing restoration methods typically exploit these priors through backbone fine-tuning, ControlNet-style modules, explicit degradation operators, or inversion. In this work, we investigate whether a frozen PTDM intrinsically possesses restoration capabilities that can be elicited through its native conditioning pathway to directly steer noisy degraded states toward clean outputs. We first probe the conditioning pathway of PTDMs through text prompts and text-token optimization, and find that neither effectively enables restoration. Interestingly, optimizing the conditioning prompt directly at the text-encoder output unlocks strong restoration behavior from the PTDM. However, naively learning this prompt to denoise noisy degraded states to a clean output is unstable due to a misalignment with the reverse sampling trajectory. To resolve this, we learn prompts within a diffusion bridge formulation that aligns training and inference dynamics, enforcing a coherent denoising path from noisy degraded states to clean images. Furthermore, we introduce a sampling-time partial inversion strategy to improve restoration performance and a prompt composition scheme for handling mixed degradations. Instantiated on the WAN text-to-video and FLUX text-to-image models, our approach elegantly enables these PTDMs to achieve strong and competitive restoration performance. The code will be made publicly available.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
All-in-one restoration, diffusion models, low-level vision

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_wYYiOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper asks whether frozen pre-trained text-to-image/video diffusion models (PTDMs) can perform restoration when steered only through their native conditioning pathway. Natural-language prompts and input-token optimization fail. A “conditioning prompt” optimized at the text-encoder output (in practice a gated residual added to the null-text context at every block) does elicit restoration. Training this prompt naively on noisy degraded states leads to train–test trajectory mismatch, so the authors learn it on a monotone noisy-to-clean diffusion bridge (EBR). They add partial inversion for an image-specific start state and a normalized velocity composition for mixed degradations. The approach is instantiated on FLUX (images) and WAN (videos), with one prompt per degradation. It is evaluated on in-distribution, OOD, mixed and unseen (UDC) benchmarks against all-in-one restoration methods and backbone-matched LoRA variants.

Critical strengths

Competitive all-in-one restoration from a fully frozen backbone with very few trainable parameters. The approach beats backbone-matched LoRA variants that train 50–180× more parameters.

The trajectory-mismatch analysis is clear and supported by ablations: naive training < DDBM < EBR, and partial inversion helps on both backbones. The evaluation covers many real-world OOD datasets for both images and videos.

Critical weaknesses

The “elicitation, not adaptation” claim is overstated. The prompt is a learned residual injected into every transformer block with per-block gates. Functionally this is deep prompt/prefix tuning with 0.34–1.57M parameters per task. No experiment separates an “intrinsic” capability from what the optimized prompt learns. There is no comparison with prefix tuning or LoRA at a matched parameter budget, and no control with a weaker or less pre-trained prior. The bridge (EBR) is adopted from prior work, which limits methodological novelty.

The fairness and rigor of the comparisons are questionable. The method is non-blind: the prompt is chosen with the degradation label, and for UDC it is chosen manually (low-light+blur). Several baselines are blind. The LoRA “Concat” baseline seems under-tuned. No variance or seeds are reported, and several test sets are very small.

Fidelity and cost trade-offs. In-distribution PSNR/SSIM trail lightweight methods. FLUX inference costs 5.51 s and 348 TFLOPs per image, against ~0.14 s for DCPT/DFPIR.

Questions for authors

How does the method compare with prefix/deep prompt tuning and with LoRA at the same parameter budget, all trained on the same EBR bridge with partial inversion?

How do the baselines perform when given the same degradation label, and how sensitive is UDC performance to the manual choice of prompt?

Please report variance across seeds and the hyperparameter tuning effort spent on the LoRA “Concat” baseline.

How much of the gain over the “Bridge (LoRA)” variant comes from partial inversion rather than from conditioning-space prompting?

7 comments
AuthorsOpenReview ↗

Response to Reviewer wYYi

We thank the reviewer for their constructive feedback and for recognizing the effectiveness of our approach, the trajectory-mismatch analysis, and the breadth of our evaluation. We address the concerns below.

  1. CW1, Q1; ``Elicitation, not adaptation'' claim is overstated; Prompt is a learned residual injected into every block with learnable gates and is functionally prefix/deep prompt tuning.

We agree that our conditioning prompt can be viewed as a form of prefix/deep prompt tuning, as both optimize conditioning representations while keeping backbone weights frozen. We have now clarified this in the revised version (Sec. B.1). However, our objective is not to introduce a new prompt-tuning mechanism, but to investigate whether restoration-relevant capabilities of a pretrained diffusion model (PTDM) can be accessed through its conditioning pathway.

As motivated in Sec. 1 (L48--53), diffusion-based editing demonstrates that frozen PTDMs can steer noisy image states toward desired outputs through language conditioning. We therefore ask whether a frozen PTDM can similarly steer noisy degraded states toward clean outputs, which would reveal restoration-relevant capability. However, natural-language instructions fail to elicit this behavior, motivating our investigation in Fig. 2. Crucially, Fig. 2 demonstrates that directly optimizing the text-encoder output enables restoration from the unaltered PTDM without diffusion-bridge training, residual conditioning, or learnable gates, establishing our central observation independently of these components.

Thus, we use prompt learning as a tool to probe and elicit restoration behavior from a frozen PTDM, analogous to how natural-language instructions can be used to guide generative diffusion models. While our final framework shares mechanisms with deep prompt tuning, we learn a single degradation-specific prompt shared across inputs and reused across blocks, rather than independent layer-specific prompts. We discuss block-wise prompts (similar to deep prompt tuning) as potential extensions in Sec. B.3. Thus, our distinction from prefix/deep prompt tuning lies primarily in our objective which is investigating whether restoration-relevant capabilities can be accessed through conditioning a frozen PTDM.

  1. CW1, Q1; No experiment separating intrinsic capability from what the prompt learns; No control with weaker or less pre-trained prior.

We conduct two experiments to support whether learned prompts leverage restoration-relevant capabilities of the PTDM rather than independently learning the restoration mapping.

Elicitation with limited conditioning capacity. We repeat the elicitation experiment of Fig. 2 on REVIDE (on WAN) using only 88 learnable tokens (32,76832{,}768 parameters), without residual conditioning, learnable gates, or bridge training. As shown in Table R1 and Fig. 8 of the revised version, this achieves 20.9220.92 dB PSNR, outperforming natural language (14.9914.99 dB) and text-encoder input optimization (15.5815.58 dB), while approaching the 226226-token text-encoder output prompt (22.0222.02 dB) with ∼28×\sim28\times fewer parameters. Since this compact prompt is shared across videos of the same degradation, it suggests that the prompt elicits restoration-relevant capabilities of the frozen backbone rather than independently learning the entire restoration mapping.

Dependence on the pretrained backbone. As suggested by the reviewer, we investigate the role of the pretrained prior by fine-tuning WAN on animated video clips, thereby altering its original prior. We then relearn the conditioning prompt under the same Fig. 2 setting (226226 tokens in text-encoder output). Despite retraining the prompt, REVIDE PSNR drops from 22.0222.02 to 18.9518.95 dB with the haze not removed effectively (see Fig. 8 of the revised version). This suggests that the restoration behavior elicited through conditioning depends on the original pretrained representations and was not recovered by prompt optimization alone under the same training protocol, indicating restoration-relevant capabilities in the pre-trained backbone.

Together, these experiments support our hypothesis that our prompts access restoration-relevant capabilities of the PTDM, rather than independently learning the complete restoration behavior. We have now clarified that ``intrinsic restoration capabilities'' refers to capabilities accessible through the conditioning interface of the PTDM (L78-80 of the revised version).

Table R1: Effect of conditioning capacity and pretrained backbone on REVIDE using WAN under the Fig. 2 setting, without residual conditioning, gates, or bridge training.

ConfigurationParamsPSNR↑SSIM↑
Natural Language014.990.698
TE input (226 tokens)925,69615.580.572
TE output (8 tokens)32,76820.920.722
TE output (226 tokens)925,69622.020.768
TE-output (226 tokens) on animated-video fine-tuned WAN925,69618.950.668
AuthorsOpenReview ↗

Response to Reviewer wYYi Contd.

  1. CW1, Q1, Q4; No comparison with prefix tuning or LoRA at matched budget; Gain over LoRA Bridge with partial-inversion.

As clarified above, our objective is to investigate restoration capabilities accessible through the frozen PTDM's conditioning pathway, rather than establish superiority over other adaptation methods. Nevertheless, we provide controlled comparisons to evaluate practical effectiveness. We acknowledge that the original LoRA Bridge baseline did not use partial inversion and thank the reviewer for pointing this out. We now evaluate it under identical inference settings (sampling steps, bridge schedule, and partial inversion), alongside parameter-matched LoRA and prefix-tuning baselines (Table R2).

Comparison with LoRA. Full LoRA, using substantially more trainable parameters, generally performs better in-distribution (ID), while our approach achieves stronger OOD performance on average, suggesting better generalization. At matched parameter budgets, our approach outperforms LoRA across most ID and OOD metrics, demonstrating the effectiveness of our approach while preserving the pretrained backbone weights.

Comparison with prefix tuning. As discussed above, our framework is closely related to prefix/deep prompt tuning, but uses a single conditioning prompt shared across transformer blocks, motivated by our investigation of a task-level restoration instruction. We now evaluate prefix-tuning (K/V embeddings at each block) which is more expressive, on WAN. Prefix tuning achieves stronger ID performance, including higher PSNR (23.51 vs. 23.30 dB), while our shared-prompt approach achieves higher OOD PSNR (24.94 vs. 24.28 dB), with competitive performance across the remaining metrics. These results indicate that more expressive prompt parameterizations can offer performance improvements, consistent with the extensions discussed in Sec. B.3. However, our central observation is that restoration behavior can be elicited through a shared conditioning signal in a frozen PTDM, without requiring independently learned block-wise prompts.

We have incorporated the updated LoRA changes along with the parameter-matched version in Table 4 and across the supplementary of the revised version. We have also discussed the comparisons with prefix tuning in Secs. B.1 and E.

Table R2: Comparison with LoRA bridges under identical inference settings (N=6 sampling steps and partial inversion), including parameter-matched baselines. The table also provides updated numbers for the Concat baseline and comparisons with prefix tuning. Best results are in bold among Ours, LoRA, and Concat baselines.

(a) In-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4525.700.8550.1100.0820.61666.98--
FLUX + Bridge (LoRA, matched)11,993.27.9423.240.8170.1490.1110.58365.60--
FLUX + Concat (LoRA)13,419.71,434.4619.960.5370.3820.2310.41252.44--
FLUX (Ours)11,994.77.8725.390.8430.1180.0900.57164.91--
WAN + Bridge (LoRA)1,633.487.4922.840.7820.1430.0950.58365.490.637
WAN + Bridge (LoRA, matched)1,547.61.7421.820.7600.1790.1220.53962.000.586
WAN + Concat (LoRA)1,633.487.5019.110.6160.2750.1810.49260.410.491
WAN (Ours)1,548.01.7423.300.7830.1480.1050.56064.260.627
WAN + Prefix tuning (matched)1,548.11.8423.510.7890.1370.0940.57264.670.632

(b) Out-of-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4520.360.7320.2520.1790.54558.73--
FLUX + Bridge (LoRA, matched)11,993.27.9417.950.6870.2960.2060.53157.81--
FLUX + Concat (LoRA)13,419.71,434.4618.340.5140.4680.2970.38846.73--
FLUX (Ours)11,994.77.8720.600.7480.2350.1740.53460.14--
WAN + Bridge (LoRA)1,633.487.4921.800.7840.1980.1360.54159.920.593
WAN + Bridge (LoRA, matched)1,547.61.7421.380.7570.2100.1540.51958.820.570
WAN + Concat (LoRA)1,633.487.5019.380.6050.3500.2370.46155.120.471
WAN (Ours)1,548.01.7424.940.7880.1910.1400.54160.120.599
WAN + Prefix tuning (matched)1,548.11.8424.280.7880.1840.1330.54759.960.599
AuthorsOpenReview ↗

Response to Reviewer wYYi contd.

  1. CW1; EBR bridge adopted from prior work, limiting methodological novelty.

We agree that the E-Bridge schedule is adopted from prior work and do not claim the bridge formulation itself as our contribution, as explicitly acknowledged in Sec. 3.2 (L259--261). Our contribution lies in identifying how an appropriate training trajectory enables the restoration behavior observed in Fig. 2 to be effectively leveraged for multistep sampling with a frozen PTDM. Specifically, we identify train-test trajectory mismatch as a key obstacle to naive conditioning prompt learning. While conventional DDBM provides a degraded-to-clean transition, its low noise near the degraded endpoint limits the restoration response (Fig. 3, Table 3). This motivates a bridge that maintains sufficiently noisy degraded states while progressively transitioning toward clean states with decreasing noise. E-Bridge satisfies these requirements, and our experiments establish its effectiveness in eliciting restoration from a frozen PTDM. Thus, our contribution is not a new bridge schedule, but identifying and addressing the trajectory requirements for eliciting restoration through a frozen PTDM's conditioning pathway.

  1. CW2, Q2; Fairness of comparisons is questionable; The method is non-blind while several baselines are blind; How do the baselines perform when given the degradation label?

Our primary backbone-matched LoRA baselines (Bridge and Concat) have access to the same degradation-type information through text instructions, enabling controlled comparisons. Among the broader SOTA baselines, DFPIR, PixWizard, and AutoDIR support text-based task instructions, which we provide during evaluation. Other methods, including DCPT, RestoreVAR, and SLAIR, operate without explicit degradation labels. Nevertheless, we include these approaches as they are state-of-the-art methods and serve as restoration-quality benchmarks for our approach. Thus, our backbone-matched experiments isolate the effectiveness of conditioning-based restoration under comparable task information, while the SOTA comparisons contextualize its practical performance.

  1. CW2, Q2; UDC degradation is manually chosen.

Since UDC is an unseen task, we compose existing degradation-specific prompts without additional training. Our choice of low-light+blur is motivated by UDC characteristics which include light attenuation and optical blur (L366), consistent with AutoDIR, which similarly identified UDC restoration as brightening and deblurring. We further investigate the sensitivity of UDC restoration to prompt selection in Fig. 12 in the revised version. The low-light prompt improves exposure but fails to correct blur, while mismatched prompts (haze or rain) provide limited to no restoration as expected. In contrast, composing the low-light and blur prompts improves both exposure and sharpness, demonstrating the importance of selecting appropriate degradation-specific prompts. We have incorporated this discussion in Sec. I.4 of the revised version.

  1. CW2, Q3; Hyper-parameter tuning effort spent on the LoRA concat baseline.

We carefully tuned LoRA-Concat to ensure stable training and strong performance for both backbones, using bf16 with learning rates of 10−510^{-5} for FLUX and 5×10−55\times10^{-5} for WAN. We also explored alternative learning rates and numerical precisions; higher learning rates with fp16 caused training divergence on FLUX, while lower learning rates yielded inferior performance on WAN. To assess convergence, we extended training substantially beyond the original budgets. On FLUX, increasing training from 50K to 125K steps improved PSNR from 17.2217.22 to 19.9619.96 dB. Further training to 162.5K steps yielded no meaningful improvement. Similarly, WAN improved from 15.4015.40 dB at 40K steps to 19.1119.11 dB at 100K steps, with no significant improvements beyond 100K. Thus, we now use the checkpoint trained at 125125K steps for FLUX and the checkpoint at 100100K steps for WAN, for our evaluations as they showed best performance.

Despite these extended training budgets, LoRA-Concat remains substantially below our approach (Table R2 above) on most metrics. Overall, our approach achieves stronger restoration with 5050--180×180\times fewer trainable parameters and substantially lower training cost, supporting the effectiveness of eliciting restoration from the frozen PTDM using our conditioning prompts. We have updated the results for the Concat baseline throughout the paper.

Reviewer_GAyfOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper explores whether frozen pretrained diffusion models can be adapted for image and video restoration through their native conditioning pathways. The authors show that optimizing conditioning representations after the text encoder is more effective than natural-language prompting or input-token optimization.

The method adopts an existing energy-oriented diffusion bridge to construct restoration trajectories, and introduces partial inversion for image-dependent initialization and velocity composition for mixed degradations. Experiments on FLUX and WAN demonstrate competitive restoration performance with relatively few trainable parameters.

Critical strengths
  1. I think the difference between input-token optimization and conditioning-space optimization is an interesting observation. Figure 2 provides evidence that optimizing representations after the text encoder can recover restoration behavior that is difficult to achieve with conventional prompting. The bridge ablation in Table 3 is also useful. EBR improves PSNR from 17.39 (naive training) and 20.57 (DDBM) to 21.86 dB, suggesting that the choice of training trajectory matters considerably for frozen-backbone restoration.

  2. The paper evaluates both FLUX and WAN across multiple degradation types, including OOD and mixed degradations. I appreciate the backbone-matched comparisons in Table 4. For example, the FLUX-based method achieves 20.60 dB OOD PSNR with 7.87M trainable parameters, compared with 19.64 dB and 1,434M trainable parameters for the LoRA Bridge baseline. The results demonstrate that conditioning-based adaptation can be effective with a much smaller trainable parameter budget.

Critical weaknesses

1.I think the paper demonstrates effective parameter-efficient adaptation rather than conclusively showing that restoration capabilities are intrinsic to pretrained diffusion models. The conditioning prompts are learned from paired degraded/clean images (Eq. 8), and Appendix D introduces trainable residual conditioning and layer-wise gates. Moreover, conditioning optimization for inverse problems has been explored by Chung et al. (ICML 2024), while the diffusion bridge is adopted from E-Bridge (ICLR 2026). I think the authors should better distinguish their contribution from existing prompt-based adaptation methods.

  1. Insufficient isolation of prompt learning and sampling strategies. Table 4 shows improvements over LoRA Bridge, but it is unclear whether the baselines use identical partial inversion and sampling settings. This is important because inversion alone improves PSNR by 1.81 dB for FLUX and 1.44 dB for WAN (Table 5), comparable to the reported gains over LoRA Bridge. I am also not fully convinced by the trajectory-alignment argument. Training states in Eq. (7) depend on ground-truth clean latents, while inference states depend on model predictions and input-dependent inverted noise (Eq. 9). Using the same bridge schedule does not guarantee matched state distributions. Table 3 demonstrates the effectiveness of EBR, but does not directly validate the claimed trajectory alignment.
Questions for authors
  1. Do the LoRA Bridge baselines in Table 4 use the same partial inversion (Eq. 9), sampling steps, and bridge settings as the proposed method? Given the substantial gains from inversion in Table 5, I think this is important for fairly attributing the improvements to prompt learning.

  2. Training states in Eq. (7) depend on ground-truth clean latents, whereas inference relies on model predictions and inverted noise. How do the authors justify the claimed train-test trajectory alignment? Is there any direct evidence beyond the performance improvements in Table 3?

  3. Since the prompts are learned from paired restoration data and injected through trainable residual conditioning and gates (Appendix D), what evidence distinguishes eliciting intrinsic restoration capabilities from conventional parameter-efficient adaptation? How does the method differ fundamentally from existing diffusion prompt-tuning approaches, such as Chung et al. (ICML 2024)?

5 comments
AuthorsOpenReview ↗

Response to Reviewer GAyf

We thank the reviewer for their constructive feedback and for recognizing our conditioning-space findings, parameter efficiency, and comprehensive evaluation. We address the concerns below.

  1. CW1, Q3; Paper demonstrates effective parameter-efficient adaptation rather than showing intrinsic restoration capabilities. Since prompts are learned from paired data, and Appendix D introduces trainable residual conditioning and learnable gates, where is the evidence for intrinsic restoration capabilities?

Our central observation in Fig. 2 is that a pretrained diffusion model (PTDM) trained for generation can be guided to predict clean latents from noisy degraded states through an appropriate conditioning signal. Intuitive natural-language instructions fail to elicit effective restoration, making it challenging to identify the required instruction. We therefore use paired degraded/clean data to identify this signal in the text-encoder output space, guiding the PTDM to denoise toward the clean latent rather than reconstruct the degraded content.

While learning this small conditioning signal can be viewed as parameter-efficient adaptation, our primary objective is not merely to introduce another adaptation method, but to investigate restoration behavior that is accessible through the PTDM's native conditioning pathway without modifying its pretrained backbone weights. To this end, we use learned prompts as a tool to probe and elicit restoration behavior from a frozen PTDM, analogous to how natural-language instructions can be used to guide generative diffusion models. Our goal is therefore to elicit restoration from the frozen PTDM itself, guided by a degradation-specific prompt shared across inputs. This differs from approaches that modify the backbone for restoration or leverage the PTDM as a clean-image prior with external guidance (Fig. 1). Crucially, Fig. 2 demonstrates that restoration can be elicited from the frozen PTDM without diffusion-bridge training, residual conditioning, or learnable gates. These observations motivate our hypothesis that the pretrained backbone possesses restoration-relevant capabilities accessible through conditioning, which we further investigate using limited-capacity prompts and backbone-modification experiments.

Elicitation with limited conditioning capacity. We repeat the direct conditioning experiment of Fig.2 on REVIDE using only 88 learnable tokens (32,76832{,}768 parameters), without residual conditioning, learnable gates, or bridge training. As shown in Table R1 and Fig. 8 of the revised version, this achieves 20.9220.92 dB PSNR, outperforming natural-language instructions (14.9914.99 dB) and text-encoder input optimization (15.5815.58 dB), while approaching the 226226-token text-encoder output prompt (22.0222.02 dB) with ∼28×\sim28\times fewer parameters. Since this compact prompt is shared across videos of the same degradation, it resembles a task-level instruction, suggesting that it elicits restoration-relevant capabilities of the frozen backbone.

Dependence on the pretrained backbone. As suggested by reviewer wYYi, we additionally investigate the role of the original pretrained prior by fine-tuning WAN on animated video clips, thereby altering its original prior. We then relearn the conditioning prompt under the same Fig. 2 setting (226226 tokens). Despite retraining the prompt, REVIDE PSNR drops from 22.0222.02 to 18.9518.95 dB (Table R1) with the haze not completely removed (Fig. 8 of the revised version). This suggests that the restoration behavior elicited through conditioning depends on the original pretrained representations and was not fully recovered by prompt optimization under the same training protocol, indicating restoration-relevant capabilities in the original pre-trained backbone.

Clarifying our contribution. Together, these experiments support our hypothesis that the learned conditioning prompts elicit restoration-relevant capabilities of the pretrained backbone rather than independently learning the complete restoration behavior. We have now clarified that ``intrinsic restoration capabilities'' refers to capabilities supported by the pretrained backbone and accessible through conditioning (L78 of the revised version). We have incorporated the additional experiments in Sec. 4.4 of the revised version.

Table R1: Effect of conditioning capacity and pretrained backbone on REVIDE using WAN under the Fig. 2 setting, without residual conditioning, gates, or bridge training.

ConfigurationParamsPSNR↑SSIM↑
Natural Language014.990.698
TE input (226 tokens)925,69615.580.572
TE output (8 tokens)32,76820.920.722
TE output (226 tokens)925,69622.020.768
TE-output (226 tokens) on animated-video fine-tuned WAN925,69618.950.668
AuthorsOpenReview ↗

Response to Reviewer GAyf contd.

  1. CW1, Q3; Differences from Chung et al. [1] and E-Bridge (EBR).

Differences with Chung et al. While both approaches optimize conditioning embeddings with a frozen PTDM, their objectives differ. Chung et al. use the PTDM as a clean-image generative prior, initializing reverse diffusion from pure Gaussian noise and optimizing the prompt separately for each observed image at test time. The prompt guides generation toward a reconstruction consistent with the degraded input using a known degradation operator.

In contrast, we learn a single task-level prompt per degradation from multiple images, which is then shared across all inputs of that degradation. Rather than adapting the prompt to reconstruct each individual observation, our objective is to identify a conditioning instruction that enables the frozen PTDM itself to restore noisy degraded states. Consequently, our method requires neither test-time prompt optimization nor a known degradation operator or external measurement-consistency guidance. We have now discussed Chung et. al. in L159-161 of the revised version.

Adoption of E-Bridge. Starting from our observation in Fig. 2 that a frozen PTDM can be conditioned to restore noisy degraded states, we identify the train-test trajectory mismatch as a key obstacle to utilizing this behavior in multistep sampling, motivating the use of a diffusion bridge. However, conventional DDBM, while providing the desired degraded-to-clean transition, lacks sufficient noise near the degraded endpoint to elicit the restoration response (Fig. 3, Table 3). We therefore identify the need for a bridge that both preserves sufficiently noisy degraded states and progressively transitions toward clean states with decreasing noise. Since E-Bridge already satisfies these requirements, we adopt its schedule as the missing component needed to effectively leverage our initial observation. We explicitly acknowledge E-Bridge as the source of this schedule in Sec. 3.2 (L 259-261), and do not claim the bridge formulation itself as our contribution.

  1. CW2, Q1; Do the LoRA Bridge baselines use the same settings?

As clarified above, our central objective is to investigate restoration behavior accessible through the frozen PTDM's conditioning pathway, rather than establish prompt tuning as a superior alternative to LoRA. We thank the reviewer for raising the partial inversion question for the LoRA bridge runs. The original LoRA Bridge comparisons did not use partial inversion. We now evaluate LoRA Bridge under identical inference settings (sampling steps, bridge schedule, and partial inversion), and additionally include parameter-matched LoRA baselines (Table R2). Full LoRA, using substantially more trainable parameters, generally performs better in-distribution (ID), while our approach achieves stronger OOD performance on average, suggesting improved generalization. At matched parameter budgets, our approach outperforms LoRA across most ID and OOD metrics, demonstrating the effectiveness of our approach while preserving the pretrained backbone weights. These results are included in Table 4 and the supplementary of the revised version.

Table R2: Comparison with LoRA bridges under identical inference settings (N=6 sampling steps and partial inversion), including parameter-matched baselines. Best results are in bold.

(a) In-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4525.700.8550.1100.0820.61666.98--
FLUX + Bridge (LoRA, matched)11,993.27.9423.240.8170.1490.1110.58365.60--
FLUX (Ours)11,994.77.8725.390.8430.1180.0900.57164.91--
WAN + Bridge (LoRA)1,633.487.4922.840.7820.1430.0950.58365.490.637
WAN + Bridge (LoRA, matched)1,547.61.7421.820.7600.1790.1220.53962.000.586
WAN (Ours)1,548.01.7423.300.7830.1480.1050.56064.260.627

(b) Out-of-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4520.360.7320.2520.1790.54558.73--
FLUX + Bridge (LoRA, matched)11,993.27.9417.950.6870.2960.2060.53157.81--
FLUX (Ours)11,994.77.8720.600.7480.2350.1740.53460.14--
WAN + Bridge (LoRA)1,633.487.4921.800.7840.1980.1360.54159.920.593
WAN + Bridge (LoRA, matched)1,547.61.7421.380.7570.2100.1540.51958.820.570
WAN (Ours)1,548.01.7424.940.7880.1910.1400.54160.120.599
AuthorsOpenReview ↗

Response to Reviewer GAyf Contd.

  1. CW2, Q2; Evidence for trajectory-alignment

We now conduct experiments to substantiate the trajectory-alignment. Recall that naive prompt learning (Eq. 1) constructs training states centered around the degraded latent, whereas inference progressively moves toward clean content. In contrast, EBR (Eq. 7) constructs training states that progressively transition from degraded to clean latents, reducing this mismatch. To verify this, we measure the latent-space Frechet distance (FD) between the actual noisy states at different timesteps encountered during inference and reference training states constructed from 42 REVIDE training videos using the respective formulations. We evaluate WAN with the trained prompts on both test and training videos to distinguish trajectory mismatch from differences in video distributions.

As shown in Table R3, naive sampling initially aligns with its training distribution but diverges substantially as sampling progresses. At t≈0.08t\approx0.08, naive yields FDs of 9.759.75 (test) and 9.889.88 (train), compared with 2.302.30 and 1.531.53 for EBR. Importantly, the divergence persists on training videos, indicating that it is not primarily caused by generalization to unseen videos. Partial inversion achieves similarly low FDs (2.382.38/1.651.65), indicating that it largely preserves EBR's distributional alignment, despite introducing a small initial difference due to its input-dependent starting state which may not be perfect due to model-prediction errors.

These results provide evidence that EBR substantially reduces the train-test trajectory mismatch encountered during naive multistep sampling. This experiment is included in Sec. 4.4 of the revised version, with the trajectory mismatch illustrated by Fig. 7.

Table R3: Train-inference trajectory alignment on REVIDE using WAN. We report the latent-space Fréchet distance (FD) between the noisy intermediate states encountered during inference and the training-state references. Lower is better.

Test videos

Methodt=0.40t=0.32t=0.24t=0.16t=0.08
Naive1.5361.8632.9835.2219.750
EBR1.5421.7921.9982.1572.297
EBR + inversion1.9322.0242.1282.2402.378

Training videos

Methodt=0.40t=0.32t=0.24t=0.16t=0.08
Naive0.6281.2712.7205.2539.875
EBR0.6290.7880.9811.2211.526
EBR + inversion1.0361.0451.1381.3301.647
Reviewer_1xknOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates whether frozen pretrained diffusion models can perform restoration through learned conditioning prompts shared across inputs. It first compares different conditioning representations, then introduces bridge-based prompt training to address sampling instability. Partial inversion and velocity composition further improve initialization and mixed-degradation restoration. Experiments cover FLUX-based image restoration and WAN-based video restoration.

Critical strengths
  1. The central finding is useful: degradation-specific conditioning prompts can enable competitive restoration without updating the pretrained backbone. Demonstrating this on both FLUX and WAN strengthens the result.
  2. The method is well motivated. The conditioning-space study establishes a restoration response, while the subsequent bridge experiments address the difficulty of turning that response into stable sampling. Table 3 supports the bridge choice, and Table 5 shows consistent benefits from partial inversion.
  3. The evaluation is broad, covering multiple degradation types, out-of-distribution datasets, and mixed or unseen degradations. The backbone-matched comparisons and component ablations provide useful evidence beyond comparisons with unrelated restoration architectures.
Critical weaknesses
  1. The contribution of initialization to the backbone-matched results needs clarification. For FLUX, prompt restoration without inversion reaches 23.58 dB, compared with 23.65 dB for LoRA Bridge; the full method reaches 25.39 dB. Whether the LoRA baseline uses the same inversion procedure therefore matters for interpreting Table 4.
  2. The fidelity–quality trade-off is not fully explained. On BSD, WAN improves no-reference quality scores but trails DFPIR in both PSNR and LPIPS. The VAE bottleneck is a plausible factor, but its contribution is not quantified, making it difficult to separate reconstruction limitations from restoration errors.
Questions for authors
  1. Do the LoRA bridge variants use the same partial inversion, sampling schedule, and inference budget as the proposed method?
  2. Can VAE reconstruction results or aligned ground-truth comparisons help explain the fidelity gap on datasets such as BSD?
  3. What are the explicit bridge sampling and partial inversion updates corresponding to the “DDIM-like” procedure?
5 comments
AuthorsOpenReview ↗

Response to Reviewer 1xkn

We thank the reviewer for their insightful comments and for recognizing the usefulness of our central finding, the motivation behind our method, and the breadth of our evaluation. We now address the concerns raised.

  1. CW1, Q1; Did the backbone-matched LoRA Bridge variants use same inference settings?: We thank the reviewer for pointing this out. The original LoRA Bridge comparisons did not use partial inversion. We now evaluate LoRA Bridge under identical inference settings (sampling steps, bridge schedule, and partial inversion), and additionally include parameter-matched LoRA baselines (Tables R1(a) and (b)). Full LoRA, using substantially more trainable parameters, generally performs better in-distribution (ID; Table R1(a)), while our approach achieves stronger OOD (Table R1(b)) performance on average indicating better generalization. At matched parameter budgets, our approach outperforms LoRA across most ID and OOD metrics, demonstrating the effectiveness of our approach while preserving the pretrained backbone weights. These results are included in Table 4 and the supplementary of the revised version.

Table R1: Comparison with LoRA bridges under identical inference settings (N=6 sampling steps and partial inversion), including parameter-matched baselines. Best results are in bold.

(a) In-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4525.700.8550.1100.0820.61666.98--
FLUX + Bridge (LoRA, matched)11,993.27.9423.240.8170.1490.1110.58365.60--
FLUX (Ours)11,994.77.8725.390.8430.1180.0900.57164.91--
WAN + Bridge (LoRA)1,633.487.4922.840.7820.1430.0950.58365.490.637
WAN + Bridge (LoRA, matched)1,547.61.7421.820.7600.1790.1220.53962.000.586
WAN (Ours)1,548.01.7423.300.7830.1480.1050.56064.260.627

(b) Out-of-distribution (mean)

MethodTotal (M)Train. (M)P↑S↑L↓D↓C↑M↑Do↑
FLUX + Bridge (LoRA)13,419.71,434.4520.360.7320.2520.1790.54558.73--
FLUX + Bridge (LoRA, matched)11,993.27.9417.950.6870.2960.2060.53157.81--
FLUX (Ours)11,994.77.8720.600.7480.2350.1740.53460.14--
WAN + Bridge (LoRA)1,633.487.4921.800.7840.1980.1360.54159.920.593
WAN + Bridge (LoRA, matched)1,547.61.7421.380.7570.2100.1540.51958.820.570
WAN (Ours)1,548.01.7424.940.7880.1910.1400.54160.120.599
  1. CW2, Q2; Explaining the fidelity gap on datasets such as BSD. We investigate whether the fidelity gap arises from VAE reconstruction or generative restoration. Reconstructing the BSD ground truth through WAN's VAE yields 34.0634.06 dB PSNR, indicating that the VAE bottleneck alone does not explain the gap. Interestingly, similar fidelity gaps are observed across other generative restoration methods on BSD (Table R2). PixWizard, RestoreVAR, SLAIR, AutoDIR without its regression-based structural-correction module and our backbone-matched LoRA baseline all exhibit lower PSNR and SSIM than DFPIR. Our method exhibits a similar pattern. This suggests a recurring fidelity--perceptual-quality trade-off in generative restoration, where perceptually plausible details may deviate from the ground truth, consistent with the perception--distortion trade-off [1]. We have incorporated this explanation in the revised manuscript (L420).

Table R2: Fidelity and perceptual quality on BSD. AutoDIR is evaluated without its regression-based structural-correction module (SCM).

MethodPSNR↑SSIM↑LPIPS↓CLIP-IQA↑MUSIQ↑
DFPIR27.760.8600.1740.37239.62
PixWizard19.710.6570.3290.36635.44
RestoreVAR21.930.7440.2070.40143.19
SLAIR21.460.7440.2050.39938.58
AutoDIR w/o SCM25.090.7730.2030.43045.15
WAN + Bridge (LoRA)19.740.7410.2480.38141.51
Ours24.330.7800.2020.41047.74

[1] Blau, Yochai, and Tomer Michaeli. "The perception-distortion tradeoff." 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition. IEEE, 2018.

AuthorsOpenReview ↗

Response to Reviewer 1xkn (Contd.)

  1. Q3; Bridge sampling and partial inversion procedure for DDIM-like procedure.

We provide the explicit details for our partial inversion and DDIM-like bridge sampling procedure below.

Partial inversion. The objective of partial inversion is to obtain an input-dependent noisy starting state instead of using randomly sampled noise. Starting from the degraded latent x0=zdegx_0=z^{\mathrm{deg}}, we perform KK Euler steps under null-text conditioning, progressively increasing the noise level from tmin⁡t_{\min} to T0T_0:

v_\theta(x_k,s_k;\varnothing).$$ Here, $s_k=t_{\min}+\frac{k}{K}(T_0-t_{\min})$ denotes the equally spaced inversion timesteps, and $s_{k+1}-s_k$ is the Euler step size. After $K$ inversion steps, the resulting state $x_K$ represents the input-dependent noisy latent at noise level $T_0$. We use this state to initialize bridge sampling, corresponding to the inverted noise $\hat\epsilon=(x_K-(1-T_0)z^{\mathrm{deg}})/T_0$ in Eq. 9. **Bridge sampling.** Starting from $z_{T_0}^{\mathrm{ebr}}=x_K$, we progressively restore the degraded latent over $N$ sampling timesteps, decreasing the noise level from $T_0$ to $t_{\min}$. Recall that our EBR formulation (Eq. 7) constructs intermediate states by interpolating between the degraded and clean latents while gradually reducing noise. At each sampling timestep $t_i$, we first predict the clean latent $\hat z_0$ using the learned conditioning prompt $p$. Since the ground-truth clean latent is unavailable during inference, we substitute this prediction into Eq. 7 and rearrange it to estimate the noise $\hat\epsilon_i$ corresponding to the current state. We then construct the next state $z_{t_{i+1}}^{\mathrm{ebr}}$ using the predicted clean latent and estimated noise at the next lower noise level. Concretely, the updates are: $$\begin{aligned} \hat z_0 &= z_{t_i}^{\mathrm{ebr}} -t_i v_\theta(z_{t_i}^{\mathrm{ebr}},t_i;p),\\\\[4pt] \hat\epsilon_i &= \frac{1}{t_i}\Bigg[ z_{t_i}^{\mathrm{ebr}} -(1-t_i)\left( \left(1-\frac{t_i}{T_0}\right)\hat z_0 +\frac{t_i}{T_0}z^{\mathrm{deg}} \right)\Bigg],\\\\[4pt] z_{t_{i+1}}^{\mathrm{ebr}} &= (1-t_{i+1})\left[ \left(1-\frac{t_{i+1}}{T_0}\right)\hat z_0 +\frac{t_{i+1}}{T_0}z^{\mathrm{deg}} \right] +t_{i+1}\hat\epsilon_i. \end{aligned}$$ Here, $t_i=T_0-\frac{i}{N-1}(T_0-t_{\min})$ denotes the equally spaced sampling timesteps, for $i=0,\ldots,N-1$. This DDIM-like procedure uses the same EBR state construction as training, with the ground-truth clean latent replaced by the model's prediction. After the final timestep, we obtain the restored clean latent from the model's clean prediction at $t_{\min}$. As mentioned in the paper, we use $T_0=0.4$, $t_{\min}=10^{-4}$, and $N=6$ sampling steps for both backbones, with $K=10$ inversion steps for FLUX and $K=6$ for WAN. We have incorporated these details in Sec.C.3 of the revised manuscript.

Dear Reviewer 1xkn,

As the discussion period is nearing its end, we would greatly appreciate your feedback on whether our responses address your concerns, so that we can promptly address any remaining questions before the deadline.

Thank you for your time and consideration.

Comments on the paper

2 comments

We thank all reviewers for their valuable feedback and suggestions. We have provided detailed responses to the concerns raised and revised the manuscript accordingly. We summarize the key updates below:

  1. Clarification of our central contribution: We have clarified the interpretation of intrinsic restoration capabilities and the relationship of our approach to parameter-efficient fine-tuning and prefix/deep prompt tuning.

  2. Additional evidence for restoration capabilities: We have introduced experiments with limited-capacity conditioning prompts and a modified pretrained backbone to further investigate whether restoration-relevant capabilities can be elicited through compact conditioning signals and how this behavior depends on the pretrained backbone.

  3. Expanded backbone-matched comparisons: We have re-evaluated LoRA Bridge baselines under identical inference settings, introduced parameter-matched LoRA comparisons, and extended the training budgets of Concat baselines.

  4. Trajectory-alignment analysis: We have added quantitative evidence measuring the discrepancy between training and inference state distributions, alongside explicit details of our partial inversion and DDIM-like bridge sampling procedures.

In addition to these key updates, the rebuttal provides several additional experiments and detailed clarifications.

We hope that these revisions and our detailed responses address the reviewers' concerns. We appreciate their time and consideration and welcome further discussion.

Authors of Submission 16591

Dear reviewers,

The authors have submitted their rebuttal. Please review their responses and provide any additional comments or updates. I would appreciate your engagement with the discussion.

Thanks, AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.