acceptodds
Withdrawn from ICLR 2027

One-Step Diffusion Inpainting via Reconstruction-Space Adaptation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Diffusion models have shown strong performance in image inpainting by leveraging powerful generative priors, but their iterative denoising process often incurs substantial inference cost. One-step diffusion models provide a promising alternative, yet removing iterative denoising also removes the opportunity for progressive error correction, allowing prediction errors produced in a single step to more directly affect the final reconstruction. In this work, we investigate how prediction errors propagate into reconstruction errors under different prediction targets and, more importantly, whether lower theoretical error propagation translates into better downstream adaptation. Although velocity prediction has lower theoretical error amplification than noise prediction under the adopted noise schedule, controlled experiments show that replacing the prediction target used during pretraining with velocity prediction degrades downstream adaptation performance under the evaluated training budget. This reveals a discrepancy between theoretical error propagation and practical adaptation performance. Motivated by this observation, we propose reconstruction-space adaptation for one-step diffusion inpainting. Instead of changing the pretrained prediction target, the proposed method preserves noise prediction while extending supervision from the prediction space to the reconstructed latent representation and decoded image, thereby aligning downstream optimization more directly with the final restoration objective. Independent-sample averaging and a lightweight residual refiner are further used as complementary components to reduce sampling-induced output fluctuations and correct remaining image-space errors. Extensive experiments on Places2 and CelebA-HQ validate the proposed adaptation strategy, showing consistent reconstruction improvements under controlled adaptation settings and competitive inpainting performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Image Inpainting, One-Step Diffusion, Reconstruction-Space Adaptation, Prediction Target, Sampling Variance

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_WSzuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The authors propose to improve the inpainting performance of one-step diffusion models using a multi-stage approach: first, the reconstruction loss is applied to different parts of output (by learning to reconstruct the noise ϵ\epsilon, the clean latent code z0z_0, and the pixel-space image xx). Second, the inpainting model output is further refined with a residual U-Net. Finally, multiple realisations of the inpainting are averaged to produce the final output.

Critical strengths
  • The method is easy to understand, and the main figure is clear.
  • The tackled problem is interesting, as the coherence between the inpainted region and the original region has been an important topic, and generating in one step makes inpainting models usable in real editing tools.
Critical weaknesses
  • The experiments need to be deepened. Multiple one-step diffusion model baselines are cited in the related works, but the only diffusion-based baseline is StrDiffusion, and it is not single-step if I am not mistaken. The authors introduce multiple one-step/few-step diffusion models without comparing against them (starting at line 62). Also, the authors never compare the speed of their model against any other model.
  • The authors state in the paper that the inpainting is one-step, but the method relies on multiple inferences to produce an average (Figure 1 C). This does not really make the model one-step if multiple inferences are required.
  • The justification for the independent-sample inference is weak (Section 2.5, line 295). Fluctuations are necessary to have diversity in the outputs, which is important for an inpainting task.
  • The Generator-only (without Residual U-Net) performance (Table 4) is on par with GraphFill (25.80 dB vs. 25.73 dB), making the claim about reconstruction-space adaptation weak.
  • I would like the authors to describe the different terms of Equation (13) in more detail. Aren't the first and second terms measuring the exact same error, but weighted differently (taking into account the mask size in the second term)? What is the contribution of the first term in this case? Could they just be merged? Either way, the authors need to describe each term and its relative utility.
  • I would like to know how the K sample averaging impacts the diversity of the inpainting. From the paragraph at line 295, it seems that the averaging might get rid of part of the diversity.
  • The error needs to be detailed for the different types of masks. Results for different mask shapes, such as those in RePaint [a], need to be reported.
  • In Table 5, the authors detail inpainting metrics for different K (= NFE); they also need to compare the performance for K diffusion steps.

Overall, I think that the manuscript needs further work before being submitted again. [a] RePaint: Inpainting using Denoising Diffusion Probabilistic Models

Questions for authors
  • Is the initial assumption well motivated? While at t=999 the velocity prediction has a lower theoretical reconstruction-error amplification than the noise prediction, it should also be taken into account that the noise may be easier to predict at this same t=999: since the noisy input xt=999x_{t=999} is mostly dominated by the noise ϵ\epsilon, the noise will be easier to predict than the velocity. This might make the motivation overstated.
Reviewer_MP5MOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper presents studies on one-step diffusion inpainting, exploring how different choices of prediction target (noise, velocity, data) affects error propagation into reconstructions and, how this relates to practical downstream adaptation of a pre-trained diffusion model. The authors propose two stage training. Stage A extends to reconstruction-space adaptation for one-step diffusion inpainting while still retains the pretrained noise-prediction target. Stage B trains a lightweight residual refiner to correct remaining errors. An additional averaging sampling strategy is also applied to reduce output fluctuations. Extensive experiments on Places2 and CelebA-HQ are provided to demonstrate the effectiveness of proposed method.

Critical strengths

None

Critical weaknesses
  • The paper writing looks very hard to follow:
    • It is still unclear how theoretical reconstruction-error is defined throughout the entire paper.
    • It is unclear how reconstruction-error connects with multi-step sampling and how resolving this could help improve performance on one-step inpainting.
    • Eq.16 seems incomplete and there could be flaw, what is Cov ? More interpretation of this formula should be provided.
  • The paper diagram lacks clarity and looks very unpolished:
    • The text looks corrupted with weird font and color. I think this looks very similar to AI-generated diagram (possibly ChatGPT). However, the authors does not discuss using AI to generate figure and diagram in the AI Use Statement, which is a critical isssue.
    • The L_hole in B component has no text definition in the manuscript
    • What does it mean by algebra in block A?
    • There is no text referring to Figure 1 in the manusript.
  • Informal math notation in Eq. 4, using RMS in Eq. 4 looks weird.
  • No related work and no limitation discussed
  • Several text artifacts: Eq. equation 2 (line 241), Eq. equation 3 (line 257), …
Questions for authors

I don't have any answers that could potentially change my assessment on this paper. This manuscripts still look unpolished and it needs significant revision on both writing and conceptual method design.

Reviewer_yZ5JOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes a reconstruction-space adaptation method for one-step diffusion image inpainting. The authors analyze how different prediction targets propagate error theoretically and affect downstream adaptation. The method retains the pretrained noise prediction target while adding latent- and image-space reconstruction supervision. They further introduce a residual refiner and independent-sample averaging to improve reconstruction quality and stability. Experiments on Places2 and CelebA-HQ show the effectiveness of the proposed approach.

Critical strengths
  1. The paper is overall well organized and easy to follow.
  2. It investigates the discrepancy between theoretical error propagation and practical downstream adaptation. This is a meaningful task for one-step diffusion inpainting.
  3. The proposed reconstruction-space adaptation solution is sound and reasonable.
  4. The paper provides quantitative and qualitative comparisons, along with ablation studies to evaluate the proposed method.
Critical weaknesses
  • The claimed discrepancy between theoretical error propagation and practical performance is not clearly supported. The theoretical comparison assumes equal prediction errors across different targets, which may not hold in practice. However, the authors compare only theoretical error amplification factors without examining actual prediction errors. Because reconstruction error depends on both prediction error and its amplification factor, the observed performance difference does not clearly show a discrepancy between theory and practice. Comparing actual prediction and reconstruction errors across different targets may help.
  • In Lines 260–263, the authors state that the latent reconstruction loss does not introduce an independent prediction target but changes the effective scale and loss geometry. However, Eq. (8) also introduces a manually chosen weight of 0.2 for the noise prediction loss, without explaining its choice. This raises some concerns about the theoretical justification of the proposed loss design, as its effectiveness may depend on heuristic loss weighting.
  • The comparison between noise prediction and velocity prediction is conducted using a model pretrained with noise prediction. Therefore, the inferior performance of velocity prediction may reflect a mismatch between the pretraining and adaptation targets, rather than the prediction target itself. This makes it difficult to connect the theoretical error-propagation analysis to the experimental findings.
  • The baseline selection raises two concerns. First, Tables 1 and 2 include different baseline sets without explaining why. Second, most of the selected baselines are not directly related to the paper's focus on one-step diffusion inpainting. The paper lacks comparisons with more relevant few-step and one-step diffusion inpainting methods.
Questions for authors
  • The paper does not explain why prediction-target selection needs to be reconsidered during downstream adaptation. What motivates changing the prediction target of a pretrained diffusion model in the first place?
  • What are the key differences between the proposed method and existing few-step and one-step diffusion inpainting methods? In particular, the underlying design principles and methodological novelty of the proposed approach are not clear.
  • For further questions, please refer to the Critical Weaknesses. I would also appreciate any clarification if I have misunderstood any aspects of the paper.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.