acceptodds
Withdrawn from ICLR 2027

Moonwalker: Learning Accurate Inversion Steps in Diffusion Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Standard DDIM inversion attempts to recover the noise underlying an image, but relies on a local approximation of the reverse process whose errors accumulate across steps. As a result, the recovered latent remains entangled with information from the source image. We introduce Moonwalker, a lightweight solution that corrects this approximation by learning ”forward” inversion directly from backward generation steps. Specifically, we train a LoRA adapter to take a cleaner latent and predict the exact previous noise from the sampling trajectory, perfectly retracing the path to the correct noisier state. The adapter is activated only during inversion and deactivated during generation, leaving the original diffusion model and sampling process unchanged and incurring no inference overhead. Moonwalker reduces the correlation between recovered latents and their source images, and we evaluate its effect on downstream performance across image content, style, and music editing benchmarks.

Primary area:
generative models
Keywords:
diffusion models, inversion, image editing, lora

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_vP5TOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

The paper introduces Moonwalker, a framework designed to learn accurate inversion steps in diffusion models by incorporating a LoRA adapter specifically within the forward inversion trajectory to minimize DDIM errors. The proposed adapter is trained in a self-supervised manner on model-generated outputs using a combination of trajectory matching, cycle consistency, and CFG loss terms. The authors assess their approach across various tasks, including image reconstruction, image editing, audio editing, and style transfer.

Critical strengths
  • The paper is well-written and easy to understand, though its organization could be enhanced (see weaknesses).
  • The qualitative style transfer results look promising.
Critical weaknesses
  • Evaluation of the image reconstruction experiments (sections 4.2 and 5.2) relies exclusively on synthetic inputs derived from ReCap-COCO captions, benchmarking strictly against standard DDIM inversion. Testing on authentic, real natural images is required to verify robustness of the method. Additionally, key baselines such as ReNoise [1], PreciseInv [2], and FlowOpt [3] are omitted from the comparison.
  • For the image editing evaluations on PIE-Bench (sections 4.3 and 5.2), the paper lacks comparisons with relevant diffusion approaches (e.g., DDPM Inversion [4], CycleDiffusion [5], Null-Text Inversion [6]) as well as flow-based methods (e.g., FireFlow [7], FlowEdit [8]).
  • The motivation for training an optimization-based LoRA module remains questionable, especially since training-free alternatives like DDPM Inversion yield comparable or better audio editing outcomes without the associated computational overhead.
  • Qualitative editing results are restricted to merely four examples. Furthermore, visible artifacts emerge in the edited outputs, such as in the cake in Figure 2 (bottom-right panel).

Minor Issues:

  • In section 3.3 title, check whether "Edition" ought to be corrected to "Editing".
  • Section 5.2 lacks a reference regarding quantitative comparison using FLUX on PIE-Bench, alongside a complete lack of visual qualitative figures for FLUX.
  • The organization of Section 3 is somewhat confusing - section 3.3 feels misplaced in the middle and might be better placed at the beginning as an overview or positioned at the end.

[1] Garibi, Daniel, et al. "Renoise: Real image inversion through iterative noising." (ECCV’24)

[2] Zuo, Jing, et al. "Precise diffusion inversion: Towards novel samples and few-step models." (NeurIPS’26)

[3] Ronai, Or, et al. "FlowOpt: Fast Optimization Through Whole Flow Processes for Training-Free Editing." (arXiv’25)

[4] Huberman-Spiegelglas, Inbar, et al. "An edit friendly ddpm noise space: Inversion and manipulations." (CVPR’24).

[5] Wu, Chen Henry, and Fernando De la Torre. "A latent space of stochastic diffusion models for zero-shot image editing and guidance." (ICCV’23).

[6] Mokady, Ron, et al. "Null-text inversion for editing real images using guided diffusion models." (CVPR’23).

[7] Deng, Yingying, et al. "Fireflow: Fast inversion of rectified flow for image semantic editing." (ICML’25).

[8] Kulikov, Vladimir, et al. "Flowedit: Inversion-free text-based editing using pre-trained flow models." (ICCV’25)

Questions for authors

See weaknesses.

Reviewer_cEw4OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Moonwalker learns to correct deterministic diffusion inversion by training a LoRA adapter on adjacent states from a frozen generator's own sampling trajectories. Given the cleaner state, the adapted predictor regresses the noise or velocity evaluated at the preceding noisier state. The adapter is enabled during inversion and disabled during reconstruction or editing; the paper also studies cycle regularization, guided training, and flow matching. Experiments cover SDXL reconstruction, SD1.5 editing and style transfer, FLUX inversion/editing, and two audio generators.

Critical strengths

The supervision is well aligned with the local error being addressed. Cached trajectory predictions provide explicit targets without requiring real-image/noise pairs, and restricting adaptation to inversion preserves the original generator and its editing interface.

There are substantial improvements in identifiable settings.

Critical weaknesses
  1. The claimed exactness and prior matching exceed what the objective or evidence establishes. Equations 4–5 give the exact inverse of a sampled step only if the learned predictor recovers that step's teacher output exactly. Finite regression error on teacher-forced synthetic states does not establish this condition on recursively inverted states or real inputs. There is no invertibility or rollout-error guarantee. Patch correlation and a Gaussian KL diagnostic also do not establish the full joint standard-Gaussian distribution or removal of all spatial dependencies. Residual errors remain: Table 9 reports a plain-region noise standard deviation of 0.9479 for rank 32. More importantly, Table 17 shows that a separately trained FLUX adapter at matched guidance 3.5 improves the reported KL from 3.9522 to 0.8322 while worsening PSNR from 11.20 to 9.47 and target CLIP from 73.917 to 67.161.

  2. The comparative evidence does not establish the value of this particular learned inversion design. The paper discusses DeepInv, FlashIn, and BiFM, but does not evaluate a learned inversion baseline. DeepInv already trains a stepwise solver to align inversion predictions with the frozen denoiser; FlashIn uses generated data and cycle consistency, while BiFM learns both directions. Moonwalker's distinction—cached adjacent-step supervision with an inversion-only adapter and an unchanged multi-step generator—is meaningful, but its accuracy/cost advantage over these alternatives remains unmeasured.

  3. The downstream benefit is concentrated and its most decisive control needs clarification. In Table 2, P2P obtains a clear gain, but DirectInv retains stronger preservation for PnP, MasaCtrl, and Pix2Pix-Zero, with small or mixed alignment changes. Table 5 usefully separates reconstruction from editing, yet its P2P DirectInv reconstruction is only 19.65 dB, versus approximately 27.3 dB for the other DirectInv/editor configurations; Moonwalker+DirectInv brings P2P to 27.28 dB. The paper should explain this discrepancy and verify the P2P control before attributing the large gain to general inversion accuracy. Table 2 selects different ranks/training variants by editor without an explicit held-out selection rule in Appendix A. Meanwhile, style gains are small and mixed (Tables 3–4), and AudioLDM2 fronts essentially coincide (Figure 3). Paired uncertainty estimates and a clear configuration-selection protocol are needed to establish which improvements are robust beyond the favorable P2P setting.

  4. The practical motivation for an inversion-first editing pipeline is insufficiently established in the context of native generation-and-editing models. FLUX.1 Kontext learns image generation conditioned jointly on text and reference-image tokens, while Qwen-Image studies a shared T2I/TI2I/I2I framework and its editing models use semantic and VAE image representations as conditions. Their standard editing pipelines generate target latents from noise without first recovering the source image's DDIM/ODE trajectory. In contrast, the main image-editing evaluation here uses SD1.5 with P2P, PnP, MasaCtrl, and Pix2Pix-Zero; the FLUX.1-dev inversion experiment does not evaluate a native editor such as Kontext.

Questions for authors

Can you replace the exactness/guarantee claims with a conditional statement and report per-step prediction errors and accumulated rollout errors on held-out generated and real inputs? How do you explain the decoupling of KL and reconstruction/editing in the matched-guidance FLUX experiment?

Can you compare with a learned solver such as DeepInv on a shared backbone and an appropriate iterative inversion baseline, reporting accuracy, NFE, latency, peak memory, adapter merge/switch overhead, and training/cache cost?

Why does the P2P DirectInv source-reconstruction control differ so substantially from the other controls?

Reviewer_n9ytOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Moonwalker is a LoRA adapter on a frozen diffusion or flow model. DDIM inversion assumes ε(x_t, t) ≈ ε(x_{t-1}, t), and the errors accumulate. The adapter is trained on the model’s own cached noise predictions along deterministic trajectories, to predict ε(x_t) from the cleaner x_{t-1} (Eq. 4), with an optional cycle term (Eq. 6). It is active only during inversion. The paper tests noise recovery on SDXL (Table 1), PIE-Bench editing (Table 2), style transfer, music editing, and CFG and FLUX variants. It claims the method “perfectly retraces” the trajectory and improves editing.

Critical strengths
  • The idea is simple and well motivated . The model’s own trajectories give free supervision, and the adapter can be merged away.
  • Table 1 shows real gains in recovered-noise statistics (KL 0.71 to ~0.05, PSNR 28.2 to 30.4). The finding that the first 2–5 steps give most of the gain is interesting.
  • The appendix reports negative results honestly (Tables 11, 14, and DirectInv in Table 2).
Critical weaknesses
  • Some cliams like being “exact” are not supported. Residual patch correlation is still ~0.10 and PSNR is ~30 dB. No reference value is given for i.i.d. Gaussian noise on this statistic, so “matches the prior” can’t be judged.
  • Weak link to better editing. Standalone Moonwalker is nearly identical to DDIM on preservation for P2P, PnP and MasaCtrl (e.g., P2P structure 70.45 vs 70.50). CLIP gains are ~0.2 or less with no confidence intervals. With DirectInv it is worse than DirectInv alone on PnP, MasaCtrl and Pix2Pix-Zero (e.g., MasaCtrl structure 28.6 vs 23.6).
  • The main P2P win looks like a baseline artifact. In Table 5, the DirectInv+P2P control has PSNR 19.65 and structure 38.8 (worse than the same control’s edited output in Table 2, structure 31.35, which is implausible), while the other editors sit at ~27.3 and ~3.4. Moonwalker+DirectInv just lands on those values. PnP and Pix2Pix-Zero are numerically identical to DirectInv without edit (MasaCtrl is slightly worse), so DirectInv may override the adapter.
  • Selection on the test set. Table 2 configurations were picked from sweeps (Table 13) on PIE-Bench with no held-out split. The cycle loss barely matters(Tables 2, 13), so Sec. 3.2 is not justified.
Questions for authors
  1. Please explain the P2P DirectInv control in Table 5 (PSNR 19.65 vs ~27.3 for other editors) and why it differs from the PnP Inversion paper’s reported DirectInv+P2P numbers. Are the Table 2 P2P gains robust to a corrected baseline?

  2. Why is Moonwalker+DirectInv numerically indistinguishable from DirectInv on reconstruction for PnP and Pix2Pix-Zero (and nearly for MasaCtrl)? Does the adapter’s trajectory have any effect once DirectInv’s correction is applied?

  3. Were the configurations in Table 2 selected on PIE-Bench itself? Please report results with configurations fixed in advance or chosen on a disjoint split, with confidence intervals or paired tests.

  4. What is the reference value of the correlation metric for true Gaussian noise? Given standalone Moonwalker does not improve preservation for most editors (Table 2), what evidence supports that reduced correlation causes better editing?

  5. Evaluate real-image inversion against nudging, ReNoise and EDICT/BELM, and give the training cost per adapter.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.