- Rating
- 3
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper introduces DNPMorph, a novel approach for image morphing using pretrained latent diffusion models. Unlike existing methods that indirectly improve morphing smoothness by interpolating model parameters, modifying attention mechanisms, or constructing complex geometric priors, DNPMorph formulates the task as the direct optimization of the noise-path latents to minimize a sequence-level smoothness objective evaluated on the decoded frames. To make this path-level optimization computationally feasible, the authors employ a zero-order surrogate update that treats the diffusion denoising chain as a black box (omitting backpropagation through it) while still propagating gradients through the decoder and the image-space smoothness loss. The path is initialized via Spherical Linear Interpolation (SLERP) in the native Gaussian noise space and refined using a two-stage coarse-to-fine schedule. The paper provides a theoretical conditional descent analysis to justify the surrogate update and conducts extensive experiments across four benchmarks. DNPMorph achieves state-of-the-art performance in path-smoothness metrics and improves FID on three out of four datasets, all while preserving the native Gaussian geometry of the diffusion prior.
Critical strengths
-
Elegant and Direct Problem Formulation: The core idea of directly optimizing the noise latents to minimize decoded-image smoothness is highly intuitive. By keeping the diffusion model fixed and treating morphing as a trajectory optimization problem in the native noise space, the method avoids the pitfalls of adversarial exploitation seen in direct pixel-space optimization (as shown in Sec 5.3), while ensuring that all intermediate frames remain strictly on the diffusion model's data manifold.
-
Comprehensive and Rigorous Validation: The paper provides an exceptionally thorough evaluation. Beyond standard quantitative metrics and user studies, the authors conduct insightful analyses to prove that the optimized latents do not arbitrarily drift but actually preserve the Gaussianity and the geometric arc of the initial SLERP path (Section 5.4). Furthermore, the inclusion of a theoretical conditional descent analysis (Theorem 1) for the zero-order surrogate update adds significant mathematical rigor to the proposed optimization strategy.
Critical weaknesses
-
High Test-Time Computational Cost: As acknowledged in the limitations and runtime analysis, DNPMorph requires iterative test-time optimization (invoking the full diffusion-process map multiple times per sequence), making it significantly slower than methods that rely on single-pass sampling dynamics or parameter interpolation (e.g., FreeMorph or GeoMorph). While the authors reasonably frame this as a "quality-compute tradeoff," this latency restricts its applicability in interactive, real-time, or large-scale batch-processing scenarios.
-
Lack of Explicit Structural/Semantic Correspondence: The method relies entirely on local, adjacent-frame smoothness objectives (like LPIPS and DINO distances) without explicit long-range correspondence, optical flow, or 3D-aware motion modeling. Consequently, when morphing between endpoint images with drastic structural, viewpoint, or semantic changes, the optimization might struggle to produce physically plausible intermediate motions, potentially leading to "melting" artifacts or semantic confusion (as noted in the limitations).
Questions for authors
-
Could the authors discuss potential avenues for accelerating this test-time optimization in future work or the rebuttal? For instance, would leveraging consistency distillation, few-step flow models, or initializing the path with a faster, lightweight proxy model reduce the required number of optimization iterations without sacrificing the final smoothness?
-
While this is a common limitation in diffusion morphing, how sensitive is the method to extreme viewpoint changes compared to structural baselines? Incorporating a lightweight structural prior (e.g., depth-aware warping or optical flow guidance) as an additional regularization term in the sequence-level objective could further enhance the method's robustness. Have the authors considered or experimented with any structural regularizers?
3 comments
Thanks for the rebuttal. I will maintain my 3: Weak acceptance for the paper.
We thank the reviewer for their constructive and positive feedback. Their review helped us improve the manuscript.
We thank the reviewer for the constructive review.
Test-time cost (W1): The revised Table 1 reports runtime and peak memory for all methods. The peak memory of DNPMorph (about 36 GB) is similar to that of IMPUS and higher than that of the single-shot baselines (5-9 GB). Two observations put this cost in context. First, at a comparable budget DNPMorph is clearly better: DNP-50 and IMPUS require similar time (20 vs. 17 minutes per pair) and similar memory, and DNP-50 outperforms IMPUS on every metric and every dataset. Second, the faster baselines are single-shot methods and cannot exchange additional compute for smoother morphs. DNPMorph, in contrast, improves gradually along the optimization, so the user can choose an operating point. We position DNPMorph for offline, high-quality morph generation and state this explicitly in the main text.
Accelerating the optimization (Q1): We thank the reviewer for these suggestions, which fit naturally into our framework. Each iteration requires only forward passes through the T-step sampler and no backward pass through the denoiser.
Consistency models [1], latent consistency models [2] and adversarially distilled models [3] reduce the number of steps from 50 to just a few. Since our method only requires a deterministic mapping from noise to clean latents, they can be used directly. Exact verification of the alignment condition also becomes cheaper in this regime.
A second avenue is initialization. Since DNPMorph locally refines its initial path (Sec. 5.4), starting from a path that is already smooth, e.g., one produced by a faster morphing method, should reduce the required number of iterations. Finally, most of the improvement occurs early in the dense stage (Fig. S14), so an early-stopping rule offers a simple further saving. We will add this discussion to the paper and make an effort to include a few-step variant in the final version.
Structural correspondence and extreme viewpoint changes (W2/Q2): We agree that explicit structural correspondence could be particularly valuable for handling large viewpoint and pose changes. Our current LPIPS and DINO objectives encourage perceptual and semantic smoothness between adjacent frames, but do not explicitly enforce physically plausible motion. Consequently, challenging transitions can still exhibit abrupt pose changes or cross-fade-like effects, as illustrated by the failure cases in App. G. Similar artifacts also appear in competing methods (Figs. 5 and S10-S12), although we have not conducted a separate quantitative comparison specifically for extreme viewpoint changes.
We have considered incorporating structural guidance into the objective, as the reviewer suggests, and view this as a particularly promising direction for future work. A key advantage of our direct optimization formulation is that the sequence-level objective S can naturally accommodate additional terms, such as optical-flow-based warp consistency [4] or depth-aware correspondence, without modifying the optimization framework. Such terms could encourage more coherent motion and structural consistency, complementing the current perceptual smoothness objectives. We have added this discussion to Sec. 5.5 and App. G of the revised manuscript.
[1] Y. Song, P. Dhariwal, M. Chen, and I. Sutskever. Consistency models. ICML, 2023.
[2] S. Luo, Y. Tan, L. Huang, J. Li, and H. Zhao. Latent consistency models: Synthesizing high-resolution images with few-step inference. arXiv, 2023.
[3] A. Sauer, D. Lorenz, A. Blattmann, and R. Rombach. Adversarial diffusion distillation. ECCV, 2024.
[4] Z. Teed and J. Deng. RAFT: Recurrent all-pairs field transforms for optical flow. ECCV, 2020.