- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces Endpoint-Calibrated Pullback Interpolation, a plug-in correction for existing diffusion interpolation paths. It constructs position-dependent references from the log-frequency-band energies of reconstructed endpoints and treats responses below these references as one-sided spectral deficits. Gradients are propagated through the remaining denoising process and decoder to adjust intermediate noisy latents. Updates are constrained by route-normal motion, latent-norm preservation, and local score compatibility, aiming to improve intermediate-frame detail while retaining the underlying interpolation framework.
Critical strengths
1.A valuable shift in problem formulation. Rather than evaluating interpolation exclusively through latent-space geometry, the method identifies deficiencies in rendered images and asks the generative process to implement their correction.
2.A restrained objective design. Reconstructed endpoints calibrate the target to the capabilities of the inversion–generation system. The one-sided objective is also less restrictive than forcing every frequency response to match a prescribed reference exactly.
3.Careful investigation of alternative explanations. RGB sharpening, fitted filtering, decoder-only refinement, objective variants, and score-constraint controls provide substantially stronger evidence than aggregate quality comparisons alone. The paper also transparently reports transition-quality trade-offs.
Critical weaknesses
1.The necessity of endpoint-specific calibration is not established. Reference-value interventions show that the controller responds to its target, but not that the current endpoint pair provides a better target than a fixed enhancement request, population-level reference, or shuffled endpoint reference.
2.Spectral deficits need not indicate missing semantically appropriate detail. Global frequency energies do not specify where texture belongs or whether an edge is structurally correct. Deformation, occlusion, and category transitions may legitimately reduce particular frequency components.
3.Sharper frames do not necessarily imply more natural interpolation. No-reference quality metrics and FID against endpoint sets do not directly measure identity transition or sequence naturalness. Table 6 also shows costs in transition statistics. Complete-sequence perceptual evaluation would help resolve this distinction. (pp. 7, 14)
4.The necessity of full continuation backpropagation remains insufficiently isolated. Each pair and route requires 1,800 denoising-step vector–Jacobian products, without reported runtime or peak memory. The decoder-only control is useful, but differs in optimization and constraints, while its mean TOPIQ score is not clearly distinguishable from ECPI’s. (p. 13; Table 13)
Questions for authors
1.How important is the current pair’s endpoint reference? Could fixed, population-level, and shuffled-endpoint references be compared under a matched latent-displacement budget?
2.How does ECPI behave when reduced intermediate-frequency energy is appropriate because of deformation or occlusion? Have you identified corrections that introduce inappropriate texture or alter structure?
3.Is there perceptual evidence evaluating complete sequences and separately assessing local detail, structural plausibility, and identity continuity, particularly given the transition-statistic trade-offs?
4.Which differences between ECPI and decoder-only refinement account for the observed advantages? Please also report correction runtime and peak memory, and clarify whether a simpler or shorter-backpropagation alternative achieves comparable results.
2 comments
Q3. Perceptual evidence across interpolation paths and temporal consistency
We provide complete 17-frame sequences for the 16 selected manuscript examples, with synchronized baseline/ECPI playback and matched FreeMorph outputs for Morph4Data.
The printed figures guide inspection of the three dimensions raised by the reviewer: aligned crops and ear contours show local detail (Figs. 15–19 and 22), vehicles show structural evolution (Figs. 23–25), and portraits show identity-related changes (Figs. 17 and 27). Appendices G and H display both endpoints and five consecutive intermediate positions; the viewer extends each example to its complete path. Appendix F also retains the two printed 17-frame sequences (Figs. 11–12).
For single-frame perceptual quality, ECPI improves TOPIQ, MUSIQ, and MANIQA across all three routes on Morph4Data without retuning, with all nine paired 95% confidence intervals above zero (Appendix I.2, Tables 16–17). These established no-reference predictors have been validated against human subjective image-quality ratings, providing perceptual-quality evidence beyond the optimized spectral objective.
To address the reported increase in unsmoothed PPL and PDV, we evaluated a predefined temporal-smoothing grid on all 90 MorphBench pairs. For each route, we used the unsmoothed baseline's mean PDV as a common ceiling and selected the smallest feasible smoothing strength independently for each arm, without optimizing for IQA. At these operating points, ECPI retains positive improvements:
| Route | ΔTOPIQ | ΔMUSIQ | Baseline alpha | ECPI alpha |
|---|---|---|---|---|
| GeoDiff | +0.01031 | +1.0468 | 0 | 0.10 |
| Tangent | +0.01399 | +1.9542 | 0 | 0.25 |
All four paired 95% confidence intervals are above zero. Selection is repeated within each pair bootstrap. Appendix I.4 and Table 19 report the protocol, gains, and achieved PDV values.
Thus, ECPI's measured frame-quality improvements persist at operating points satisfying a common mean-PDV ceiling on this cohort. The unsmoothed examples in Appendices F–H complement these measurements with direct visual inspection of detail and transition behavior.
Q4. Full continuation and computational cost
To address the confounds in the earlier decoder-only control, we added a same-state, same-controller comparison with a one-step predicted-clean-plus-decoder Jacobian approximation. Both arms update the same noisy latent, use the same actual baseline deficits, endpoint references, route-normal/score constraints, and controller rules, and are evaluated through the complete continuation. The substituted component is the readout Jacobian; weights and doses are recomputed under the same rules using each Jacobian.
On the frozen 12-pair mechanism cohort across three routes, full pullback achieves 1.74–2.11 times the mean spectral-deficit recovery at equal midpoint displacement. On complete, perceptually resampled 17-frame paths, the gains over the approximation are (Appendix I.3, Table 18):
| Route | TOPIQ gain | MUSIQ gain |
|---|---|---|
| GeoDiff | +0.00299 | +0.281 |
| Tangent | +0.01044 | +0.936 |
| SLERP | +0.01024 | +0.747 |
All six paired 95% confidence intervals are above zero. The full-path comparison uses the same controller and movement cap, with realized displacement determined separately in each arm. Full pullback also yields higher spectral excess and higher PDV on SLERP.
On an RTX 4090, in FP32 at 512 x 512, correction takes approximately 143 s versus 31 s, and existing-route-to-final-output time is 173 s versus 60 s, for full versus approximate gradients. Peak allocated memory, including resident models, is 11.23 versus 12.59 GiB for these implementations. Both timings include the actual-baseline forward computation; inversion, text-condition optimization, route construction, model loading, I/O, and IQA scoring are excluded.
The controlled comparison demonstrates stronger target recovery and measured perceptual-quality gains from full-continuation gradients relative to the tested approximation, with an explicit runtime cost. Appendix I.5, Tables 20–21, provides the measured-stage costs and separately labeled pipeline estimates.
We thank the reviewer for the constructive questions. We added reference-source interventions, a targeted visual audit, selected multi-frame comparisons in Appendices G and H, and a controlled simpler-gradient comparison to examine endpoint calibration, perceptual quality, temporal consistency, and computational cost.
Q1. Importance of the current pair's endpoint reference
We compared current-pair, population-fixed, and three preselected shuffled endpoint references on GeoDiff MorphBench pairs. The population-fixed reference uses mean reconstructed endpoint log-band energies from a separate 12-pair calibration cohort, excluded from the primary recipient analysis.
For active corrections, we use the same actual spherical displacement (0.0025 times the endpoint chord). Shuffled-reference results are averaged within each recipient before pair-level inference.
The updated reference-source comparison shows consistent advantages for the current-pair reference (Appendix I.1, Table 15):
All four confidence intervals are above zero. These results provide direct evidence that using the current pair's reconstructed endpoints as the spectral calibration reference improves perceptual-quality scores relative to both population-level and unrelated endpoint references.
The displacement is matched per active correction; reference-dependent activation rates may differ, so aggregate correction dose is not necessarily identical. The population-fixed reference is a constant log-energy target, distinct from a fixed enhancement amount relative to each intermediate image.
Q2. Semantically appropriate spectral reductions and structural changes
In 12 high-transition cases selected solely from baseline adjacent-frame variation, the median ECPI RGB change was 10.3% of the local baseline transition, and most showed no conspicuous isolated texture artifacts or contour breakage. These cases were selected independently of ECPI outputs and IQA scores following quantitative screening of 4,050 matched-position comparisons (Appendix I.6). Thus, substantial baseline transitions can coexist with modest ECPI interventions in the inspected cases.
The image comparisons show what the corrections change: facade and vehicle details become more distinct, while some corrections also adjust object contours (Appendix G, Figs. 15–19). The separate ten-case high-change audit flagged two cases for possible over-restoration. We also show two selected unfavorable cases in Figs. 20–21, with mixed brow/eye strokes and displaced facial features under correction. Appendix I.6 distinguishes these visual observations from the spectral flags.
This behavior is consistent with ECPI's calibrated, bounded intervention: reduced spectral energy during deformation or occlusion need not indicate missing detail, and the one-sided objective does not require exact matching to the endpoint reference. The score and latent-movement constraints restrict the correction, while the images reveal its actual perceptual effect.