- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Demiurge rerenders a monocular video along multiple camera trajectories while maintaining a shared, time-indexed point-cloud memory. Each iteration generates an RGB video, recovers its depth, and incorporates the resulting observations into memory. The system combines memory-conditioned appearance generation, cache-consistent depth completion, dual-stream warping for appearance preservation and occlusion reasoning, and optional geometry-aware refinement. Evaluation covers 128 videos with 14 target trajectories each, supplemented by large-parallax examples and per-timestamp 3D Gaussian reconstructions.
Critical strengths
1)The dual-stream design addresses a clear trade-off between preserving appearance and maintaining sufficient geometry for occlusion handling. Its contribution is supported by ablations: removing it reduces View Sync from 64.893 to 51.498 and increases P-MSE from 0.0010 to 0.0049.
2)The method shows consistent improvements across perceptual quality and reconstruction compatibility on 128 videos with 14 trajectories each. Together with the component ablations, these results support its practical value for cross-trajectory video rerendering.
Critical weaknesses
1)The consistency metrics provide useful but indirect evidence. P-MSE covers only input-visible regions, View Sync evaluates only the final frame without geometric verification, and 3DGS reconstruction is evaluated on its input views. A held-out or revisit test would better support consistency in newly generated regions.
2)The depth-conversion description needs clarification. The model predicts normalized disparity, but Eqs. (55)–(58) describe affine fits to depth without an explicit reciprocal conversion. This may be a notation omission, but the exact procedure is important for reproducibility.
Questions for authors
-
Could you clarify how normalized disparity is converted to depth in Eqs. (55)–(58)? Is a reciprocal operation omitted from the description, or does the implementation directly fit disparity to depth?
-
Do you have held-out or revisit results that assess consistency in newly generated regions, particularly beyond the final frame? Such evidence would help establish how well the reported gains extend to the central cross-trajectory consistency claim.
3 comments
Thank you for the detailed rebuttal and for updating the manuscript.
The clarification of the disparity-to-depth procedure in Eqs. (55)–(58) addresses my reproducibility concern. Performing the affine alignment in disparity space and then applying the reciprocal conversion is now explicit, and the revised equations should make the depth recovery procedure easier to follow.
The new held-out 3DGS evaluation is also a useful addition. Reconstructing from seven generated views and comparing renderings at the remaining seven viewpoints is a more direct test of cross-view consistency than the original input-view protocol, and Demiurge is the strongest method under this split. The qualitative comparison of newly revealed content at an intermediate timestamp further helps support consistency beyond the final frame.
Taken together, these responses address the two issues raised in my review. I will keep my original rating of 3 (Weak acceptance).
Dear Reviewer UL3a,
We sincerely thank you for your reply and for carefully considering our responses and revised manuscript! We are pleased that our clarifications and additional results have addressed your concerns. We greatly appreciate your constructive feedback and continued positive assessment of our work.
Best,
Authors
Dear Reviewer UL3a, we are grateful for your careful review of our manuscript and for the constructive feedback you provided. We appreciate your recognition of how our dual-stream design balances appearance preservation and occlusion handling, as well as the consistent improvements in perceptual quality and reconstruction compatibility. We hope our responses below address the points you raised.
Q1. Disparity-to-depth conversion in Eqs. (55)–(58)
We sincerely thank you for pointing out this ambiguity. In fact, we perform global affine alignment in disparity space and apply the reciprocal conversion afterward. The alignment calibrates the normalized disparity prediction to the scale of the cached disparity ξi=1/(Di+1), where Di denotes the partial cached geometric depth. Subsequently, we convert the aligned disparity ξi to geometric depth using Gi=1/(ξi+ϵinv)−1. Here, ϵinv is a small constant (e.g., 1e-5) added to the denominator for numerical stability.
We have updated Eqs. (55)–(58) in the revised manuscript (Line 1034 ~ Line 1068). We sincerely thank you again for highlighting this point, which has helped us make the depth recovery procedure clearer and easier to reproduce.
Q2. Held-out reconstruction and consistency in newly revealed regions.
We sincerely thank you for raising this point. Following your suggestion, we extend our 3DGS evaluation to held-out views. We uniformly sample seven of the fourteen generated views to reconstruct the Gaussians. We render the reconstruction at the remaining seven viewpoints and compare the renderings with the corresponding generated frames. We apply the same view split to all methods.
As demonstrated above, Demiurge achieves the best results across all three metrics. Compared with the strongest baseline for each metric, it improves PSNR by 2.897 dB and SSIM by 0.034, while reducing LPIPS by 0.052. These results provide further evidence that our generated videos are more consistent across different viewpoints. We have included the held-out-view evaluation in the revised manuscript (Appendix M, Table 5, Lines 1662–1688).
To complement these quantitative results, we have added a qualitative comparison of newly revealed content across two trajectories in the revised manuscript(Appendix G, Line 1515 ~ Line 1548, Figure 7). In particular, at the intermediate timestamp T3, both trajectories show the same region R, which is absent from the corresponding input frame. The doorway and nearby artwork in this region have a consistent appearance and layout across the two generated views.
We sincerely thank you again for this thoughtful suggestion, which has helped us examine the consistency of newly revealed content before the final frame.