- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents AMOR, a memory‑guided omnidirectional RGB‑D diffusion framework for static scenes. Taking an initial RGB panorama, its camera pose, and a specified target camera pose as inputs, the model progressively generates panoramic color images and radial depth for novel views, and fuses credible outputs into an explicitly maintained 3D scene memory.
The method consists of three core modules: ① A separate single‑channel depth prediction branch to avoid depth numerical distortion introduced by RGB‑VAE encoding. ② Surface‑guided memory readout, which performs ray‑intersection rendering on existing 3D surfels to generate conditioning panoramas. ③ Conservative memory update, which conducts scale alignment and geometric validation, filters invalid predictions, and mitigates geometric drift.
Experiments are conducted on the Holo360D, Replica, and TartanGround datasets, with comparisons against Pano2Room, HY‑World 2.0, Matrix‑3D, and PanoWorld. AMOR achieves better LPIPS and FID, as well as competitive PSNR and SSIM, and obtains promising results on the TartanGround depth task. Ablation studies verify the effectiveness of surface‑guided readout and conservative memory update. A fixed‑prediction replay experiment demonstrates the usefulness of the memory‑fusion strategy.
Critical strengths
- It builds a complete generation‑fusion closed‑loop. Instead of decoupling novel‑view synthesis and 3D memory update, it establishes a full pipeline of “memory readout ‑ RGB‑D generation ‑ validated memory update”. It specifically addresses the problem of error accumulation under continuous scene generation, and bears practical significance for the task of incremental roaming over static scenes.
- The component design is targeted and supported by reasonably‑designed controlled ablation evidence. The separate single‑channel depth prediction branch avoids depth distortion caused by the VAE. Ablations show that surface‑guided readout outperforms naive point projection. In the fixed‑prediction replay experiment, diffusion generation outputs are fully fixed; only the memory‑fusion policy is modified, yet map precision is improved. This proves that performance gains originate from the memory strategy itself.
Critical weaknesses
Experimental validation is insufficient. Omissions exist across multiple experimental sections, which may fail to fully support the paper’s claims.
- The authors note that baselines have inconsistent output resolutions and comparisons are performed after resizing all outputs to the same dimension, which may introduce unfairness. It would be valuable to conduct validation experiments to quantify the losses brought by resizing, for instance by resizing the authors’ own model outputs to different resolutions.
- Anti‑drift performance via trajectory revisit is validated on only one single scene. Additional quantitative or qualitative results on more scenes are desirable.
- The paper mentions that conservative update improves precision at the cost of reduced completeness. Though this precision‑completeness trade‑off is acknowledged, there lacks analysis quantifying the magnitude of this drop and whether such degradation is acceptable.
- Ablation studies should be supplemented with sensitivity‑analysis experiments for key hyper‑parameters.
Questions for authors
- Memory initialization relies on initial depth outputs from MoGe‑2 followed by a scale‑alignment procedure. Please clarify whether this scale‑alignment step depends on ground‑truth (GT) depth from the dataset, or whether GT depth is entirely unnecessary for real‑world deployment. Also please confirm whether all compared baselines have access to equivalent scale‑related inputs. This implementation detail affects real‑world applicability of the method as well as the fairness among baseline comparisons.
- The paper acknowledges inconsistent native output resolutions among baselines; comparisons are made after resizing outputs to identical resolution, which may cause unfairness. Please perform validation experiments to quantify the impact introduced by resizing, for example by resizing outputs of your own model to different resolutions, or adopt alternative approaches to verify whether such unfairness severely affects experimental outcomes and whether reported performance rankings still hold. If such experiments cannot be completed, please explicitly state this evaluation limitation in the main text.
- Anti‑drift validation via trajectory revisit is performed only on the Hospital scene. Meanwhile, ablations for core‑module effectiveness are conducted on a small subset of merely 9 scenes. Are there revisit results from more scenes or longer trajectories to examine whether errors keep accumulating under successive updates? If additional revisit results are unavailable, please explicitly state the validated scope of this anti‑drift effect in the manuscript to help readers understand the conditions under which this conclusion applies.
- As shown in Sec.4.4 of the paper, conservative update improves precision while degrading completeness. The paper acknowledges this precision‑completeness trade‑off, yet experimental analysis quantifying the magnitude of completeness degradation and whether this degradation is acceptable is missing. Please add such analysis.
- Please supplement sensitivity‑analysis ablation experiments for key hyper‑parameters.
- In the depth estimation evaluation under RGB-only observation (Figure 7b), AMOR achieves better RMSE, while DepthMaster obtains a lower AbsRel. Analysis for this observation should be supplemented.