- Rating
- 3
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper presents a strong and carefully engineered framework for panoramic generation and editing. The combination of geometry-aware pretraining, large-scale task-specific data construction, and unified RGB-only adaptation is interesting and practically valuable. The experimental results are convincing at the system level, and I believe the work would be useful to the community.
My main reservation is that the paper does not fully isolate the contribution of geometry-aware pretraining itself. The current evidence shows that the complete Canvas360 pipeline is effective, but additional controlled experiments would be needed to more clearly attribute the improvements to the proposed geometric prior learning.
Overall, I lean toward acceptance because the empirical gains, dataset contribution, and practical relevance are substantial, while the remaining concerns appear addressable through further analysis rather than fundamental flaws.
Critical strengths
The paper addresses an important challenge in panoramic generation: preserving global scene structure and boundary consistency under equirectangular projection. The proposed solution is well motivated, combining RGB-depth pretraining, modality separation, and boundary-aware modeling into a unified framework. The method achieves consistent improvements on panorama-specific metrics and several downstream tasks, showing that the proposed training strategy is practically effective.
The construction of Canvas360Dataset is a substantial contribution. The dataset provides large-scale panorama context-target pairs for multiple in-context tasks, including style transfer, inpainting, outpainting, and editing. The evaluation is also comprehensive, covering generation quality, boundary consistency, reconstruction accuracy, user preference, and ablation studies. The authors make a serious effort to analyze different components rather than only reporting final benchmark numbers.
Critical weaknesses
- The main claim of the paper is that geometry-aware pretraining learns a transferable panoramic prior. However, the current experiments mostly compare the complete Canvas360 pipeline against existing methods. A more controlled comparison would be important, for example:RGB-only panoramic pretraining + the same downstream fine-tuning;RGB-depth pretraining without specific geometric components;no panoramic pretraining with the same downstream data.
Such experiments would clarify whether the observed improvements are mainly due to depth-based geometric supervision, panoramic pretraining itself, or the large amount of task-specific fine-tuning data.The current ablation studies verify that individual components improve the full model, but they do not completely establish the causal role of geometry-aware pretraining in downstream transfer.
-
The paper shows improved panorama fidelity and boundary consistency, but the evidence for learning a transferable geometric representation is still somewhat indirect. For example, the depth visualization and qualitative comparisons are helpful, but more explicit geometric evaluations (e.g., consistency of reconstructed 3D structure, viewpoint consistency, or cross-view evaluation) would strengthen the argument that the model learns a reusable geometric prior rather than only improving image statistics.
-
RGB-depth modeling and geometric supervision have been explored in previous panoramic generation works. The strongest novelty of this paper appears to be the combination of geometry-aware pretraining with a unified RGB-only in-context editing framework and large-scale task adaptation. The paper would benefit from a clearer discussion separating what is new in the proposed framework from existing RGB-depth panoramic generation approaches.
Questions for authors
A key claim is that geometry-aware RGB-depth pretraining provides a transferable prior for downstream RGB-only tasks. Have the authors considered a matched RGB-only panoramic pretraining baseline using the same data scale and training budget? This would help isolate the contribution of depth supervision.
How much of the downstream improvement comes from Stage 1 pretraining versus the 900K context-target pairs used in Stage 2? Could the authors provide learning curves or comparisons with reduced downstream training data?
For Velocity Circular Padding, the paper reports improvements in boundary consistency for the full model. Could the authors provide direct boundary metrics (e.g., LRCE-RGB) for the VCP ablation itself to quantify its isolated contribution?
Does the learned prior generalize beyond the four evaluated tasks? For example, would the pretrained model benefit other panoramic tasks such as viewpoint-conditioned generation or 3D-aware editing?
9 comments
-
What does removing depth supervision reveal, and does Stage 1 improve downstream learning efficiency? (W1, Q2)
The B versus B+D comparison in item 1 removes auxiliary depth generation and supervision while retaining RGB-only panoramic pretraining. It isolates the contribution of depth under matched pretraining and downstream conditions.
Following the reviewer's helpful suggestion, we also provide inpainting learning curves in Appendix L, Fig. 18. Orange denotes initialization with Stage 1, and blue denotes initialization without Stage 1. After training begins, the Stage-1-initialized model achieves higher PSNR and lower LPIPS at the recorded checkpoints under the same downstream training budget. The PSNR advantage is already visible early in fine-tuning and persists through 1,000 steps, supporting better adaptation within the same downstream optimization budget.
The component ablation isolates depth supervision, while the learning curves evaluate the benefit of Stage 1 as a whole. The 900K pairs describe the constructed downstream corpus. Our additional study uses the fixed 20K subset to test whether pretraining remains useful with limited downstream supervision. We provide the learning-curve analysis requested by the reviewer, with conclusions restricted to this small-scale setting rather than extrapolated to a full-corpus data-scaling study.
-
Does VCP itself improve horizontal boundary consistency? (Q3)
Thank you for suggesting a metric directly aligned with VCP's purpose. Following the Table 7 ablation setting at 512 × 1024 resolution, we evaluated LRCE-RGB for the baseline, standard circular padding, VCP, and the final combination, as also reported in our response to Reviewer PbWc.
Method LRCE-RGB ↓ (×10⁻²) B 1.60 B + standard circular padding 1.15 B + VCP 0.74 B + D + PO + + VCP 0.79 VCP reduced LRCE-RGB by 53.75% relative to B and by 35.65% relative to standard circular padding. The final combination also maintained low boundary error. These results directly quantify VCP's intended effect on seam continuity, while the distributional metrics in the paper document its trade-offs on other properties.
-
Can the geometric evidence go beyond qualitative RGB and depth visualizations? (W2)
We appreciate the reviewer's recognition of the qualitative evidence and the constructive suggestion to make the geometric evaluation more explicit. We have added a quantitative assessment of depth relationships across completion boundaries, using the same protocol reported in our response to Reviewer PbWc. For each original and completed panorama, we estimate depth using the same estimator and measure the change in the mean log-depth difference between 5-pixel bands inside and outside the completion mask. Comparing with the original panorama accounts for genuine scene discontinuities, and the log-depth difference is invariant to global multiplicative depth scale. The metric and results are included in Appendix M.
Task Method Depth boundary error ↓ (×10⁻²) Inpainting FLUX.1-dev LoRA 2.83 B 2.63 B+D 2.43 Outpainting FLUX.1-dev LoRA 9.18 B 7.64 B+D 7.16 Adding depth supervision to B reduced this error by 7.60% for inpainting and 6.28% for outpainting, supporting better preservation of depth relationships across completion boundaries.
We also include point-cloud reconstructions in Appendix M, Fig. 19, and the supplementary material. Our jointly generated depth maps use a normalized image representation through the RGB VAE and are not calibrated for precise metric reconstruction. We therefore estimate depth from the generated RGB panoramas using MoGe-2 and back-project the RGB pixels into 3D. The recovered walls, floor, and furniture provide qualitative support for coherent scene geometry in the generated RGB images, complementing the quantitative boundary evaluation. White gaps reflect missing surface samples due to occlusion and depth filtering, including the removal of points near depth discontinuities and at extreme distances.
-
What distinguishes Canvas360 from existing RGB–depth panoramic generation methods? (W3)
We appreciate the suggestion to clarify the methodological distinction. Canvas360's central contribution is to learn geometry within a shared panoramic generator and transfer that representation to a unified RGB-only manipulation model. The design connects depth-supervised representation learning, explicit ERP periodicity, and a common conditioning interface for four downstream tasks.
Specifically, Canvas360 uses depth as an auxiliary generative target during pretraining. RGB and depth share the pretrained VAE and are concatenated along the token-sequence dimension within one transformer, allowing cross-modal interaction in the backbone's existing latent space without a separate depth encoder or expanded channel dimension. Modality-specific positional offsets and similarity regularization support this joint modeling. The shared weights then initialize one model for four RGB-only tasks, with neither depth input nor depth generation required at inference. Thus, the role of depth is to shape a reusable generative prior that remains useful when the depth stream is removed. The matched B versus B+D experiment supports this transfer mechanism, reducing outpainting FAED by 31.70% and increasing completion-region PSNR by 1.00 dB.
The distinction from DreamCube is in the representation and adaptation strategy. DreamCube also reuses a pretrained VAE, but concatenates RGB and depth latents along the channel axis in a synchronized cubemap U-Net. Canvas360 instead uses sequence-level joint modeling in ERP and transfers the learned prior to a shared RGB-only context-and-prompt interface. In the matched outpainting comparison reported in our response to Reviewer PbWc, Canvas360 obtains LPIPS/FAED/PSNR of 0.42/1.79/17.16, compared with DreamCube's 0.64/3.13/11.43. This supports the effectiveness of the complete framework in that setting.
VCP additionally couples consecutive longitude indices with soft boundary-feature synchronization before every transformer block and direct supervision of the circularly padded velocity field. It therefore incorporates horizontal periodicity into both the internal computation and the training objective. Its 35.65% LRCE-RGB reduction over standard circular padding provides direct evidence for this component's contribution.
-
Does the pretrained prior support tasks beyond the four evaluated in the main paper? (Q4)
We thank the reviewer for this valuable suggestion, which closely aligns with Reviewer oNzz's interest in layout-constrained composition. We have added a layout-controlled generation experiment to Appendix K, Fig. 17. Following our editing-data construction procedure, we annotated 30K examples with binary object-layout masks. A value of 1 marks object regions to retain, and 0 marks the remaining regions. These masks construct spatial reference conditions through the existing image-conditioning interface. We initialized from the Stage 1 model and fine-tuned on these examples for 20 epochs.
These examples show that the framework can be adapted to layout-conditioned generation through task-specific fine-tuning. Viewpoint-conditioned generation and 3D-aware editing are valuable further directions. The present experiment evaluates layout-conditioned adaptation, while camera-pose-conditioned generation across translated viewpoints remains outside the current evaluation.
We thank the reviewer for recognizing Canvas360's well-motivated design for preserving global scene structure and boundary consistency, its consistent gains on panorama-specific metrics and downstream tasks, and its practical value. We also appreciate the reviewer's recognition of Canvas360Dataset as a substantial contribution and of our comprehensive evaluation and component analyses.
We particularly appreciate the suggestion to separate the effects of panoramic pretraining, depth supervision, and downstream adaptation. This closely aligns with Reviewer PbWc's recommendation and helps us strengthen the evidence for transfer. Within the rebuttal time window, we conducted a focused, small-scale controlled study using the same setting reported in our response to Reviewer PbWc. The comparisons below use 512 × 1024 panoramas, the same 20,000 downstream context–target pairs, 1,000 fine-tuning steps, and the same 500 held-out validation panoramas. B and B+D use the same pretraining panoramas and training schedules, with auxiliary depth supervision added in B+D. All other settings follow the main paper. This reduced-data study directly tests the effect of initialization under matched downstream conditions, while the learning curves provide complementary evidence about learning efficiency.
Does depth supervision improve downstream transfer over matched RGB-only panoramic pretraining? (Q1, W1)
We compared the RGB-only panoramic pretraining baseline B with its depth-supervised counterpart B+D from Table 7. Both models were adapted to the downstream tasks with RGB-only inputs and outputs under the matched setting above. We report PSNR over both the full image and the completion region, defined as the masked hole for inpainting and the region outside the input field of view for outpainting.
Adding depth supervision reduced FAED by 17.02% for inpainting and 31.70% for outpainting, and improved both PSNR measures on both tasks. The gain was larger for outpainting, where completion-region PSNR increased by 1.00 dB. Inpainting LPIPS remained essentially unchanged, while outpainting LPIPS improved. These matched comparisons support an additional benefit from depth-supervised pretraining for RGB-only completion beyond RGB-only panoramic pretraining.
How much does panoramic pretraining improve completion under identical downstream fine-tuning? (W1, Q2)
We compared FLUX.1-dev LoRA, fine-tuned directly on the downstream pairs without Stage 1, with B under the same downstream data and optimization budget.
Panoramic pretraining improved all four metrics on both tasks. Completion-region PSNR increased by 0.50 dB for inpainting and 1.64 dB for outpainting. Because downstream data and training schedules were identical, the gains support the contribution of panoramic pretraining under a fixed adaptation budget. Together with B versus B+D, this separates the benefit of RGB-only panoramic pretraining from the additional benefit of depth supervision.