acceptodds
Under review as a conference paper at ICLR 2027

Enhancing In-context Panoramic Generation via Geometric-aware Pretraining

Abstract

Panoramic image generation must preserve scene structure across a full spherical field of view, but horizontal wrap-around and latitude-dependent distortion make scene completion and editing difficult. We present Canvas360, a two-stage framework that learns a geometry-aware panoramic prior with parallel RGB–depth pretraining and transfers it to a unified RGB-only generator for four in-context tasks, namely style transfer, inpainting, outpainting, and editing. The pretraining stage combines pseudo-depth supervision, modality-specific representations, and panorama-aware boundary handling to capture scene structure while respecting equirectangular adjacency. To support this two-stage design, we construct Canvas360Dataset with annotated RGB–depth panoramas and task-specific context–target pairs for prior learning and downstream adaptation. On text-to-panorama generation, Canvas360 reduces panorama-specific distributional distance by 49.4% relative to the strongest baseline and outperforms the evaluated baselines in semantic alignment, perceptual quality, and boundary consistency. Completion and editing evaluations further show improvements in perceptual similarity, panoramic fidelity, and reconstruction accuracy. These results indicate that geometry-aware pretraining provides a transferable prior for panoramic generation.

Primary area:
generative models
Keywords:
Panoramic Image Generation, In-context Image Generation, Geometry-aware Representation Learning, Image Generation, Generative Models

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_j1SaOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a strong and carefully engineered framework for panoramic generation and editing. The combination of geometry-aware pretraining, large-scale task-specific data construction, and unified RGB-only adaptation is interesting and practically valuable. The experimental results are convincing at the system level, and I believe the work would be useful to the community.

My main reservation is that the paper does not fully isolate the contribution of geometry-aware pretraining itself. The current evidence shows that the complete Canvas360 pipeline is effective, but additional controlled experiments would be needed to more clearly attribute the improvements to the proposed geometric prior learning.

Overall, I lean toward acceptance because the empirical gains, dataset contribution, and practical relevance are substantial, while the remaining concerns appear addressable through further analysis rather than fundamental flaws.

Critical strengths

The paper addresses an important challenge in panoramic generation: preserving global scene structure and boundary consistency under equirectangular projection. The proposed solution is well motivated, combining RGB-depth pretraining, modality separation, and boundary-aware modeling into a unified framework. The method achieves consistent improvements on panorama-specific metrics and several downstream tasks, showing that the proposed training strategy is practically effective.

The construction of Canvas360Dataset is a substantial contribution. The dataset provides large-scale panorama context-target pairs for multiple in-context tasks, including style transfer, inpainting, outpainting, and editing. The evaluation is also comprehensive, covering generation quality, boundary consistency, reconstruction accuracy, user preference, and ablation studies. The authors make a serious effort to analyze different components rather than only reporting final benchmark numbers.

Critical weaknesses
  1. The main claim of the paper is that geometry-aware pretraining learns a transferable panoramic prior. However, the current experiments mostly compare the complete Canvas360 pipeline against existing methods. A more controlled comparison would be important, for example:RGB-only panoramic pretraining + the same downstream fine-tuning;RGB-depth pretraining without specific geometric components;no panoramic pretraining with the same downstream data.

Such experiments would clarify whether the observed improvements are mainly due to depth-based geometric supervision, panoramic pretraining itself, or the large amount of task-specific fine-tuning data.The current ablation studies verify that individual components improve the full model, but they do not completely establish the causal role of geometry-aware pretraining in downstream transfer.

  1. The paper shows improved panorama fidelity and boundary consistency, but the evidence for learning a transferable geometric representation is still somewhat indirect. For example, the depth visualization and qualitative comparisons are helpful, but more explicit geometric evaluations (e.g., consistency of reconstructed 3D structure, viewpoint consistency, or cross-view evaluation) would strengthen the argument that the model learns a reusable geometric prior rather than only improving image statistics.

  2. RGB-depth modeling and geometric supervision have been explored in previous panoramic generation works. The strongest novelty of this paper appears to be the combination of geometry-aware pretraining with a unified RGB-only in-context editing framework and large-scale task adaptation. The paper would benefit from a clearer discussion separating what is new in the proposed framework from existing RGB-depth panoramic generation approaches.

Questions for authors

A key claim is that geometry-aware RGB-depth pretraining provides a transferable prior for downstream RGB-only tasks. Have the authors considered a matched RGB-only panoramic pretraining baseline using the same data scale and training budget? This would help isolate the contribution of depth supervision.

How much of the downstream improvement comes from Stage 1 pretraining versus the 900K context-target pairs used in Stage 2? Could the authors provide learning curves or comparisons with reduced downstream training data?

For Velocity Circular Padding, the paper reports improvements in boundary consistency for the full model. Could the authors provide direct boundary metrics (e.g., LRCE-RGB) for the VCP ablation itself to quantify its isolated contribution?

Does the learned prior generalize beyond the four evaluated tasks? For example, would the pretrained model benefit other panoramic tasks such as viewpoint-conditioned generation or 3D-aware editing?

9 comments

We thank the reviewer for recognizing Canvas360's well-motivated design for preserving global scene structure and boundary consistency, its consistent gains on panorama-specific metrics and downstream tasks, and its practical value. We also appreciate the reviewer's recognition of Canvas360Dataset as a substantial contribution and of our comprehensive evaluation and component analyses.

We particularly appreciate the suggestion to separate the effects of panoramic pretraining, depth supervision, and downstream adaptation. This closely aligns with Reviewer PbWc's recommendation and helps us strengthen the evidence for transfer. Within the rebuttal time window, we conducted a focused, small-scale controlled study using the same setting reported in our response to Reviewer PbWc. The comparisons below use 512 × 1024 panoramas, the same 20,000 downstream context–target pairs, 1,000 fine-tuning steps, and the same 500 held-out validation panoramas. B and B+D use the same pretraining panoramas and training schedules, with auxiliary depth supervision added in B+D. All other settings follow the main paper. This reduced-data study directly tests the effect of initialization under matched downstream conditions, while the learning curves provide complementary evidence about learning efficiency.

  1. Does depth supervision improve downstream transfer over matched RGB-only panoramic pretraining? (Q1, W1)

    We compared the RGB-only panoramic pretraining baseline B with its depth-supervised counterpart B+D from Table 7. Both models were adapted to the downstream tasks with RGB-only inputs and outputs under the matched setting above. We report PSNR over both the full image and the completion region, defined as the masked hole for inpainting and the region outside the input field of view for outpainting.

    TaskPretrainingLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)Completion-region PSNR ↑ (dB)
    InpaintingRGB-only (B)0.060.4723.8214.52
    RGB–depth (B+D)0.060.3924.0614.57
    OutpaintingRGB-only (B)0.616.3110.449.93
    RGB–depth (B+D)0.594.3111.4510.93

    Adding depth supervision reduced FAED by 17.02% for inpainting and 31.70% for outpainting, and improved both PSNR measures on both tasks. The gain was larger for outpainting, where completion-region PSNR increased by 1.00 dB. Inpainting LPIPS remained essentially unchanged, while outpainting LPIPS improved. These matched comparisons support an additional benefit from depth-supervised pretraining for RGB-only completion beyond RGB-only panoramic pretraining.

  2. How much does panoramic pretraining improve completion under identical downstream fine-tuning? (W1, Q2)

    We compared FLUX.1-dev LoRA, fine-tuned directly on the downstream pairs without Stage 1, with B under the same downstream data and optimization budget.

    TaskInitializationLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)Completion-region PSNR ↑ (dB)
    InpaintingFLUX.1-dev LoRA0.070.6323.0814.02
    RGB-only panoramic pretraining (B)0.060.4723.8214.52
    OutpaintingFLUX.1-dev LoRA0.657.488.828.29
    RGB-only panoramic pretraining (B)0.616.3110.449.93

    Panoramic pretraining improved all four metrics on both tasks. Completion-region PSNR increased by 0.50 dB for inpainting and 1.64 dB for outpainting. Because downstream data and training schedules were identical, the gains support the contribution of panoramic pretraining under a fixed adaptation budget. Together with B versus B+D, this separates the benefit of RGB-only panoramic pretraining from the additional benefit of depth supervision.

  1. What does removing depth supervision reveal, and does Stage 1 improve downstream learning efficiency? (W1, Q2)

    The B versus B+D comparison in item 1 removes auxiliary depth generation and supervision while retaining RGB-only panoramic pretraining. It isolates the contribution of depth under matched pretraining and downstream conditions.

    Following the reviewer's helpful suggestion, we also provide inpainting learning curves in Appendix L, Fig. 18. Orange denotes initialization with Stage 1, and blue denotes initialization without Stage 1. After training begins, the Stage-1-initialized model achieves higher PSNR and lower LPIPS at the recorded checkpoints under the same downstream training budget. The PSNR advantage is already visible early in fine-tuning and persists through 1,000 steps, supporting better adaptation within the same downstream optimization budget.

    The component ablation isolates depth supervision, while the learning curves evaluate the benefit of Stage 1 as a whole. The 900K pairs describe the constructed downstream corpus. Our additional study uses the fixed 20K subset to test whether pretraining remains useful with limited downstream supervision. We provide the learning-curve analysis requested by the reviewer, with conclusions restricted to this small-scale setting rather than extrapolated to a full-corpus data-scaling study.

  2. Does VCP itself improve horizontal boundary consistency? (Q3)

    Thank you for suggesting a metric directly aligned with VCP's purpose. Following the Table 7 ablation setting at 512 × 1024 resolution, we evaluated LRCE-RGB for the baseline, standard circular padding, VCP, and the final combination, as also reported in our response to Reviewer PbWc.

    MethodLRCE-RGB ↓ (×10⁻²)
    B1.60
    B + standard circular padding1.15
    B + VCP0.74
    B + D + PO + Lsim\mathcal{L}_{\mathrm{sim}} + VCP0.79

    VCP reduced LRCE-RGB by 53.75% relative to B and by 35.65% relative to standard circular padding. The final combination also maintained low boundary error. These results directly quantify VCP's intended effect on seam continuity, while the distributional metrics in the paper document its trade-offs on other properties.

  3. Can the geometric evidence go beyond qualitative RGB and depth visualizations? (W2)

    We appreciate the reviewer's recognition of the qualitative evidence and the constructive suggestion to make the geometric evaluation more explicit. We have added a quantitative assessment of depth relationships across completion boundaries, using the same protocol reported in our response to Reviewer PbWc. For each original and completed panorama, we estimate depth using the same estimator and measure the change in the mean log-depth difference between 5-pixel bands inside and outside the completion mask. Comparing with the original panorama accounts for genuine scene discontinuities, and the log-depth difference is invariant to global multiplicative depth scale. The metric and results are included in Appendix M.

    TaskMethodDepth boundary error ↓ (×10⁻²)
    InpaintingFLUX.1-dev LoRA2.83
    B2.63
    B+D2.43
    OutpaintingFLUX.1-dev LoRA9.18
    B7.64
    B+D7.16

    Adding depth supervision to B reduced this error by 7.60% for inpainting and 6.28% for outpainting, supporting better preservation of depth relationships across completion boundaries.

    We also include point-cloud reconstructions in Appendix M, Fig. 19, and the supplementary material. Our jointly generated depth maps use a normalized image representation through the RGB VAE and are not calibrated for precise metric reconstruction. We therefore estimate depth from the generated RGB panoramas using MoGe-2 and back-project the RGB pixels into 3D. The recovered walls, floor, and furniture provide qualitative support for coherent scene geometry in the generated RGB images, complementing the quantitative boundary evaluation. White gaps reflect missing surface samples due to occlusion and depth filtering, including the removal of points near depth discontinuities and at extreme distances.

  1. What distinguishes Canvas360 from existing RGB–depth panoramic generation methods? (W3)

    We appreciate the suggestion to clarify the methodological distinction. Canvas360's central contribution is to learn geometry within a shared panoramic generator and transfer that representation to a unified RGB-only manipulation model. The design connects depth-supervised representation learning, explicit ERP periodicity, and a common conditioning interface for four downstream tasks.

    Specifically, Canvas360 uses depth as an auxiliary generative target during pretraining. RGB and depth share the pretrained VAE and are concatenated along the token-sequence dimension within one transformer, allowing cross-modal interaction in the backbone's existing latent space without a separate depth encoder or expanded channel dimension. Modality-specific positional offsets and similarity regularization support this joint modeling. The shared weights then initialize one model for four RGB-only tasks, with neither depth input nor depth generation required at inference. Thus, the role of depth is to shape a reusable generative prior that remains useful when the depth stream is removed. The matched B versus B+D experiment supports this transfer mechanism, reducing outpainting FAED by 31.70% and increasing completion-region PSNR by 1.00 dB.

    The distinction from DreamCube is in the representation and adaptation strategy. DreamCube also reuses a pretrained VAE, but concatenates RGB and depth latents along the channel axis in a synchronized cubemap U-Net. Canvas360 instead uses sequence-level joint modeling in ERP and transfers the learned prior to a shared RGB-only context-and-prompt interface. In the matched outpainting comparison reported in our response to Reviewer PbWc, Canvas360 obtains LPIPS/FAED/PSNR of 0.42/1.79/17.16, compared with DreamCube's 0.64/3.13/11.43. This supports the effectiveness of the complete framework in that setting.

    VCP additionally couples consecutive longitude indices with soft boundary-feature synchronization before every transformer block and direct supervision of the circularly padded velocity field. It therefore incorporates horizontal periodicity into both the internal computation and the training objective. Its 35.65% LRCE-RGB reduction over standard circular padding provides direct evidence for this component's contribution.

  2. Does the pretrained prior support tasks beyond the four evaluated in the main paper? (Q4)

    We thank the reviewer for this valuable suggestion, which closely aligns with Reviewer oNzz's interest in layout-constrained composition. We have added a layout-controlled generation experiment to Appendix K, Fig. 17. Following our editing-data construction procedure, we annotated 30K examples with binary object-layout masks. A value of 1 marks object regions to retain, and 0 marks the remaining regions. These masks construct spatial reference conditions through the existing image-conditioning interface. We initialized from the Stage 1 model and fine-tuned on these examples for 20 epochs.

    These examples show that the framework can be adapted to layout-conditioned generation through task-specific fine-tuning. Viewpoint-conditioned generation and 3D-aware editing are valuable further directions. The present experiment evaluates layout-conditioned adaptation, while camera-pose-conditioned generation across translated viewpoints remains outside the current evaluation.

Reviewer_oNzzOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper presents a generative framework for panoramic imaging, obtained by fine-tuning Flow Matching and Diffusion Transformer in multiple stages; first by considering RGB-D signals with annotated depths, and then considering 4 specific tasks related to text 2 image generation, style transfer, outpainting and inpainting. Authors compile a training dataset of 900K images, and compare with SOTA baselines mostly on quality of generated images, by using also subjective assessment through a user study.

Critical strengths
  1. The paper is well organized and polished.
  2. Qualitative results appear convincing, and extensive comparisons against baselines are provided.
  3. Supplementary contain additional results.
Critical weaknesses
  1. The framework does not contain strong elements of novelty, apart of the training strategy, and the dataset compilation. The discussion of related work does not help in this sense; in most cases authors mention they built on top of previous priors, without saying how.
  2. Authors do not mention whether they want to release the dataset for replicability (the code provided in supplementary appear to be a minimal skeleton).
  3. It is not clear to me how authors decided to choose the tasks for fine-tuning the proposed framework, and more important, how the framework generalizes on different generation tasks: for example photorealistic style transfer for indoor images (Tukur et al, 2026)1^1, or more complex composition tasks (for example, force scenes to respect layout constraints).
  4. Qualitative results presentation: I would have expected in the supplementary material, some examples of generated panoramic images to be able to assess artifacts at full-scale resolution; and I would have expected assessment of the layout consistency (as mentioned by authors in line 59), for example by showcasing the point clouds obtained with RGB-D generation, especially for indoor scenes.
  5. It is not clear to me how initial depths for training are generated (but maybe I lost this detail while reading).
  6. It seems to me that their encoding is designed to support longitudinal consistency matching ERP for panoramic images, and that assumes gravity alignment. I am wondering what is the framework robustness with respect to tilt.
  7. Another very important task is novel view synthesis, but authors do not even mention it in their paper.

[1] Tukur, M., Jashari, S., Shah, U., Alzubaidi, M., Bettio, F., Pintore, G., ... & Agus, M. (2026). End-to-end framework for panoramic indoor style transfer: From photorealistic generation to immersive WebXR exploration. Image and Vision Computing, 106182.

Questions for authors
  1. Provide example related to tasks outside the ones used for training, like photorealistic style transfer or novel view synthesis
  2. Provide higher resolution qualitative examples (insets of generated images), and examples of generated panoramas
  3. Assess architectural consistency of generated panoramas (provide point clouds).
9 comments

We thank the reviewer for recognizing the clear organization and polished presentation of our paper, the convincing qualitative results, and the extensive comparisons with baseline methods. We also appreciate the reviewer's acknowledgment of the additional results in the supplementary material. The comments on novelty, reproducibility, task generalization, and geometric consistency are helpful for clarifying the contribution and strengthening the evaluation.

We welcome further discussion of how this work relates to the broader goals of panoramic imaging. Canvas360 focuses on panoramic generation and editing from a fixed camera center, with experiments primarily using gravity-aligned panoramas. Camera-translation-based novel view synthesis and robustness to arbitrary tilt are valuable complementary directions. We appreciate the reviewer bringing these perspectives to the discussion and have added experiments, visualizations, and clarifications to explain both the contribution within the current setting and the extensions it motivates.

  1. What are the methodological contributions beyond existing panoramic generation methods and circular padding? (W1)

    Canvas360 introduces a geometry-aware pretraining and transfer formulation that makes depth supervision useful for RGB-only panoramic manipulation. During pretraining, RGB and depth are jointly generated through the same pretrained VAE and transformer. Their latents are concatenated along the token-sequence dimension, enabling cross-modal interaction without a separate depth encoder or expanded backbone channels. Modality-specific positional offsets and similarity regularization support joint modeling in this shared latent space. The learned weights then initialize a unified model for four RGB-only tasks, with no depth input or depth generation at inference. The methodological contribution lies in how geometry is learned in the shared generator and retained for downstream RGB manipulation. In the matched B versus B+D control, depth supervision reduces outpainting FAED by 31.70% and increases completion-region PSNR by 1.00 dB (Appendix L and our response to Reviewer PbWc, item 3).

    VCP provides a second methodological contribution by making ERP periodicity explicit in both transformer computation and velocity supervision. With copied positional indices, standard circular padding repeats boundary content without representing adjacent longitude positions across the seam. VCP instead assigns consecutive longitude indices, softly synchronizes boundary features before every transformer block, and directly supervises the circularly padded velocity field. These mechanisms couple the representation of the seam to the flow-matching objective throughout the network. The matched ablation reduces LRCE-RGB by 35.65% over standard circular padding, directly supporting this design (our response to Reviewer PbWc, item 5).

  2. Will the dataset and a complete implementation be released for reproducibility? (W2)

    We have already committed to releasing the code, trained model weights, and dataset in the paper's Reproducibility Statement, together with documentation of data sources and intended-use restrictions. Release links were omitted to preserve anonymity during double-blind review. To address the concern about the supplementary code, the release will include the complete training, inference, and evaluation implementations, the configurations corresponding to the reported experiments, and data-preparation documentation. We will make the release commitment and the contents of the reproducibility package more prominent.

  3. Why were style transfer, inpainting, outpainting, and editing selected? (W3)

    These four widely used tasks address complementary needs in constructing, modifying, and diversifying panoramic environments. Their relevance extends to virtual-scene creation for embodied AI, where panoramas can provide a compact visual representation of the surrounding environment. Panorama-based world-generation systems such as HunyuanWorld 1.0 illustrate the connection between panoramic synthesis and subsequent 3D scene construction.

    Outpainting expands a limited real-image observation into a full-surround visual context, providing an entry point for real-to-sim scene construction. Inpainting reconstructs missing regions, while editing changes objects and scene content to support environment modification. Style transfer diversifies appearance while retaining scene structure, enabling different visual variants of an environment. All four operations can be expressed through the same RGB-reference-and-text interface, making them a coherent set of tasks for evaluating a reusable panoramic prior.

  1. Why evaluate layout-constrained composition as the additional task, and how does photorealistic style transfer relate to the current setting? (W3, Q1)

    We appreciate the suggestion to examine photorealistic indoor style transfer, including the setting discussed in Tukur et al. (2026). Style transfer with a separate style-reference image introduces an additional visual condition beyond the content panorama. In our token-concatenation formulation, supporting this setting would involve three image-token streams for the content image, the style reference, and the noisy target latent. Our current model instead uses one clean RGB reference and a noisy target latent, with the desired transformation specified by text. Supporting a separate style image would therefore require extending the conditioning interface and training setup. We chose layout-constrained composition to evaluate adaptation to an additional task while retaining the existing reference-image interface.

    We tested layout-constrained composition using the data-construction procedure for editing. We construct a binary mask that assigns 1 to regions containing objects to preserve and 0 to the remaining regions. We use this mask to construct the reference condition, providing spatial guidance for object arrangement through the existing reference-image interface. We annotated 30K examples using this procedure and fine-tuned the stage-one pretrained model for 20 epochs. Appendix K, Fig. 17 presents inference results from the adapted model. These examples show that the framework can be adapted to layout-conditioned generation through task-specific fine-tuning.

  2. How does Canvas360 relate to novel view synthesis? (W7, Q1)

    We appreciate the opportunity to clarify the role of depth in our framework. Canvas360 uses depth supervision to improve the scene structure of panoramic generation and its transfer to RGB-only manipulation. Its current formulation generates panoramas from a fixed camera center and does not include camera-pose-conditioned synthesis across translated viewpoints. Novel view synthesis is a valuable extension, particularly for connecting panoramic generation with navigable virtual environments. We will explicitly discuss this direction and clarify the scope of the current framework.

  3. Can full-resolution panoramas and enlarged qualitative examples be provided? (W4, Q2)

    Yes. We have added individual panoramas at their native 1024 × 2048 resolution to the supplementary material, together with enlarged crops in the revised appendix. These examples allow inspection of fine details and artifacts, including structures that cross the horizontal boundary, without the downscaling required by the current multi-image figures.

  4. Can architectural consistency be assessed through point-cloud visualizations? (W4, Q3)

    Yes. We have added point-cloud visualizations in Appendix M, Fig. 19, and the supplementary material, with an emphasis on indoor scenes. Our jointly generated depth maps use a normalized image representation through the RGB VAE and are not calibrated for precise metric reconstruction. We therefore estimate depth from the generated RGB panoramas using MoGe-2 and back-project the RGB pixels into 3D. The recovered wall, floor, and object structures provide qualitative support that the generated RGB panoramas capture coherent scene geometry. White gaps reflect missing surface samples due to occlusion and depth filtering, including the removal of points near depth discontinuities and at extreme distances.

  5. How are the initial depth maps used for training generated? (W5)

    The Geometry-aware Text-to-Panorama Pretraining subsection specifies that we estimate panoramic pseudo-depth using DAP. The Pseudo-depth Processing paragraph in the appendix describes the subsequent processing. We clip the estimated depth at 10 for indoor scenes and 100 for outdoor scenes, normalize the clipped maps, and encode them with the VAE. The processed maps serve as auxiliary targets for joint RGB–depth generation. Both modalities are generated from noise during pretraining, and clean depth is never supplied as a conditioning input. We will make this pipeline easier to locate in the revised presentation.

  1. Does the framework assume gravity alignment, and how does this relate to tilted panoramas? (W6)

    Our training data predominantly contain gravity-aligned ERP panoramas, and the training augmentation uses horizontal yaw shifts. The reported experiments establish performance in this upright panoramic setting. Gravity alignment concerns the data distribution, whereas horizontal periodicity remains a property of ERP even after a spherical rotation.

    When a tilted view of a generated scene is desired, the panorama can be rotated on the sphere and resampled into ERP, changing its viewing orientation without regenerating the scene. This provides a direct way to obtain tilted outputs from the generated panoramas. Robustness to arbitrarily tilted conditioning images is a separate question that the current experiments do not measure. We will clarify this distinction and the orientation setting used in our evaluation.

Reviewer_PbWcOpenReview ↗
Rating
3
Confidence
4
Informativeness
3
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes Canvas360, a two-stage framework for panoramic image generation in equirectangular projection. The authors claim three main contributions. First, they transfer a geometry-aware panoramic prior to four in-context tasks through a single RGB-only model with a shared flow-matching objective. Second, they develop parallel RGB-depth pretraining on pseudo-depth, using positional offsets to separate modalities, a similarity regularizer, and velocity circular padding to encode horizontal continuity. Third, they construct Canvas360Dataset, with 100K annotated RGB-depth panoramas and 900K context-target pairs for the four tasks.

Critical strengths

S1. The method gives consistent gains across all four in-context tasks. Inpainting, outpainting, and editing beat the strongest baseline on LPIPS, FAED, and PSNR, and the text-to-panorama results show a large FAED gap and the lowest boundary error.

S2. The paper is transparent about its limitations. The appendices acknowledge that depth’s downstream contribution is not isolated, that geometry checks are qualitative, and that DiT360 has lower FID, and they report detailed training settings and dataset construction.

Critical weaknesses

W1. The central claim (transfer of depth-based pretraining to downstream tasks) is supported only qualitatively and for completion. The only controlled comparison under the same fine-tuning setting is Fig. 9 (vs. FLUX.1-dev), no downstream experiment isolates the contribution of depth (Appendix E), and geometric consistency is evaluated qualitatively apart from boundary error (Appendix J).

W2. The effects of individual components are mixed across metrics, and some are not validated with the metric matching their purpose. VCP targets boundary continuity, but LRCE-RGB is reported only in the cross-method comparison (Table 6), not in the VCP ablation. In Table 7, B+VCP worsens FID and FAED, and the full model has worse FID, FID_pole, and BRISQUE than the RGB baseline.

W3. For the downstream comparisons (Tables 8–10), the training conditions of the baselines and the source of the evaluation data are not specified. In addition, DreamCube, an RGB-D panorama generation method, is cited but not included in the T2P comparison.

Questions for authors

Q1 (W1). Could you quantify the comparison in Fig. 9 (pretrained Canvas360 vs. FLUX.1-dev) on at least one task with LPIPS/FAED/PSNR, and report a metric that directly measures geometric consistency?

Q2 (W2). Could you provide an ablation of LRCE-RGB with and without VCP (and with standard circular padding), together with results for the final combination?

Q3 (W3). What data were the downstream baselines trained on, and where do the 500 evaluation inputs and reference images per task come from? Is a comparison with DreamCube possible?

5 comments

We thank the reviewer for highlighting Canvas360's performance across the four in-context tasks, particularly the gains in LPIPS, FAED, and PSNR for inpainting, outpainting, and editing. We also appreciate the reviewer's recognition of the text-to-panorama results on FAED and boundary continuity, and of our explicit discussion of what the current experiments establish about depth supervision and geometry.

We appreciate the reviewer's helpful suggestions on evaluating downstream transfer and measuring geometric consistency.

  1. Can the full-model comparison in Fig. 9 be quantified? (W1, Q1)

    Yes. We first quantify the original comparison between pretrained Canvas360 and directly fine-tuned FLUX.1-dev using the inpainting protocol of Table 9. Both models use the same downstream fine-tuning setting. The Canvas360 row reproduces the inpainting results in Table 9.

    MethodLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)
    Directly fine-tuned FLUX.1-dev0.1180.41525.44
    Canvas3600.0960.37125.87

    Canvas360 reduces LPIPS by 18.64% and FAED by 10.60%, and improves PSNR by 0.43 dB. This quantifies the transfer benefit of the complete pretrained model shown in Fig. 9. We additionally conduct the smaller controlled study below to separate the contributions of RGB-only panoramic pretraining and auxiliary depth supervision.

Setting for the additional W1 ablations. At 512 × 1024 resolution, we initialize the downstream models from the RGB-only panoramic checkpoint B and the RGB–depth checkpoint B+D in Table 7. These checkpoints use the same pretraining panoramas and training schedules, with auxiliary depth supervision added in B+D. We fine-tune both on the same 20,000 downstream context–target pairs for 1,000 steps. The FLUX.1-dev LoRA control starts from the original backbone and uses the same downstream data and optimization budget, without Stage 1. Other settings follow the main paper. All three models are evaluated on the same 500 validation panoramas held out from all Canvas360 training. We report PSNR over both the full image and the completion region, which is the masked hole for inpainting and the region outside the input field of view for outpainting.

  1. How much does panoramic pretraining improve completion? (W1)

    We compared the directly fine-tuned FLUX.1-dev LoRA baseline with B under the setting above to quantify the benefit of RGB-only panoramic pretraining.

    TaskMethodLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)Completion-region PSNR ↑ (dB)
    InpaintingFLUX.1-dev LoRA0.070.6323.0814.02
    B0.060.4723.8214.52
    OutpaintingFLUX.1-dev LoRA0.657.488.828.29
    B0.616.3110.449.93

    Panoramic pretraining improved all four metrics on both tasks. The completion-region PSNR increased by 0.50 dB for inpainting and 1.64 dB for outpainting, supporting improved reconstruction within the completion regions.

  2. What does depth supervision contribute to downstream completion? (W1)

    We compared B with B+D using the same pretraining panoramas, training schedules, downstream pairs, and validation inputs to assess the contribution of auxiliary depth supervision.

    TaskMethodLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)Completion-region PSNR ↑ (dB)
    InpaintingB0.060.4723.8214.52
    B+D0.060.3924.0614.57
    OutpaintingB0.616.3110.449.93
    B+D0.594.3111.4510.93

    Adding depth supervision reduced FAED by 17.02% for inpainting and 31.70% for outpainting. It improved both full-image and completion-region PSNR on both tasks. Inpainting LPIPS was essentially unchanged, while outpainting LPIPS decreased by 3.28%.

  1. Does depth supervision improve geometric consistency at completion boundaries? (W1)

    We estimated depth from the original and completed panoramas using the same estimator. For each mask, we extracted 5-pixel bands on its inner and outer sides, denoted by Ωiin\Omega_i^{\mathrm{in}} and Ωiout\Omega_i^{\mathrm{out}}. We measured the absolute change in their mean log-depth difference and averaged over the validation panoramas.

    Δi(D)=1∣Ωiin∣∑p∈Ωiinlog⁡D(p)−1∣Ωiout∣∑p∈Ωioutlog⁡D(p).\Delta_i(D)=\frac{1}{|\Omega_i^{\mathrm{in}}|}\sum_{p\in\Omega_i^{\mathrm{in}}}\log D(p)-\frac{1}{|\Omega_i^{\mathrm{out}}|}\sum_{p\in\Omega_i^{\mathrm{out}}}\log D(p).

    E=1N∑i=1N∣Δi(Dicompleted)−Δi(Dioriginal)∣.E=\frac{1}{N}\sum_{i=1}^{N}\left|\Delta_i(D_i^{\mathrm{completed}})-\Delta_i(D_i^{\mathrm{original}})\right|.

    This error is invariant to global multiplicative depth scale and accounts for genuine depth discontinuities in the original panorama.

    TaskMethodDepth boundary error ↓ (×10⁻²)
    InpaintingFLUX.1-dev LoRA2.83
    B2.63
    B+D2.43
    OutpaintingFLUX.1-dev LoRA9.18
    B7.64
    B+D7.16

    Adding depth supervision to B reduced this error by 7.60% for inpainting and 6.28% for outpainting, supporting better preservation of depth relationships across completion boundaries.

  2. Does VCP improve horizontal boundary continuity compared with standard circular padding? (W2)

    We followed the experimental setting of Table 7 at 512 × 1024 resolution and evaluated LRCE-RGB for the baseline, standard circular padding, VCP, and the final combination.

    MethodLRCE-RGB ↓ (×10⁻²)
    B1.60
    B + standard circular padding1.15
    B + VCP0.74
    B + D + PO + Lsim\mathcal{L}_{\mathrm{sim}} + VCP0.79

    VCP reduced LRCE-RGB by 53.75% relative to B and by 35.65% relative to standard circular padding. These results quantify the seam-alignment improvements shown in our qualitative ablation. The final combination also maintained low boundary error, although B+VCP obtained the lowest value.

  3. What are the results of adding VCP to the full RGB–depth configuration? (W2)

    Under the same Table 7 setting, we evaluated the cumulative RGB–depth configuration with and without VCP.

    MethodFID ↓FIDpole\mathrm{FID}_{\mathrm{pole}} ↓FIDequ\mathrm{FID}_{\mathrm{equ}} ↓FAED ↓IS ↑CS ↑QAquality\mathrm{QA}_{\mathrm{quality}} ↑QAaesthetic\mathrm{QA}_{\mathrm{aesthetic}} ↑BRISQUE ↓NIQE ↓
    B + D + PO + Lsim\mathcal{L}_{\mathrm{sim}}51.4855.6329.744.931.8834.734.754.2216.233.84
    B + D + PO + Lsim\mathcal{L}_{\mathrm{sim}} + VCP51.6254.9528.894.961.8634.724.794.2513.883.81

    Adding VCP improved both regional FID scores, Q-Align quality and aesthetics, BRISQUE, and NIQE. Overall FID and FAED increased slightly, while IS and CS decreased slightly. These results show the measured trade-offs of the final combination.

  4. How do we interpret the mixed effects across metrics? (W2)

    We agree that the component ablations show trade-offs across metrics, and we will make these trade-offs explicit. Our FID implementation resizes each 2:1 ERP panorama to 299 × 299 before extracting Inception features, altering its aspect ratio and panoramic spatial structure. This preprocessing, together with features learned from perspective images, may limit sensitivity to ERP geometry and seam continuity. This is a limitation of the measurement, rather than an established explanation for the observed FID increases. We retain FID as a measure of distributional fidelity and complement it with panorama-trained FAED and the direct boundary metric LRCE-RGB.

    Our observations illustrate why these measures are complementary. VCP makes a focused change to boundary handling, yet it produces clearer seam alignment and the LRCE-RGB reductions reported above. Adding depth alone lowers FAED from 5.37 to 4.74 in Table 7, but we observed dark outputs for some prompt–seed combinations. These failures motivated positional separation and similarity regularization. The combined RGB–depth variant produced fewer dark artifacts in the inspected examples and raised Q-Align quality and aesthetics from 4.23/3.41 to 4.75/4.22, while FAED rose from 4.74 to 4.93. The visual improvements were therefore only partly reflected in the distributional scores. We report these trade-offs and use the direct boundary measurement to substantiate VCP's intended effect.

  1. How were the downstream baselines evaluated, and where do the 500 evaluation examples per task come from? (W3, Q3)

    Thank you for pointing out that the evaluation protocol was insufficiently specified. For Tables 8–10, we use the baselines' officially released checkpoints or APIs, without additional fine-tuning on Canvas360Dataset. We will clarify the baseline usage and evaluation-data construction in the revision.

    For each task, we construct 500 evaluation examples from Matterport3D panoramas using the task-specific processing and input–target construction described in the paper. Neither the selected source panoramas nor any images derived from them through post-processing were used in any stage of Canvas360 training, including pretraining and downstream fine-tuning. Inpainting and outpainting retain the original complete panorama as the reference, while style-transfer and editing pairs follow their respective construction procedures. All methods are evaluated on the same task-specific examples.

    We chose Matterport3D because it is an established panoramic benchmark used by several panorama-specific baselines, including PAR and SE360. This provides a common scene domain and reduces dependence on the web and synthesized components of our own corpus. The comparisons with general image models such as FLUX assess how well strong image generators and editors handle ERP distortion and horizontal wrap-around, and whether dedicated panoramic modeling offers practical benefits. These comparisons evaluate complete systems with their respective training histories. The matched 20K-pair experiments above separately assess pretraining and depth supervision under identical downstream adaptation conditions.

  2. Can DreamCube be included in the comparison? (W3)

    Thank you for suggesting this comparison. For DreamCube, the input consists of a single perspective RGB view, its corresponding depth map, and a text prompt. We predict these depth maps specifically for the validation inputs. This image-conditioned setting closely matches the outpainting task in Table 9. DreamCube and Canvas360 were evaluated on the same validation set with the same text prompts. We evaluated the generated panoramas using LPIPS, FAED, and full-image PSNR. The Canvas360 results below are from Table 9.

    TaskMethodLPIPS ↓FAED ↓Full-image PSNR ↑ (dB)
    OutpaintingDreamCube0.643.1311.43
    Canvas3600.421.7917.16

    Canvas360 achieved lower LPIPS and FAED and 5.73 dB higher PSNR in this comparison.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.