acceptodds
Under review as a conference paper at ICLR 2027

PanoWeaver: Generative Virtual Camera for Panoramic Reframing

Abstract

Panoramic cameras capture a complete field of view in a single shot, yet converting such omnidirectional observations into well-composed photographs remains challenging. To address this issue, we introduce panorama reframing, a new photography-oriented task that jointly determines where to look and what the resulting perspective view should look like. Then, we propose PanoWeaver, a coupled camera-image generation framework that treats the spherical field of view (SFoV) as a virtual camera action and the perspective image as its visual outcome. Built upon a DiT backbone, PanoWeaver jointly generates both through bidirectional interaction, coupling explicit camera inference with generative visual modeling to learn camera–image correspondence. Moreover, we construct PanoComp-100K, a large-scale panoramic composition dataset containing 100,200 samples with SFoV states, perspective views, and human composition annotations. Extensive experiments demonstrate that PanoWeaver outperforms existing alternatives in panorama reframing, enabling a practical shoot-first and reframe-later paradigm for 360° photography. The source code, trained models, and dataset will be made available to the public.

Primary area:
generative models
Keywords:
panoramic image composition, panorama reframing, joint camera–image generation, text-conditioned image generation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_yNDhOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PanoWeaver, a model for finding the desired perspective view from an input panoramic image given text condition. The author proposed a hybrid framework that jointly estimates the camera parameters and denoises the target rgb image. To train such a model, the author collected 100K panoramas with corresponding perspective views. Experimental results demonstrated certain amount of improvement upon existing methods.

Critical strengths
  • This paper is mostly well-written and easy-to-follow.
  • The 100K image labeling seems a huge effort for human to label themselves.
Critical weaknesses
  1. Insufficient justification for joint image generation. The target view is a deterministic projection of the input panorama given a low-dimensional camera state. The paper lacks a controlled baseline that fuses visual and text features (e.g., via standard attention layers), directly predicts camera parameters, and renders the resulting view.

  2. Limited gains and missing efficiency evaluation. In the text-free setting, the proposed method achieves comparable rather than superior spherical IoU to UNIC (0.465 vs. 0.468). Given the 9B backbone and 50-step sampling procedure, the absence of inference latency and computational cost, it is unclear whether this simple task worth such a complicated model to handle.

  3. Missing iterative rendering-feedback baseline. Although the paper includes VLM baselines, it does not report a tool-assisted agentic approach that iteratively proposes camera parameters, inspects the actual rendered view, and refines the framing. I recommend the authors to directly test this task with general-purpose coding agent harness (e.g., claude code or codex). This would directly test whether visual feedback can improve camera selection without jointly generating image pixels.

  4. Evaluation does not clearly isolate the claimed benefits. Spherical IoU directly measures agreement with the annotated viewport, whereas improvements in PSNR, SIFT, or VGGT do not necessarily establish better photographic composition. Given that the degree of freedom is only five, that many metrics seem unnecessary. Besides, given that the paper aims at helping people with photography, I think a human preference score is needed for evaluation.

  5. Minor issues

  • The paper distinguishes their task as a "new photography-oriented task," however, the task is already handled in existing frameworks as mentioned in its own related works section.
  • Figures 6 and 7 are too small to clearly see the difference between images.
  • Reference formatting is inconsistent, e.g., some entires use full venue names while others use abbreviated names.
Questions for authors
  • How did you collect the 100K labels just by human? Can you disclose the details, e.g., how many people worked on it and how much time it took on average to label one image? It seems a very huge amount of work.

Besides, please address my concerns raised in the weaknesses section.

1 comment

Thank you for the detailed rebuttal. However, some concerns remain unresolved.

  • Regarding model size and computational cost, I am still not fully convinced why this discriminative task requires a diffusion model taking 105 seconds per image. As a thought experiment, suppose we approach ImageNet classification through joint image generation and label prediction, increasing the model size from tens of millions to tens of billions of parameters and inference time from a fraction of a second to roughly 100 seconds. While this might improve performance, the gains could simply come from the increased model capacity and compute. The CLIP ViT-B/32 baseline is still roughly two orders of magnitude smaller than the proposed model. Without comparisons under similar parameter and compute budgets, it is difficult to attribute the gains to the proposed formulation. Furthermore, even if the formulation contributes, I remain unconvinced that the improvement justifies such a substantial computational cost for this task.

  • Regarding the agentic baseline, the iteration budget is limited to three rounds, and the underlying model is GPT-5.6-Sol. Would a stronger model with a larger iteration budget match or surpass the proposed method? Given the improvement from one to three rounds, this seems worth investigating.

Reviewer_ThjoOpenReview ↗
Rating
3
Confidence
4(+1)
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper works on panorama reframing, namely selecting the proper FoV within a panorama and enhancing it.

The application include two modes:

  • text-guided reframing: follow prompts to propose the relevant SFoV
  • text-free reframing: automatically select the best SFoV The SFoV will be then further enhanced with a generative model.

To enable those applications, the paper proposes to use a panorama-conditioned image diffusion model. It uses multi-stream attention with token-concatenation of [noisy image latent, noisy SFoV latent | clean panorama latent, clean text embedding]. The final output is [clean image, clean SFoV] after decoders. Since SFoV token is a new modality where the base model (FLUX2-klein) hasn't seen before, dedicated AdaLN / attention layers are injected separately for full-finetuning. Other modules are adapted with LoRA (rank=16).

To train this model, the authors created a dataset with human annotation AND VLM evaluation. The dataset consists of 100K panoramas curated from youtube.

To evaluate this model, the authors incorporated multiple metrics.

  1. region proposal correctness via mIoU
  2. pixel space geometry similarity via VGGT consistency
  3. pixel space photometric similarity via PSNR
  4. neural image assessment via NIMA

The paper produces comparable neural assessment results and good region proposals.

Critical strengths
  • The paper is very easy to read. The presentation is clear and informative.
  • The approach is simple but solid. It could be easy to reproduce for any person with a relevant background.
  • The joint FoV proposal and image reframing in the diffusion setup with joint attention looks neat and consistent.
  • The data contribution is substantial, including heavy human annotations.
  • The evaluation metrics have been carefully designed and are plausible.
Critical weaknesses
  • The paper works on a relatively niche field, namely panorama reframing. The applications and impact could be relatively limited. My personal take is that
  1. an invert problem (image to panorama) can be more challenging, harder to evaluate, but more informative for generative world modeling
  2. further, a video version of the dataset and the pipeline can be much impactful: annotated pair of limited FoV AND the entire world evolving together could evoke many interesting explorations.
  • The paper does not present consistent SOTA results in Table 1 & 2. The ranking is a bit scattering across metrics.
  • The choice of PSNR is questionable. PSNR is extremely sensitive to minor pixel shifts and the metric is not reliable in the problem setup. LPIPIS or DISTS are better metrics.
  • NIT: the Venue/Year column in the tables looks AI slop and could be removed. Citations already included the venues.
Questions for authors
  • How difficult would it be to generalize the annotation pipeline to videos? How many frames/hours of raw data have you curated? Those are extremely valuable video sources and I greatly recommend an extension on this topic in the follow-up works.
  • Are there ablation of the LoRA rank? 16 seems pretty conservative, does it mean the task isn't very hard and FLUX has seen panorama data during pretraining? I wonder if it will work zero-shot after finetuning the adaptor for the camera proposal part directly.
Reviewer_66wmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper presents a method for panorama reframing, i.e., selecting a photographic composition from a panorama and producing the corresponding image (optionally conditioned on text). It augments a pretrained diffusion transformer with a camera pathway and jointly denoises the SFoV state and image through shared attention. It also introduces a dataset, named as PanoComp-100K, containing > 10K examples. Experiments demonstrate the effectiveness of the proposed method.

Critical strengths
  1. The paper tries to solve a useful and interesting problem.
  2. A new dataset suitable for the proposed problem is proposed, and it contains panorama–perspective pairs with captions.
  3. The evaluation metric of VGGT-based score for viewpoint consistency is proposed and sounds meaningful.
Critical weaknesses
  1. The proposed pipeline is rather straightforward to me, and I didn't see any novel design specified for such problem.
  2. It has two branches, while it does not establish the benefit of joint generation. For example, a simple baseline is to first predict the camera and then conduct native-panorama projection.
  3. The evaluation is also not very thorough. It seems that the 200-image validation set has one reference composition per panorama and automatically generated captions. How about using different caption sources like from user input? The user study is also expected.
  4. As the paper pointed out, the exploration of video frames is missing and needs further study.
Questions for authors

Please see the weaknesses part.

Comments on the paper

1 comment

In the future, I recommend looking at few step LoRAs, e.g. https://huggingface.co/NoiseToNova/Flux-3-Klein-turbo-pdd-4_8-steps. In general diffusion models have an interesting property: if you train a few-step LoRA and a task-specific LoRA on the base model separately, when you stack them together, it often works out-of-box and gives you reasonable performance gain (2x from no CFG and 5x from fewer steps) without losing too much generation ability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.