acceptodds
Under review as a conference paper at ICLR 2027

SEG: Leveraging Self-Expanding Geometric State for Successive View Synthesis

Abstract

Recent advances in both reconstruction and multi-view diffusion have greatly improved the quality and geometric consistency of novel view synthesis (NVS). However, most existing methods either adopt joint multi-view synthesis for predetermined target views, or rely on video generation models for continuous synthesis, which limits their ability in successive view synthesis where incremental and unordered view queries are required. To address this limitation, we present SEG, a successive view synthesis framework built around a Self-Expanding Geometric state. The key insight of SEG is to turn context-dependent geometric latent into self-expanding states with consistency during expansion and causality under successive queries. Therefore, we construct our SEG state from early geometric encoder level based on consistency analysis, and design causal-prefix ground-truth state for supervision. We pair these latents with KV reference memory to form the final SEG state for efficient successive queries. Based on SEG state, we design our pipeline with a causal geometric diffusion model for state synthesis and expanding, and introduce a geometric adapter to map generated latents to multi-level features for RGB readout. We show that SEG state provides a well-suited solution for successive NVS, including extensive experiments on RealEstate10K and DL3DV with strong view synthesis quality, viewpoint accuracy, and computational efficiency.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Successive View Synthesis, Geometric State, Self-Expanding

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_oM3iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes SEG for successive novel view synthesis with incremental and unordered viewpoint queries. Its key idea is a self-expanding geometric state (S_k=(Z_k,M_k)), which combines geometric latents extracted from DA3 with KV reference memory to reuse previously generated content. The framework further introduces causal-prefix supervision to avoid future-view information, causal geometric diffusion for state expansion, and a Geometric Adapter for RGB readout.

Experiments on RE10K and DL3DV show improvements in synthesis quality, camera pose accuracy, and inference efficiency over the evaluated baselines. Ablations further evaluate the contributions of the state design, causal-prefix supervision, and robustness to unordered queries.

Critical strengths
  1. Meaningful problem setting and reasonable geometric state design.

The paper addresses successive NVS with incremental and unordered viewpoint queries, which is practically relevant for interactive scene exploration. Leveraging geometry-pretrained latents with causal-prefix supervision and reusable KV memory provides a reasonable approach to accumulating generated context while maintaining causal conditioning.

  1. Promising empirical results.

SEG consistently improves image quality and pose accuracy over the evaluated baselines on RE10K and DL3DV, while achieving lower inference cost than the compared generative approaches. The component ablations and unordered-query evaluation further support the effectiveness of the proposed pipeline.

Critical weaknesses
  1. Insufficient comparison with closely related autoregressive NVS methods.

CausNVS (Kong et al., 2025) already supports incremental and unordered viewpoint queries through causal diffusion and pose-aware KV caching. Although cited, it is not included in the experimental comparisons. This makes it difficult to assess SEG's improvement over existing methods addressing essentially the same problem setting.

  1. Limited evidence for the contribution of geometric latent representation.

The main distinction of SEG from existing causal NVS methods (e.g. CausNVS) lies in generating and retaining geometry-pretrained latents rather than conventional image latents. However, the current ablations mainly validate state expansion and individual components, without isolating the advantage of geometric latents over conventional latent representations with comparable causal KV memory. Thus, the effectiveness of the paper's most distinctive technical contribution remains insufficiently established.

Questions for authors

See weakness.

5 comments
AuthorsOpenReview ↗

Official Response to Reviewer oM3i (Part 1/2)

We sincerely thank Reviewer oM3i for the constructive review and insightful questions. We address all the points raised in review as follows:

W1. Comparison with CausNVS

Thank you for raising this comparison. Besides the choice of geometric state representation, we highlight two further differences:

  1. Original motivation. Our key insight is that retained history should support successive queries across geometrically unordered viewpoints. This motivates our joint design of a geometric state and causal-prefix supervision to maintain geometric consistency while supporting incremental queries. CausNVS focuses on streaming queries, whereas our work emphasizes geometric reasoning across successively queried, unordered viewpoints, with carefully designed evaluation settings, 3D metrics and comparisons.

  2. Causal exploration with a geometric state. While CausNVS uses a causal pipeline with 2D VAE latents, our geometric state introduces additional challenges: maintaining representation stability during state expansion and preventing future-view information leakage during training. We address both through our geometric state design and causal-prefix supervision, and evaluate their effectiveness experimentally.

We have expanded the CausNVS discussion in Section 2 (Related Work) and Appendix E (Discussion).

As we could not access an official implementation or pretrained checkpoints for CausNVS through its project page or public repositories when conducting these experiments, we reimplemented CausNVS based on the original paper. We compare this reimplementation with SEG under the evaluation protocol of Table 3 in our paper:

DatasetMethodPSNR ↑SSIM ↑LPIPS ↓Observed PSNR ↑Observed SSIM ↑Unobserved PSNR ↑Unobserved SSIM ↑
RE10KCausNVS*14.010.5250.53014.120.52813.440.471
RE10KSEG (ours)14.680.5420.45914.830.54714.010.476
DL3DVCausNVS*13.710.4030.52713.860.40913.300.365
DL3DVSEG (ours)15.080.4610.47215.230.46714.710.424

As shown above, SEG achieves better performance than our CausNVS reimplementation on both datasets. We further assess 3D geometric consistency through camera pose accuracy as reported below, which verifies our claim that geometry is essential in successive NVS:

DatasetMethodATE ↓RPE-t ↓RPE-r ↓
RE10KCausNVS*0.3270.4423.251
RE10KSEG (ours)0.1570.2041.687
DL3DVCausNVS*0.4200.5743.488
DL3DVSEG (ours)0.1330.1851.322

We have incorporated these comparisons into Table 3 for image quality and Table 4 for camera pose accuracy.

AuthorsOpenReview ↗

Official Response to Reviewer oM3i (Part 2/2)

W2. About geometric representation

Thanks. During the rebuttal period, we have conducted experiments comparing a 2D VAE variant with SEG. Specifically, we replace the SEG state with the 2D VAE latents of Stable Diffusion XL [1] while keeping all other settings unchanged. The results are shown in the following table:

DatasetSettingsPSNR ↑SSIM ↑LPIPS ↓
RE10Kw/ 2D VAE state13.820.5280.597
RE10Kw/ SEG state (ours)14.680.5420.459
DL3DVw/ 2D VAE state13.740.4440.639
DL3DVw/ SEG state (ours)15.080.4610.472

Despite using the same causal pipeline and training strategy, the 2D VAE variant produces lower-quality renderings. We attribute this gap to poorer alignment with the queried camera poses during successive and unordered queries. In addition to rendering quality, we evaluate geometric consistency through camera pose accuracy, which is particularly important for constructing spatially coherent scene representations. We follow the evaluation protocol used in Table 4 of our paper and report the results below:

DatasetSettingsATE ↓RPE-t ↓RPE-r ↓
RE10Kw/ 2D VAE state0.3780.5063.129
RE10Kw/ SEG state (ours)0.1570.2041.687
DL3DVw/ 2D VAE state0.4990.5824.420
DL3DVw/ SEG state (ours)0.1330.1851.322

The 2D VAE variant also performs substantially worse in camera pose accuracy. This indicates that the choice of state representation contributes to viewpoint accuracy beyond the benefits of history reuse and causal modeling. In particular, geometric features provide spatial cues that help align successive predictions with the queried camera poses. These results are now included in Section 4.4 of the revised paper, i.e. Table 9 reports 2D image quality, and Table 10 reports camera pose accuracy.

We have also added Figure 6 in Section 4.4 to compare the rendered views and estimated camera poses of the two state representations. Our SEG preserves the arrangement of scene structures and their alignment with the queried viewpoints more consistently across successive generations.

[1] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. International Conference on Learning Representations (ICLR), 2024.

Dear Reviewer oM3i,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 21404

Reviewer_VbZhOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SEG addresses successive novel-view synthesis with incrementally arriving, spatially unordered camera queries. It combines a geometric latent archive with cached attention memory, using causal-prefix targets to train latent generation and a geometric adapter for RGB readout. Experiments on RealEstate10K and DL3DV evaluate synthesis quality, camera-pose accuracy, query-order robustness, and computational efficiency.

Critical strengths
  1. The state construction addresses a concrete difficulty in reusing multiview features. Extracting each training target from its own image prefix avoids future-view information in earlier latent targets, while the expanding archive and cached attention enable reuse across queries. The causal-prefix ablation supports this design.

  2. The method achieves a useful combination of quality and efficiency. Table 3 reports PSNR improvements of 0.87 dB on RealEstate10K and 1.04 dB on DL3DV over the strongest respective baselines, with lower runtime and memory than the tested generative methods. Table 8 also shows substantially smaller degradation under query shuffling.

Critical weaknesses
  1. The specific advantage of the geometric state is insufficient isolation. Table 7’s fixed-state control removes generated-history reuse, while the VGGT variant still uses geometric features. Neither comparison establishes whether the improvement requires geometric latents rather than ordinary image latents with the same causal architecture and cache.

  2. Persistent scene consistency is not directly established. The longer-rollout and query-order experiments report aggregate reconstruction metrics but do not measure whether revisiting a camera preserves previously generated content. This distinction matters for the proposed interactive exploration setting.

Questions for authors
  1. With the causal backbone and cache held fixed, how does the proposed geometric state compare with an image-latent history representation?

  2. Under prefix-only RGB readout, how consistent are repeated predictions at the same camera after intervening queries, and how does this consistency vary across query histories?

2 comments
AuthorsOpenReview ↗

Official Response to Reviewer VbZh

We sincerely thank Reviewer VbZh for the constructive review and insightful questions. We address all the points raised in review as follows:

W1 & Q1. Advantage of the geometric state

Thanks for suggesting a controlled ablation using 2D image latents within the same causal pipeline. During the rebuttal period, we have conducted experiments comparing a 2D VAE variant with SEG. Specifically, we replace the SEG state with the 2D VAE latents of Stable Diffusion XL [1], keeping all other settings unchanged. The results are shown in the following table:

DatasetSettingsPSNR ↑SSIM ↑LPIPS ↓
RE10Kw/ 2D VAE state13.820.5280.597
RE10Kw/ SEG state (ours)14.680.5420.459
DL3DVw/ 2D VAE state13.740.4440.639
DL3DVw/ SEG state (ours)15.080.4610.472

Despite using the same causal pipeline and training strategy, the 2D VAE variant produces lower-quality renderings. We attribute this gap to poorer alignment with the queried camera poses during successive and unordered queries. In addition to rendering quality, we evaluate geometric consistency through camera pose accuracy, which is particularly important for constructing spatially coherent scene representations. We follow the evaluation protocol used in Table 4 of our paper and report the results below:

DatasetSettingsATE ↓RPE-t ↓RPE-r ↓
RE10Kw/ 2D VAE state0.3780.5063.129
RE10Kw/ SEG state (ours)0.1570.2041.687
DL3DVw/ 2D VAE state0.4990.5824.420
DL3DVw/ SEG state (ours)0.1330.1851.322

The 2D VAE variant also performs substantially worse in camera pose accuracy. This indicates that the choice of state representation contributes to viewpoint accuracy beyond the benefits of history reuse and causal modeling. In particular, geometric features provide spatial cues that help align successive predictions with the queried camera poses. These results are now included in Section 4.4 of the revised paper, i.e. Table 9 reports 2D image quality, and Table 10 reports camera pose accuracy.

We have also added Figure 6 in Section 4.4 to compare the rendered views and estimated camera poses of the two state representations. Our SEG preserves the arrangement of scene structures and their alignment with the queried viewpoints more consistently across successive generations.

[1] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. International Conference on Learning Representations (ICLR), 2024.

W2 & Q2. Repeated view queries

Many thanks for highlighting the importance of repeated queries under an evolving history. Specifically, we have added a round-trip evaluation on RE10K and DL3DV with the camera-query order 1 → 2 → 3 → 4 → 3 → 2 → 1. We evaluate only the three return queries and compare each prediction with the ground-truth image at the corresponding camera. Each camera is therefore revisited after additional queries have expanded the history, allowing us to evaluate reconstruction fidelity when returning to previously queried viewpoints.

DatasetMethodPSNR ↑SSIM ↑LPIPS ↓
RE10KGLD15.010.5410.447
RE10KGen3R13.840.4940.536
RE10KGenRec15.130.5390.460
RE10KSEG (ours)16.400.5880.405
DL3DVGLD15.060.4640.459
DL3DVGen3R13.680.3710.556
DL3DVGenRec15.160.4630.479
DL3DVSEG (ours)16.380.4990.448

The table shows that SEG consistently outperforms the other baselines, supporting the effectiveness of its geometry-aware design. Given the successive design of our method, we also compare results from the initial 1 → 2 → 3 pass with those from the return 3 → 2 → 1 pass. We observe an average PSNR drop of only 0.12 dB, indicating consistent performance when revisiting earlier viewpoints.

We have added this round-trip protocol and the ground-truth reconstruction results below to Appendix C (Table 13).

Dear Reviewer VbZh,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 21404

Reviewer_QhVmOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper addresses successive view synthesis, in which camera queries arrive one at a time and in arbitrary spatial order, as in interactive scene exploration. Joint multi-view methods synthesize a predetermined set of target views and do not reuse generated content, while video-based methods are organized around continuous camera trajectories, so neither handles incremental, unordered queries well. SEG addresses this with a Self-Expanding Geometric state that pairs an archive of geometric latents with a KV reference memory. Both are appended after each prediction without altering earlier entries, so generated content accumulates as reusable context. The latents are taken from the earliest level of a frozen Depth Anything 3 encoder, which the authors find to be the most stable when new views are added. For training, each ground-truth target is encoded only from its own image prefix, which keeps information from future views out of earlier targets. A causal geometric diffusion model generates one view at a time by attending to the stored memory, and a geometric adapter maps the generated latents to multi-level features for RGB readout. The work shows that geometric latent diffusion can be extended from fixed-set generation to a causal, incrementally growing state, offering an efficient path toward interactive exploration of a scene from sparse observations.

Critical strengths

1. Strong empirical support for the successive, unordered setting. SEG outperforms reconstruction-based, joint multi-view and video-based baselines on both RE10K and DL3DV, in image quality as well as in camera pose accuracy, while using less time and memory than all generative baselines. Most importantly, the paper directly tests its central claim: when queries are shuffled, video-based methods degrade clearly, while SEG stays almost unchanged (Table 8). This matters to my assessment because order robustness is the main motivation for the method, and it is supported by a targeted experiment rather than only by qualitative examples.

2. A well-motivated design of the retained state. The choice of which features to keep is grounded in an analysis of how stable each DA3 level is when new views are added, and causal-prefix supervision prevents future views from leaking into earlier training targets. The append-only state keeps earlier entries unchanged, which is a sensible property for incremental queries. The ablations show that each of these choices contributes to the final performance. This is consequential because it gives a clear rationale for building the state on geometric latents, rather than presenting the design as a set of arbitrary choices.

Critical weaknesses

1. Limited novelty and no analysis of why a geometric state is needed. The pipeline mainly combines existing components: geometric latent diffusion from GLD, causal attention with KV caching for incremental generation, and PRoPE for camera conditioning. Mirage (Wang et al., 2026), which is not cited, already maintains a persistent latent memory that accumulates generated content and can be queried at arbitrary target poses, and CausNVS already formulates novel view synthesis as autoregressive multi-view diffusion with KV caching. Compared with these, the main difference appears to be using DA3 tokens as the memory instead of 2D image latents. However, the paper does not analyze why this choice is necessary. The stability analysis only compares levels within DA3, and the encoder ablation only replaces DA3 with VGGT, which is also a geometric model. There is no variant that runs the same causal pipeline on 2D image latents. This matters because, without such a comparison, it is unclear whether the gains come from the geometric state itself or simply from reusing generated history, which prior work already does.

2. The benefit of the self-expanding state shrinks over longer sequences. The main motivation of SEG is that accumulating generated content as reusable context helps successive exploration. If this holds, I would expect the advantage over methods that do not reuse history to grow, or at least persist, as more views are queried. In the results, however, SEG clearly outperforms GLD in the main 7-view setting, while in the 15-view setting the gap almost disappears and GenRec even achieves a better perceptual score. I wonder why the benefit of the accumulated state decreases as the sequence gets longer, and how the quality evolves as the number of queries grows. This matters because successive, long-horizon exploration is the central claim of the paper, and the current results support it mainly for short sequences.

[1] Wang, Weijie, et al. "Latent spatial memory for video world models." arXiv preprint arXiv:2606.09828 (2026).

Questions for authors

On the geometric state (Weakness 1)

To show why a geometric state is needed, could you train the same causal pipeline on 2D image latents (for example, VAE latents) instead of DA3 tokens? This would help separate the benefit of the geometric representation from the benefit of reusing generated history. A brief discussion of how SEG differs from Mirage [1] and CausNVS would also help clarify the contribution.

On longer sequences (Weakness 2)

Could you show how quality changes as the number of queries grows, for example with per-query metrics over a 15-view or longer sequence for SEG and GLD? I also wonder why the advantage over GLD becomes smaller with 15 views, for example whether errors in the generated latents accumulate through the memory.

3 comments
AuthorsOpenReview ↗

Official Response to Reviewer QhVm (Part 1/2)

We sincerely thank Reviewer QhVm for the constructive review and insightful questions. We address all the points raised in review as follows:

W1(a) & Q1(a). The contribution of the geometric state

Thanks for suggesting a controlled ablation using 2D image latents within the same causal pipeline. During the rebuttal period, we have conducted experiments comparing a 2D VAE variant with SEG. Specifically, we replace the SEG state with the 2D VAE latents of Stable Diffusion XL [1] while keeping all other settings unchanged. The results are shown in the following table:

DatasetSettingsPSNR ↑SSIM ↑LPIPS ↓
RE10Kw/ 2D VAE state13.820.5280.597
RE10Kw/ SEG state (ours)14.680.5420.459
DL3DVw/ 2D VAE state13.740.4440.639
DL3DVw/ SEG state (ours)15.080.4610.472

Despite using the same causal pipeline and training strategy, the 2D VAE variant produces lower-quality renderings. We attribute this gap to poorer alignment with the queried camera poses during successive and unordered queries. In addition to rendering quality, we evaluate geometric consistency through camera pose accuracy, which is particularly important for constructing spatially coherent scene representations. We follow the evaluation protocol used in Table 4 of our paper and report the results below:

DatasetSettingsATE ↓RPE-t ↓RPE-r ↓
RE10Kw/ 2D VAE state0.3780.5063.129
RE10Kw/ SEG state (ours)0.1570.2041.687
DL3DVw/ 2D VAE state0.4990.5824.420
DL3DVw/ SEG state (ours)0.1330.1851.322

The 2D VAE variant also performs substantially worse in camera pose accuracy. This indicates that the choice of state representation contributes to viewpoint accuracy beyond the benefits of history reuse and causal modeling. In particular, geometric features provide spatial cues that help align successive predictions with the queried camera poses. These results are now included in Section 4.4 of the revised paper, i.e. Table 9 reports 2D image quality, and Table 10 reports camera pose accuracy.

We have also added Figure 6 in Section 4.4 to compare the rendered views and estimated camera poses of the two state representations. Our SEG preserves the arrangement of scene structures and their alignment with the queried viewpoints more consistently across successive generations.

[1] Dustin Podell, Zion English, Kyle Lacey, Andreas Blattmann, Tim Dockhorn, Jonas Müller, Joe Penna, and Robin Rombach. SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis. International Conference on Learning Representations (ICLR), 2024.

AuthorsOpenReview ↗

Official Response to Reviewer QhVm (Part 2/2)

W1(b) & Q1(b). Relationship and comparison with CausNVS and Mirage

Thanks. We have carefully compared SEG with both CausNVS and Mirage and summarize the key differences as follows:

  1. Original motivation. Our key insight is that retained history should support successive queries across geometrically unordered viewpoints. This motivates our joint design of a geometric state and causal-prefix supervision to maintain geometric consistency while supporting incremental queries. CausNVS focuses on streaming queries, whereas our work emphasizes geometric reasoning across successively queried, unordered viewpoints, with carefully designed evaluation settings, 3D metrics and comparisons.

  2. Causal design for a geometric state. While CausNVS uses a causal pipeline with 2D VAE latents which has been explored in video generation approaches, our geometric state introduces additional challenges: maintaining representation stability during state expansion and preventing future-view information leakage during training. We address both through our geometric state design and causal-prefix supervision, and evaluate their effectiveness experimentally.

  3. Implicit geometric state. Unlike Mirage, which relies on explicit depth projection, SEG uses an implicit geometric state to avoid dependence on potentially inaccurate depth estimates and reduce error accumulation.

We have expanded Section 2 (Related Work) and added Appendix E (Discussion) to clarify these distinctions.

During the rebuttal period, we have also evaluated both methods. We evaluated Mirage using its official implementation. As we could not access an official implementation or pretrained checkpoint for CausNVS through its project page or public repositories when conducting these experiments, we reimplemented CausNVS based on the original paper. We compare these baselines with SEG under the evaluation protocol of Table 3 in our paper:

DatasetMethodPSNR ↑SSIM ↑LPIPS ↓Observed PSNR ↑Observed SSIM ↑Unobserved PSNR ↑Unobserved SSIM ↑
RE10KMirage13.410.4940.53213.560.49912.680.434
RE10KCausNVS*14.010.5250.53014.120.52813.440.471
RE10KSEG (ours)14.680.5420.45914.830.54714.010.476
DL3DVMirage13.300.3860.57013.500.39112.710.332
DL3DVCausNVS*13.710.4030.52713.860.40913.300.365
DL3DVSEG (ours)15.080.4610.47215.230.46714.710.424

SEG outperforms both baselines in image quality. We also perform comparisons with both baselines on 3D camera consistency as follows, and it is clear that our SEG still outperforms both baselines due to the usage of our SEG state and the implicit design to avoid depth error accumulation:

DatasetMethodATE ↓RPE-t ↓RPE-r ↓
RE10KMirage0.2140.2792.011
RE10KCausNVS*0.3270.4423.251
RE10KSEG (ours)0.1570.2041.687
DL3DVMirage0.2960.3682.086
DL3DVCausNVS*0.4200.5743.488
DL3DVSEG (ours)0.1330.1851.322

We have incorporated these image-quality comparisons into Table 3 and the corresponding ATE and RPE results into Table 4 of our revised paper.

W2 & Q2. Performance on longer query sequences

Thanks. We apologize for the unclear description of frame intervals. Our model was trained with one source view and seven target views sampled at a 20-frame interval, whereas the longer-sequence evaluation in Table 6 uses a 6-frame interval. This sampling mismatch contributes to the narrower performance margin relative to baselines trained with variable frame intervals. We conduct an additional experiment with denser frame sampling by reducing the training frame interval while keeping the architecture and training sequence length unchanged. We evaluate the resulting model on DL3DV using matched prefixes of 23 and 31 target queries. The results below show that SEG outperforms the compared baselines at both sequence lengths:

Target queriesMethodPSNR ↑SSIM ↑LPIPS ↓
23GLD12.740.4000.605
23GenRec12.230.3800.591
23SEG (ours)13.110.4270.588
31GLD12.590.3970.611
31GenRec12.080.3760.609
31SEG (ours)12.990.4190.598

We have added these results to Table 14 in Appendix C (Additional Quantitative Results) in our revised paper.

Dear Reviewer QhVm,

Thank you for your thoughtful review and constructive suggestions, which have significantly improved the quality and clarity of our paper. We sincerely hope that our rebuttal has fully addressed all of your concerns. As the discussion period is ending soon, if you need any further clarification or additional information, please do not hesitate to contact us. We truly appreciate your time and effort devoted to reviewing our work.

Best regards, Authors of Submission 21404

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of the Author-Reviewer Discussion Phase

Dear PCs, SACs, ACs and all of our reviewers,

We sincerely thank you for your time and constructive feedback, and we appreciate all of you for your comments highlighting the strengths of our work:

  1. A meaningful problem with strong empirical support of successive, unordered view synthesis setting, where we directly test our central claim via comparisons on ordered and unordered view synthesis.

  2. A well-motivated and reasonable design of the retained state, with a clear rationale for building the state on geometric latents.

  3. Newly designed evaluation settings with promising empirical results, where our method achieves a useful combination of quality and efficiency.

During the discussion, we completed all the requested controlled ablations and comparisons, added evaluations of query order, revisits, and longer query sequences, and clarified the contribution of SEG. We summarize how these additions address the original concerns below. We hope this summary assists the AC in assessing the current state of our paper, which we believe has been substantially strengthened in response to the reviewers' constructive feedback:

  1. Isolating and verifying the contribution of the geometric state (Addressing QhVm, VbZh, oM3i). We trained a 2D VAE-state variant with the same backbone, expanding mechanism, and training strategy. The SEG state improves PSNR by 0.86 dB on RE10K and 1.34 dB on DL3DV, while reducing absolute trajectory error from 0.378 to 0.157 and 0.499 to 0.133, respectively. These results establish the contribution of the geometric state representation beyond causal history reuse.

  2. Inclusion of baselines with related preprint methods including Mirage and CausNVS (Addressing QhVm, oM3i). We implement evaluated Mirage from official repo and CausNVS using our reimplementation due to the close-source situation. SEG achieves better image quality and camera-pose accuracy than both evaluated baselines on both datasets. We also applied the ordered vs. unordered protocol to CausNVS. On DL3DV, shuffling queries reduces PSNR by 0.64 dB for CausNVS and 0.09 dB for SEG, demonstrating SEG's strong ability to unordered query.

  3. Evaluating revisited viewpoints under expanded history (Addressing VbZh) and extending evaluation to longer sequences (Addressing QhVm). We added a round-trip protocol to evaluate the return queries against ground truth. SEG achieves the best among all baselines on both datasets. We clarified the training and evaluation sampling intervals and conducted an additional experiment with denser training-view sampling. On matched prefixes of 23 and 31 target queries from DL3DV trajectories, SEG outperforms GLD and GenRec in all three image-quality metrics, demonstrating generation beyond the training query horizon.

  4. Motivation and contribution clarification (Addressing oM3i). Our contribution centers on generating and retaining geometric foundation-model features as a self-expanding scene state for successive, unordered NVS, including following aspects:

  • Problem. Our work targets successive view synthesis with unordered queries, where the views relevant to the current query can lie anywhere in the history. Existing causal NVS can accept such queries, but, as we show below, does not handle them well.
  • Core idea. To address this problem, we use geometric foundation-model features as the retained state, so that the relations between the current query and earlier views are grounded in scene geometry rather than inferred from appearance.
  • Technical contribution. How to make such features work as a causally expanding state (stability under expansion and causal-prefix supervision) is the technical contribution that realizes this idea, not the whole of it.

We again thank the PCs, SACs, ACs and all of our reviewers for their time and effort in evaluating our work.

Sincerely,

Authors of Submission 21404

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.