acceptodds
Under review as a conference paper at ICLR 2027

Vid2Orbit: Adapting Video Diffusion as Multi-View Diffusion for Single-Image Scene Reconstruction

Abstract

Video diffusion models provide strong priors for single-image 3D reconstruction, but their temporal representation is designed for dense video rather than sparse reconstruction views. In the native video VAE, the reset frame follows a higher-fidelity single-frame path, whereas subsequent frames are temporally compressed; under viewpoint motion, these neighboring frames provide limited new angular coverage while carrying larger codec errors. We verify this effect without diffusion sampling: resetting the frozen VAE for every ground-truth view improves round-trip PSNR by 6.25 dB at view spacing. Motivated by this finding, we introduce Vid2Orbit, which repurposes a pretrained video diffusion model as a sparse multi-view generator. Each requested camera uses the VAE's native single-frame path, while the diffusion transformer jointly generates the complete view set. We further introduce camera-aligned feature exchange: intermediate multi-view features are aggregated in a shared 3D field and read back along camera rays. The resulting camera-indexed views provide a practical interface to COLMAP and Gaussian reconstruction.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
computer vision, video diffusion, 3D reconstruction

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_fhwpOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes a method called Vid2Orbit, which adapts a pre-trained video extension model to a sparse multi-view generation model, aiming to improve the quality of 3D scene reconstruction from a single image. The authors point out that traditional video image processing temporal compression mechanisms lose important details when the viewpoint changes significantly. Therefore, this paper proposes a camera alignment feature interaction module based on a shared 3D feature field. This module improves multi-view consistency by resetting the VAE for each viewpoint, assigning independent latent variables to each target camera, jointly generating a complete viewpoint sequence using a video extension transformer, and introducing real geometry monitoring. Experimental results show that this method improves the reconstruction fidelity of the VAE and demonstrates certain advantages in COLMAP point cloud reconstruction and ReSplat visualization results.

Critical strengths
  • This research has a clear motivation and employs a relatively innovative problem-solving approach. This paper focuses on the mismatch between the temporal compression mechanism of video diffusion models and the requirements of sparse multi-view reconstruction. Through VAE reconstruction experiments, we analyze the detail loss caused by temporal compression and propose a per-view resetting strategy. This method is reasonable and effective, providing insights for its application in 3D reconstruction tasks.

  • This method has practical value and achieves good experimental results. This paper combines techniques such as per-view encoding, synchronous multi-view generation, and 3D feature interaction. The generated results can be directly applied to existing reconstruction workflows, such as COLMAP and ReSplat. Experimental results show that the generated viewpoints can provide more effective multi-view matching information, and multiple visualizations also demonstrate good detail preservation and scene consistency, proving the applicability of the proposed method.

Critical weaknesses
  • The actual contribution of the core 3D feature exchange module has not been fully validated. Although the paper lists camera-aligned feature exchange as one of its main innovations, existing experiments have failed to clearly demonstrate its superior performance compared to using RGB adaptation alone. According to Table 2 and Appendix B.5, the original RGB adaptation model achieved 14,197.60 dense points on the Synthetic 20 dataset, while the results dropped to 12,407.10 and 12,355.10 points respectively after introducing the 3D feature exchange branch. While these results are not the result of fully controlled module ablation, they still raise questions about the effectiveness and necessity of this core design, weakening the paper's technical contribution.

  • The quantitative evaluation of 3D reconstruction quality is insufficient, and the fairness of comparative experiments needs improvement. I believe this is the biggest problem I found after reading the paper. This paper mainly uses COLMAP point cloud amount, feature trajectory, and reprojection error to evaluate reconstruction performance, but these metrics do not directly reflect the accuracy of the real geometry. Although this paper uses synthetic data containing real geometric information, there is a lack of direct evaluation of geometric errors or the rendering quality of new perspectives. Furthermore, I believe that this paper has too few quantitative evaluation indicators and the experimental scale is not large enough.

Questions for authors
  • Why are the reconstruction metrics on Synthetic Dataset 20 lower than those on the RGB Adaptive Dataset after introducing Camera Adjustment Feature Improper? Can fully modular resection experiments be performed under uniform training conditions to clarify the model configuration in each experiment and verify the independent contribution of shared 3D feature fields?
  • Since the number of COLMAP point clouds does not directly represent 3D geometric precision, can true improvements in final 3D reconstruction performance be demonstrated by reporting metrics such as chamfer distance, depth error, and new perspective rendering quality using existing synthetic data?
  • Different baseline models maintain the same camera parameters, field of view, and custom scaling. Do these differences affect the COLMAP matching and reconstruction results? How are these factors controlled? Or can more consistent comparative experiments be provided?
  • As shown in Table 1, since K=1 and K=4 differ in the number of latent variables, the improvement in PSNR is likely partly attributed to an increase in representation capacity. Can you further analyze whether the view-by-view reconstruction strategy still holds a significant advantage under the same latent variables or computing resource budget?
3 comments
AuthorsOpenReview ↗

Response to Reviewer fhwp

Response to Reviewer fhwp

Thank you for the questions on attribution, accuracy, cameras and capacity; we respond with matched training and controlled evaluations.

1. Branch contribution and decreased point count (W1/Q1). The drop spans training stages: Table 2 evaluates the RGB-adapted generator, whereas Tables 3/4 compare ordinary/distilled students that both contain the field and share a later initialization. The Synthetic20 stages are:

Model/stageDense pts.Reproj. px↓
RGB, 16k+1k (Table 2)14197.601.233
Ordinary, prepared field +50012407.101.206
Distilled, same start/budget12355.101.189

Appendix B.5 repeats the RGB checkpoint; Table 4 expands Table 3's pair with sparse-point, track and reprojection statistics. Branch adaptation changes observations and COLMAP's retained correspondences, allowing fewer points with lower retained reprojection error; accuracy therefore requires direct surface evaluation. Within the teacher pair, Real20 gains 12.39% dense points and reprojection error falls from 1.425 to 1.357 px, while Synthetic20 dense yield changes by −0.42%.

For your matched-training ablation (Appendix C.3), G0/F/G1 share one RGB initialization, data, 2,000 updates, seed 73, backbone, optimizer and camera/codec settings without a teacher, using the same full-orbit person evaluation:

Matched modelPerson AbsRel↓Chamfer cm↓
G0: RGB only0.286833.98
F: field, RGB loss0.282333.49
G1: field + geometry0.244928.40

G1 lowers mean person AbsRel/Chamfer by 14.6%/16.4% versus RGB-only G0 and Chamfer by 15.2% versus F, isolating a supervision benefit with the F/G1 architecture fixed. Figures 7/9 visualize the learned G1 surface structure.

2. Direct image and reconstruction accuracy (W2/Q2). For your depth, surface-distance and rendering questions, complete distilled Vid2Orbit and baselines use common cameras on additional synthetic data (Appendix C.2). Identical fixed-camera ReSplat uses 16 generated views per method, testing held-out ±9°/±27° views on a shared GT-mutually-visible person region:

ModelRender PSNR dB↑AbsRel↓Chamfer cm↓
Vid2Orbit20.630.101313.03
Lyra 213.630.297228.83
UniWorld-View16.280.306936.88

Relative to the best baseline for each metric, render PSNR rises 4.34 dB and mean AbsRel/Chamfer fall 65.9%/54.8%. These gains show more accurate observed-person geometry and novel-view rendering under the same reconstruction procedure. RGB uses the full GT region; depth requires opacity ≥0.95; symmetric mean Euclidean Chamfer uses no ICP/GT scale fitting. Scores average valid views then scenes equally.

Direct RGB uses 19 non-source targets; person PSNR averages RGB errors inside nonempty GT person masks, while other metrics use full images:

ModelFull PSNR dB↑SSIM↑LPIPS↓Person PSNR dB↑
Vid2Orbit12.9920.40590.567714.099
Lyra 211.4310.37020.623410.108
UniWorld-View11.7260.34670.606910.695

Full-orbit PSNR penalizes both generation errors and plausible unseen-content completions that differ from the single GT. Against the best baseline per metric, Vid2Orbit gains 1.27 dB full-image and 3.40 dB person PSNR and reduces LPIPS by 6.5%, supporting better target-view fidelity in both domains.

3. Cameras, FOV and scale (Q3). To control camera/FOV confounds, tests use common requested intrinsics/poses and lossless 832×480 outputs; baseline scale uses predicted geometry without target-GT fitting. Frames 0,4,...,76 of 81-frame outputs select the requested positions without the repeated endpoint, and direct scoring excludes the source. Fixed cameras prevent pose optimization from absorbing generation errors, complementing the original native-camera tables (Appendix B.7).

4. Equal latent capacity and codec cost (Q4). We fix VAE weights and latent dimensions (Section 4.2; Appendix C.6): K=1 encodes 10/20 requested views and K=4 encodes 37/77 dense GT frames into the same budgets, scoring identical target cameras:

Latent slicesK=1 PSNR dBK=4 PSNR dB
1034.5628.97
2034.7129.33

K=1 gains 5.59/5.38 dB and wins in every evaluated scene within each budget, demonstrating better view fidelity without increasing latent capacity. With no diffusion sampling in this test, the improvement is attributable to the codec, supporting per-view reset for sparse reconstruction. For measured cost at 20 slices:

PathTime s↓Peak allocated GiB↓
Serial K=10.9221.80
Whole-clip K=43.4694.56

K=1 reduces measured codec time by 73.4% and peak allocated memory by 60.4%, combining higher fidelity with lower codec cost in these implementations. Time includes encode/decode and CPU transfer, memory includes resident evaluation networks, and loading/warm-up are excluded; these are codec-stage measurements, not end-to-end generator costs.

Thank you for the explanation. I believe the basic questions have been answered quite clearly, and the article is of good writing quality. Considering the content of the article and the scores from other reviewers, I will maintain my original score and support the article's acceptance. Good luck!

AuthorsOpenReview ↗

Response to Reviewer fhwp's Comment

Thank you for your thoughtful review and support. We are pleased that the response has clarified the main questions, and we appreciate your recognition of the paper’s presentation. Your constructive comments have helped us strengthen both the experimental evidence and the explanation of our method.

Reviewer_ke9SOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This manuscript presents Vid2Orbit, a framework that uses video generation model for scene reconstruction from a single image. It advocates generating relatively sparse multi-view images while processing each requested camera view through the single-frame path of causal VAE without temporal compression, rather than generating more densely sampled frames. It not only can reduce the intrinsic codec errors brought by temporal compression, but also can establish a one-to-one camera–latent mapping thus enabling the proposed camera-aligned readout, where hidden states of different views can exchange information through 3D field.

Critical strengths
  1. The experimental results demonstrate that the images generated by the proposed method achieves better reconstructability than those produced by the baselines.
Critical weaknesses
  1. The proposed per-view VAE processing introduces a substantial distribution shift for the pretrained video generator, which may limit the extent it can benefit from pretraining.
  2. Some important implementation details are not clearly described, including how exactly the teacher model is trained. Moreover, the Reproducibility Statement is misleading, it states that “We will provide configuration files, evaluation scripts, and an anonymous code package with the submission,”. But they cannot be found in either the main paper or the supplementary material.
Questions for authors
  1. Although the comparison in Fig. 2(c) and Table 1 shows the quality advantage for K=1, the two settings (K=1 and K=4) have clearly different costs. How about the result if both of them use the same number of slices?
  2. Why supervise a density field using ground-truth depth and then fuse features through volume rendering, instead of directly using attention in the cross-camera feature exchange? Also, to what extent does the finally learned density field reflect the actual scene geometry?
3 comments
AuthorsOpenReview ↗

Response to Reviewer ke9S

Response to Reviewer ke9S

Thank you for the questions on K=1 adaptation, capacity, geometry and teacher training; we address them with controlled comparisons and the training procedure below.

1. K=1 adaptation and equal capacity (W1/Q1). The VAE, base DiT and camera adapter stay frozen while rank-64 LoRA updates self-/cross-attention Q/K/V/O projections, retaining joint denoising. To test adaptation to the input-distribution change (Appendix C.5), we fix camera conditioning and latent scaling and compare RGB stages under identical K=1 inference (30 steps, seed 0, image guidance 2) at 19 non-source synthetic target cameras:

RGB adaptation stagePSNR dB↑SSIM↑LPIPS↓
Pretrained parent, no project LoRA8.5520.23110.7225
After 16k updates11.7280.34980.6056
After 16k+1k updates13.0290.39790.5600

Adaptation raises mean PSNR by 4.48 dB and reduces LPIPS by 22.5%, with all three metrics improving in every scene at both transitions. These progressive gains show that LoRA can adapt the pretrained model to the per-view representation while retaining its frozen backbone and joint denoising. The final row is Table 2's RGB ancestor after filtered-data refinement, before geometric training.

For your equal-slice question, we use the same frozen VAE and match latent dimensions (Section 4.2; Appendix C.6): K=1 encodes 10/20 views and K=4 encodes 37/77 dense GT frames, with both scored at identical target cameras (PSNR, dB):

SlicesK=1K=4K=1, no resetK=4, no reset
1034.5628.9734.5128.29
2034.7129.3334.7529.08

K=1 gains 5.59/5.38 dB in mean PSNR and wins in every scene within each budget; excluding the shared reset frame leaves gains of 6.22/5.66 dB. Thus, the advantage extends to the temporally grouped views and persists at equal capacity. Because this test precedes diffusion sampling, it isolates detail loss in the codec and supports per-view reset for sparse reconstruction.

2. Why density supervision, and what the field represents (Q2). The field complements joint DiT attention with explicit surface and visibility semantics: K=1 gives each latent grid one camera, allowing lifting/readback along known rays with first-hit weights w_i=T_i(1-exp(-sigma_i Δ_i)), transmittance T_i and an escape event. GT depth supervises visible hits and preceding free space while occluded space remains unknown, so exchange follows predicted ray termination rather than only feature similarity.

To test whether the field reflects geometry, F/G1 share initialization, data, 2,000 updates, seed and architecture without a teacher (Appendices C.3–C.4); only G1 adds geometric supervision. Expected-depth errors use the same eligible smooth person rays on a 30×52 grid, over 20 views per scene, without opacity filtering:

Internal depth MAE, cm↓F: RGB lossG1: + geometry
Home16.639.51
Paris11.753.68

Supervision reduces internal depth MAE by 42.8% on Home and 68.7% on Paris, showing that the field learns more accurate surface locations with the architecture fixed. Downstream mean person Chamfer also falls 15.2%, from 33.49 to 28.40 cm, consistent with improved reconstruction alongside better internal readout. Figures 7/9 show the corresponding GT RGB, GT depth and G1 readout.

3. Teacher training and reproducibility (W2). Teacher preparation (Appendix C.8) starts from the joint RGB/geometry checkpoint after 50 updates and trains only the 122,880-parameter output projection for 100 steps at learning rate 1e-4, using GT first-hit readout in the flow objective. This prepared branch becomes the frozen teacher and initializes both students; it is neither a second DiT nor an EMA:

StageUpdates / examplesTrainable parameters
GT-readout preparation100 / 1Output projection only
Ordinary/distilled students500 / 3Student LoRA + shared branch

Both students use batch 1, seed 73 and LoRA/branch learning rates 1e-6/1e-4, so Tables 3/4 isolate teacher objectives with branch architecture and initialization matched. At zero-indexed DiT block 24, the teacher reads detached current student features with GT first-hit weights while the student uses predicted density; targets are recomputed each step and only student parameters receive gradients.

The distilled loss is L_flow + 0.01 L_geometry + 0.01 L_residual + 0.001 L_KL, with normalized residual MSE and first-hit KL including escape, balancing person interior, boundaries and valid background; the ordinary arm omits the last two terms, and inference removes teacher and GT. This transfers surface-aligned residuals and ray-termination distributions into the student without requiring GT geometry at inference. Appendices C.8–C.9 detail teacher training and evaluation. The full code-and-results archive exceeds the 100 MB upload limit, so we provide implementation descriptions for now and plan to open-source the code.

I appreciate the authors’ efforts on disclosing more implementation details in the revised appendix and correcting the misleading statement in the original Reproducibility Statement. However, these revisions do not change my main assessment of this work, so I am inclined to maintain my initial rating.

AuthorsOpenReview ↗

Response to Reviewer ke9S's Comment

Thank you for your careful review and for acknowledging the clarifications. We appreciate your perspective on the overall contribution, as well as the questions you raised about implementation and reproducibility. Your feedback has been particularly helpful in making the training and evaluation procedures easier for readers to understand.

Reviewer_xWnsOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

Vid2Orbit adapts a pretrained camera-controlled video diffusion model to generate 20 sparse orbital views for single-image, person-centered scene reconstruction. Its main contributions are independently applying the frozen video VAE's first-frame path to each view to avoid temporal compression, and exchanging intermediate multi-view features through a camera-aligned shared 3D field within the diffusion transformer. The shared branch uses geometric supervision and optional ground-truth-guided readout distillation. The generated views serve as inputs to COLMAP or ReSplat for downstream reconstruction.

Critical strengths
  1. A useful diagnosis of information loss in the video codec. The frozen-VAE round-trip experiment separates codec distortion from diffusion-generation errors and demonstrates that temporal grouping degrades sparse orbital views. This is consequential because it identifies an actionable source of detail loss before generation or reconstruction, and motivates a simple way to reuse an existing video model for sparse multi-view synthesis.
  2. Evidence that the generated view sets are useful for downstream reconstruction. On 20 synthetic inputs, the main comparison reports significantly more dense COLMAP points and more than twice as many tracks spanning at least three views as UniWorld-View, together with lower retained reprojection error. Independent ReSplat reconstructions and supplementary videos provide complementary qualitative evidence on synthetic and real inputs.
Critical weaknesses
  1. Insufficient direct validation of generated views and reconstructed geometry. Section 4.3 presents only qualitative generation comparisons on two real inputs. Despite having synthetic multi-view ground truth, the paper does not report quantitative image-quality evaluation of the generated target views, camera-control accuracy, or reconstruction accuracy against GT geometry.
  2. The contribution of the shared 3D field and geometric supervision is not isolated. Both arms of the teacher study already contain the shared field and ordinary geometric loss, so the study cannot establish whether these additions improve upon RGB-only adaptation. Appendix B.5 reports the same 14,197.60 synthetic dense-point result as Table 2 for the RGB-adapted ancestor; the headline reconstruction gain therefore does not establish the benefit of the geometric branch. The only paired test concerns joint teacher distillation, using three fine-tuning examples and a single training/generation seed setting, with gains on Real20 but no aggregate dense-point gain on Synthetic20. Without a matched RGB-only comparison, the necessity and benefit of a central proposed component remain unresolved.
  3. The incremental novelty over closely related work is insufficiently established. FrameCrafter [1] already uses independent per-view encoding and decoding through the video VAE's single-frame path, while ConsistNet [2] already aggregates multi-view features in 3D and reads them back along camera rays to enforce consistency during diffusion. Vid2Orbit differs in its single-image full-orbit setting, integration into a video DiT, and GT-guided readout training, but the methodological advance offered by these differences is not clearly established. A new combination could constitute a valuable contribution; however, the missing component ablations and incomplete teacher specification leave its distinct benefit uncertain. Consequently, the paper does not yet demonstrate a sufficiently clear and validated advance over these precedents to support acceptance.

[1] Wu, Qi, et al. "FrameCrafter: Novel View Synthesis as Video Completion." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026. [2] Yang, Jiayu, et al. "Consistnet: Enforcing 3d consistency for multi-view images diffusion." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.

Questions for authors
  1. Additional evidence request — Does the geometric branch improve upon matched RGB-only adaptation? Can you provide a comparison that holds the starting RGB checkpoint, training data, optimization budget, and evaluation protocol fixed while adding the shared branch and geometric supervision, before introducing teacher distillation? The current paired study begins after shared-branch adaptation, so this comparison would directly determine whether the central geometric addition contributes an independent benefit.
  2. Clarification — What is the specific methodological advance over the closest precedents? Given FrameCrafter's independent per-view VAE processing and ConsistNet's 3D feature aggregation and ray-based readback, which parts of Vid2Orbit introduce a distinct mechanism, and which adapt existing mechanisms to the single-image full-orbit setting? How do the specific differences address a limitation of these precedents? A precise distinction would materially affect my assessment of the contribution's novelty and significance.
  3. Additional evidence request — Do the reconstruction gains correspond to better generated target views? Can you provide quantitative evaluation of generated views against synthetic GT at corresponding target cameras, with the camera/FOV and view-selection conventions documented for each baseline? This would help determine whether the reported COLMAP gains are accompanied by improved target-view fidelity, and how the method comparison is affected by the differing camera protocols.
3 comments

Response to Reviewer xWns

Thank you for the questions on accuracy, attribution and novelty; we respond with direct validation, matched training and method clarification.

1. Direct accuracy and camera validation (W1/Q3). We compare complete distilled Vid2Orbit and baselines on additional synthetic data at common cameras, using lossless 832×480 outputs and 19 non-source targets; means weight valid views then scenes equally. Person PSNR uses RGB errors only inside nonempty GT person masks; SSIM/LPIPS use full images:

ModelFull PSNR dB↑SSIM↑LPIPS↓Person PSNR dB↑
Vid2Orbit12.9920.40590.567714.099
Lyra 211.4310.37020.623410.108
UniWorld-View11.7260.34670.606910.695

Full-orbit PSNR penalizes both generation errors and plausible unseen-content completions that differ from the single GT. Against the best baseline per metric, Vid2Orbit gains 1.27 dB full-image and 3.40 dB person PSNR and reduces LPIPS by 6.5%, supporting better target-view fidelity in both domains.

For direct geometry (Appendix C.2), identical fixed-camera ReSplat uses 16 generated views per method and tests held-out ±9°/±27° views on a common GT-mutually-visible person region:

ModelRender PSNR dB↑AbsRel↓Chamfer cm↓
Vid2Orbit20.630.101313.03
Lyra 213.630.297228.83
UniWorld-View16.280.306936.88

Against each metric's best baseline, render PSNR gains 4.34 dB and mean AbsRel/Chamfer fall 65.9%/54.8%, linking better views to more accurate person geometry under the same reconstruction procedure. RGB uses the full GT region; depth requires opacity ≥0.95; Chamfer is symmetric mean Euclidean distance without ICP/GT scale fitting.

For camera control, free SfM fixes intrinsics and fits extrinsics with one global Sim(3) alignment; both common scenes have ≥16 registered views spanning ≥270° for every method:

ModelRotation °↓Center RMSE/radius %↓
Vid2Orbit1.312.73
Lyra 24.768.20
UniWorld-View8.9712.32

Rotation/center errors fall 72.6%/66.7% versus the stronger baseline, Lyra 2. Since global alignment cannot correct view-dependent deviations, this supports better orbit adherence on the common registered subset.

2. Isolating the branch and interpreting point yield (W2/Q1). Table 2 and Appendix B.5 report the same RGB checkpoint; Tables 3/4 test distillation on three examples at a fixed seed, with field and geometry loss in both arms. Synthetic20 stages are:

Model/stageDense pts.Reproj. px↓
RGB, 16k+1k (Table 2)14197.601.233
Ordinary, prepared field +50012407.101.206
Distilled, same start/budget12355.101.189

Branch adaptation changes generated observations and COLMAP's retained correspondences, allowing fewer points with lower retained reprojection error; point count alone cannot determine accuracy. Within the teacher pair, Synthetic20 dense yield changes by −0.42%, whereas Real20 gains 12.39% and reprojection error falls from 1.425 to 1.357 px.

For the RGB-only control (Appendix C.3), G0/F/G1 share one RGB initialization, training data, 2,000 updates, seed 73, optimizer, backbone and camera/codec settings, with no teacher; only the field and its supervision differ:

Matched modelPerson AbsRel↓Chamfer cm↓
G0: RGB only0.286833.98
F: field, RGB loss0.282333.49
G1: field + geometry0.244928.40

G1 lowers full-orbit mean person AbsRel/Chamfer by 14.6%/16.4% versus RGB-only G0 and Chamfer by 15.2% versus F, isolating a supervision benefit with the F/G1 architecture fixed. Figures 7/9 visualize G1 geometry.

3. Contribution beyond the closest precedents (W3/Q2). Our continuous-video trials exposed a coverage–fidelity trade-off: slow rotation needs long clips and autoregression complicates closure, while fast rotation blurs high-motion regions. Reconstruction needs sharp overlapping views; K=1 gives each latent grid one camera with explicit lifting/readback rays, retaining joint DiT denoising.

FrameCrafter uses per-view coding for unordered posed-view NVS, permutation invariance and multiple targets; we connect it to sparse full-orbit reconstruction and density-based first-hit exchange, with surface/free-space supervision and surface-aligned teacher residuals. This adds visibility semantics to ConsistNet-style 3D aggregation/readback, without GT or a teacher at inference (Section 2).

For the temporal-compression limitation motivating K=1, frozen-VAE PSNR (dB) compares equal latent dimensions (Appendix C.6): K=1 encodes 10/20 views and K=4 encodes 37/77 dense GT frames, scoring identical target cameras:

Latent slicesK=1K=4
1034.5628.97
2034.7129.33

The 5.59/5.38 dB gains isolate codec detail preservation at equal capacity without diffusion sampling, supporting per-view reset for sparse reconstruction.

Thank you for the substantive response. The common-camera target-view evaluation and direct geometry results address my concern that the reported gains were supported only by reconstruction-yield proxies. The matched G0/F/G1 study also directly addresses my request for a controlled RGB-only comparison and provides evidence for the benefit of geometric supervision, although the field alone yields only a small improvement. The contribution remains incremental relative to FrameCrafter and ConsistNet, but the response better specifies the methodological differences and provides stronger evidence for their practical value. I therefore lean toward increasing my rating from 2 to 3 (weak acceptance).

AuthorsOpenReview ↗

Response to Reviewer xWns's Comment

Thank you for your thoughtful reassessment and constructive feedback. We are pleased that the additional evidence has addressed your main concerns and clarified the practical value of our approach. The shared field provides the representation on which geometric supervision operates, and the controlled comparison shows that explicit supervision improves reconstruction accuracy with the field architecture held fixed. We also appreciate your perspective on the contribution relative to prior work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.