- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
OrbitNVS addresses object-centric novel view synthesis from one or a few reference images by adapting Wan2.1-I2V-14B to generate RGB and normal-map sequences along specified camera trajectories. The method combines camera adapters conditioned on Plücker ray embeddings, a parameter-sharing normal branch that interacts with the RGB branch through self-attention, and two-stage training that adds decoder-space pixel supervision after latent-space flow-matching training. The reported results outperform SV3D, SEVA, and EscherNet in PSNR, with the largest advantages in the single-reference setting.
Critical strengths
- Substantial and consistent improvements in single-reference object-centric NVS. OrbitNVS achieves the highest reported PSNR across both GSO and OmniObject3D under all three evaluated camera trajectories. For level orbits, it improves over the strongest PSNR baseline by, and retains an advantage as elevation variation increases. The magnitude and consistency of these gains constitute strong evidence of the method’s effectiveness in its stated setting.
- Useful empirical evidence for the normal branch and pixel-space training. Removing the normal branch reduces PSNR by 0.9–1.4 dB across the six single-reference settings, while removing pixel supervision reduces it by 0.2–1.0 dB. The pixel-loss comparison controls the total number of training steps, making the evidence for this training objective more persuasive.
Critical weaknesses
- Insufficient comparison with related baselines.
- Lack direct comparisons against RGB/normal generation approach including Wonder3D [1], Wonder3D++ [2], Era3D [3]. These methods provide public implementations and weights for multi-view RGB/normal generation and their downstream mesh reconstruction does not preclude comparing the preceding generation stage.
- Methods like MVGenMaster [4] also offer a relevant RGB NVS baseline with variable reference views and camera conditioning
- Image-to-3D system including Hunyuan3D [5] or Trellis [6] could additionally be evaluated by rendering its generated assets at the test viewpoints
- Key methodological details and the pixel-supervision design remain insufficiently specified.
- Cross-branch attention: The exact operation that exchanges information between the RGB and normal branches is not specified.
- Reference masks: It is unclear how frame-level reference masks align with temporally compressed VAE latents, particularly when known and unknown frames share a compressed temporal unit
- Pixel-loss target: Equation 7 compares decoded noisy intermediate states, but the choice of this target over clean-image supervision is not justified.
- Optimization details: The paper does not clearly specify whether the VAE decoder is frozen or how the pixel and latent losses are weighted. I assume VAE decoder should be frozen while training DiT, but the authors are recommended to clearly specify it and justify how is it different from training latent loss alone theoretically.
- The experimental scope limits the broader conclusions about NVS. Quantitative validation is confined to rendered object-centric benchmarks, and the real-photograph demonstrations use background-removed inputs. There is no scene-level evaluation testing substantial depth variation, background structure, and camera translation through an environment. Object-centric evaluation is appropriate for the stated task, but it does not establish the broader applicability of the adaptation strategy or general superiority over methods such as SEVA. Scene-level evidence, or a more consistently restricted framing of the conclusions, is needed to substantiate the broader significance of the work.
[1] Long, Xiaoxiao, et al. "Wonder3d: Single image to 3d using cross-domain diffusion." 2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2024.
[2] Yang, Yuxiao, et al. "Wonder3d++: Cross-domain diffusion for high-fidelity 3D generation from a single image." IEEE Transactions on Pattern Analysis and Machine Intelligence 48.2 (2025): 1674-1688.
[3] Li, Peng, et al. "Era3d: High-resolution multiview diffusion using efficient row-wise attention." Advances in Neural Information Processing Systems 37 (2024): 55975-56000.
[4] Cao, Chenjie, et al. "Mvgenmaster: Scaling multi-view generation from any image via 3d priors enhanced diffusion model." 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2025.
[5] Hunyuan3D, Team, et al. "Hunyuan3d 2.1: From images to high-fidelity 3d assets with production-ready pbr material." arXiv preprint arXiv:2506.15442 (2025).
[6] Xiang, Jianfeng, et al. "Native and compact structured latents for 3d generation." Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition. 2026.
Questions for authors
- Clarification — cross-branch interaction. What exact self-attention operation connects the RGB and normal branches? Please specify how their tokens and Q/K/V are organized and which parameters are shared.
- Clarification — reference conditioning. How are frame-level reference masks represented after temporal VAE compression? A tensor-shape example with one known frame and several unknown frames in the same compressed temporal unit would clarify whether individual reference positions are preserved.
- Clarification — pixel supervision. Does the implementation follow Equation 7 literally, comparing decoded noisy intermediate states? If so, why choose this target over clean-image supervision, is the decoder frozen, and how are the pixel and latent losses weighted?
- Clarification — evaluation protocol. Given the different output resolutions and frame counts, how are target camera poses and image resolutions matched across methods when computing metrics? Are reference views excluded, and is the same foreground-masking policy applied?
- Additional evidence — relevant baselines. Can you provide a compatible comparison with a joint RGB/normal generator, such as Wonder3D++ or Era3D, and a recent camera-conditioned NVS method, such as MVGenMaster? These comparisons would help establish whether the reported advantage extends to the closest alternative approaches.
- Additional evidence — quality versus cost. How does OrbitNVS compare with the evaluated baselines in runtime and peak memory under matched hardware and output settings? Does the normal branch retain a quality advantage over the RGB-only variant at a matched inference-time budget? Future-work suggestion: Scene-level evaluation would be valuable, but is not a prerequisite if the paper consistently restricts its claims to object-centric NVS.