acceptodds
Withdrawn from ICLR 2027

WARPHAMMER: DENSIFYING SCENE WARPS WITH 3D OBJECT PRIORS FOR EXTREME VIEW SYNTHESIS

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Projection-conditioned novel view synthesis (NVS) warps an explicit 3D recon- struction of the input view into the target camera and conditions a generator on the warped rendering. This works well for small viewpoint changes but degrades sharply under large orbital motion: the warp becomes sparse around the orbited ob- ject, where hidden surfaces dominate the new view and mirror-like artifacts emerge, causing the generator to lose both pixel content and the implicit camera cue carried by the warp. We introduce WarpHammer, a training-free framework that resolves this failure mode by augmenting the warped scene with an explicit 3D reconstruc- tion of the object obtained from a native 3D generative prior (e.g., SAM3D). The reconstructed object adds missing foreground surfaces and occludes background points that should no longer be visible, restoring both appearance and camera cues without fine-tuning the base model. The same explicit object representation further unlocks a capability current NVS pipelines do not support: incorporating auxiliary views of the object from sources outside the target scene, for example, a casual snapshot of a car paired with a manufacturer studio shot of the same model. We process the reference and auxiliary images jointly with a pretrained multi-view geometry foundation model, which predicts a unified point cloud that we fuse into the 3D object reconstruction. This yields substantially more faithful geometry than single-image reconstruction, without requiring user-provided camera poses for the auxiliary views. On five benchmarks, WarpHammer produces stable novel views at viewpoint deviations where strong baselines collapse, and is the first scene-level NVS method that can naturally fuse auxiliary, pose-unknown object views from an external source.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
novel view synthesis, video diffusion models, camera control, 3D object priors, object-centric 3D reconstruction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mnK3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes a solution to a known failure mode of pojection-conditioned novel view synthesis. Namely that under large camera motion the warped point cloud becomes sparse and the generator loses appearance and geometric evidence to constrain the new image. The papers proposed fix (Warphammer) is a training free approach built on Gen3C, TrajectoryCrafter and SAM3D. It uses SAM3D to reconstruct the 3d object based on the masked reference point cloud. Textures are not filled in by Sam3d, instead, the diffusion model takes care of them. The paper further introduces an auxiliary mode to further constrain the NVS with a second view. The second view is registered to the first using VGGT and a closed form similarity transform. The paper proposed a new metric (OrbErr), an orbital fidelity metric that combines azimuth/elevation error with DINOv2 feature displacement. Across 5 datasets, Warphammer improves masked-object PSNR, LPIPS and SSIMl. Overall, the problem is real and interesting and I think the auxiliary-view idea is practical. However, I have some concerns about the experiments. The pixel metrics operate in a range where they carry little meaning, the auxiliary-view results are measured at a view that is also given as input, and several reported numbers are hard to reconcile. Combined with limited technical novelty, my current assessment is a weak reject.

Critical strengths
  1. The paper is very well motivated and the well illustrated (Figures). It also follows common intuition that warps become less expressive under large camera orbits and that this would lead to decreased NVS performance makes sense to me. I enjoyed the coverage analysis in Appendix E, which serves as a nice confirmation of this.
  2. The method is training free and modular, which would make this easy to adopt.
  3. Introduing the auxiliary view setting is a genuinely interesting use case. I am not familiar with any other NVS method that allows fusing unposed catalog or studio images of the same object into the representation. Recovering the relative pose through joint VGGT prediction, instead of requiring it as input, is a sensible design.
  4. The choice to separate appearance from geometry is sensible, as it avoids texture and lightning mismatches with the 3D generator.
  5. The OrbErr sanity check is well designed. It convincingly shows that rotation error misses wrong-object failures.
  6. I appreciate the breath of the evaluations and that the limitations were addressed honestly. The paper covers five datasets and seven baselines, including the concurrent FreeOrbit4D, and the limitations section openly discusses segmentation dependence, reconstruction errors and runtime.
Critical weaknesses
  1. I have some concerns on how meaningful the pixel metrics are here. Baselines score only about 7-8dB maked PSNR on average, and sometimes as low as 3-5dB. WarpHammer’s headline gain is +4.22 dB, to 11.67 dB. But Appendix D.2 shos that only shifting an image by 3% leads to PSNR of 14.3 against itself. At these values PSNR mostly measures alignment, scale and placement, not reconstruction quality. I would encourage to instead lead with metrics that are robust to misalignment and to move PSNR and SSIM to a supporting role.
  2. The auxiliary-view results are measured at a view that is also an input (accordint to Appendix C). The main claim of Table 2, that an unposed object image recovers much of the benefit of a ground truth second view, is then largely a claim about reproducing an input. I think evaluation of held-out intermediate frames would be much better to support this claim. Also, if the second view given to GEN3C-MV and SEVA-MV is the same evaluated endpoint, these methods essentially receive the target image together with its exact pose. Yet, they only reach ~14.3 dB, which suggests the evaluation cannot register even a near-copy of the target. It is also counterintuitive that these baselines have higher RotErr with ground-truth poses (193.04 and 176.96) than without them (97.13 and 81.38).
  3. Table 1 is an unweighted mean over datasets of very different size, from Objectron with 9 pairs to SynView-X with 900, hiding important per dataset variation. Table 6 shows on SynView-X, the authors’ own benchmark and the only one with OrbErr, WarpHammer ranks fourth of seven on PSNR. On Objectron, WarpHammer’s RotErr is 11.76 while every baseline is between 261.5 and 313.4. Why such a large gap?
  4. RotErr and TransErr are summed over frames rather than averaged. This is only valid if every method generates the same number of frames, and the backbones and baselines have different native video lengths. It is also unclear how generated frames are matched to the real intermediate frames used as ground truth on CO3D, Objectron and Mip-NeRF 360.
  5. There is an appearance cost that the main paper doesn’t discuss: in Table 6, WarpHammer has the lowest VBench aesthetic quality on CO3D, Mip-NeRF 360 and OmniObject3D and is well below most baselines on SynView-X. However, the aggregated VBench table (Table 7) does not include aesthetic quality at all. Table 7 also shows SEVA ahead on subject consistency and image quality, and Table 1 shows WarpHammer is not best on CLIP-F. I suspect the gray depth-code conditioning contributes to this? I would encourage the authors to report aesthetic quality in Table 7 and discuss the trade-off.
  6. The ablation is limited and leaves the key design choices untested. Table 3 uses 50 SynView-X samples without saying how they were selected. On this subset, the full single-view model beats “w/o 3D prior” by 2 dB PSNR, whereas on the full SynView-X set (Table 6) GEN3C is ahead of WarpHammer on PSNR. More importantly, the main design choices are never ablated quantitatively (occlusion culling alone, the depth code against SAM3D’s own texture, SAM3D against other 3D generators; neutralized against RGB auxiliary conditioning).
  7. I think overall, the technical novelty is moderate. The method combines existing components (SAM3D, VGGT, GEN3C/TrajectoryCrafter) with standard operations: depth-buffer occlusion and a Kabsch alignment. The core idea of completing the geometric proxy before conditioning a video model is shared with EX-4D, CogNVS and the concurrent FreeOrbit4D, and the comparison with FreeOrbit4D is a single qualitative example. The main contributions in my mind are therefore the auxiliary-view setting and the evaluation tools.
  8. The runtime cost is not quantified. Also, the method requires a dominant, rigid, segmentable object that is visible enough for SAM3D to reconstruct. A misplaced mesh can also remove correct background points, and the segmentation sensitivity study covers a single example. The added inference time from SAM3D and VGGT is mentioned in the limitations but never measured.
Questions for authors
  1. Could you report auxiliary-view results at held-out intermediate frames rather than at the endpoint view supplied as input?
  2. For GEN3C-MV and SEVA-MV with ground-truth poses, is the second input view the same image as the evaluated endpoint? If so, why do they reach only about 14.5 dB, and why is their RotErr higher with ground-truth poses than without?
  3. How many frames does each method generate, how are generated frames matched to real ground-truth intermediate frames, and how does this affect the summed RotErr and TransErr?
  4. Can you explain the Objectron RotErr (11.76 vs. 261.5–313.4 for all baselines)?
  5. How were the 50 ablation samples chosen? Could you add ablations for occlusion culling alone, the depth code against SAM3D texture, and alternative 3D generators, ideally on the full SynView-X set?
  6. Could you add aesthetic quality to Table 7 and comment on the drop relative to the baselines?
  7. Is there a quantitative evaluation with genuinely out-of-scene or appearance-mismatched auxiliary images? How should the neutralization flag be set in practice?
  8. What is the added runtime of the SAM3D and VGGT steps relative to the base backbones?
Reviewer_poJzOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper introduces WarpHammer, a training-free framework for projection-conditioned novel-view synthesis under large camera motions. It augments the sparse scene warp with an object mesh reconstructed by SAM3D, providing geometry for unseen surfaces and suppressing incorrectly visible background points. The method also supports an unposed auxiliary object image: VGGT aligns its geometry with the reference scene, while appearance-neutral depth conditioning avoids transferring mismatched texture or lighting. The paper introduces SynView-X and OrbErr and evaluates the method on five datasets.

Critical strengths

(1) Combining a 3D generative prior with point-cloud-based scene projection is a practical and promising idea.

(2) The method is training-free, modular, and tested with two NVS backbones.

(3) Unposed auxiliary images and geometry-appearance separation are useful practical extensions.

Critical weaknesses

(1) The method strongly depends on SAM3D producing accurate, well-aligned geometry. Incorrect reconstruction (especially for plants, thin structures, or geometrically complex objects) can introduce false surfaces and incorrectly remove valid scene points through mesh-based occlusion. Robustness to such reconstruction errors is not studied sufficiently.

(2) The technical novelty is limited. The method primarily integrates existing components for segmentation, 3D reconstruction, multi-view geometry estimation, point-cloud rendering, and diffusion, rather than introducing a fundamentally new model or learning algorithm.

(3) Multi-object scenes require separate segmentation, reconstruction, alignment, and compositing for each object. The paper provides only a qualitative multi-object example and does not adequately analyze scalability, inconsistent object geometry or scale, and mutual occlusion.

Questions for authors

(1) How sensitive is WarpHammer to errors in the geometry, pose, and scale of the SAM3D reconstruction? In particular, could the authors provide controlled perturbation experiments that vary mesh translation, rotation, scale, and shape quality, and report their effects on image fidelity, OrbErr, and foreground-background occlusion? It would also be helpful to report how frequently inaccurate SAM3D reconstructions cause failures and whether the system can detect or reduce the influence of an unreliable mesh.

(2) Can the authors report the end-to-end runtime and peak memory usage of WarpHammer-GEN3C and WarpHammer-TC, compared with their unmodified GEN3C and TrajectoryCrafter baselines?

Reviewer_m4zQOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

WarpHammer improves extreme-view synthesis by combining SAM3D object geometry with a scene point-cloud cache to condition pretrained video diffusion models without additional training. It optionally incorporates an unposed auxiliary image through VGGT alignment. Experiments across five datasets and two backbones demonstrate substantial image-quality improvements, although the contributions of auxiliary geometry and depth encoding remain insufficiently isolated.

Critical strengths
  1. Readable presentation. The paper is generally well organized, with readable, largely grammatical prose. The motivation and overall pipeline are easy to follow.
  2. In Table 1, GEN3C’s macro-average PSNR improves from 7.45 to 11.67, and LPIPS decreases from 0.767 to 0.444. TrajectoryCrafter also improves from 7.58 to 10.79 PSNR,which show some good evidence.
Critical weaknesses
  1. Important implementation and comparison details are unclear:Table 2 leaves the pose-estimation and input-adaptation procedures for the “no GT pose” baselines unclear. These omissions limit reproducibility and confidence in the auxiliary-view comparison.
  2. The auxiliary mechanisms lack sufficiently controlled validation:Figure 5 uses luminance grayscale rather than depth encoding (Appendix B.3), leaving the specific advantage of depth encoding unverified.
Questions for authors
  1. How is the fused point set in Equation (6) represented and supplied to SAM3D? What auxiliary inputs and processing paths remain in the w/o VGGT ablation?
  2. Do Table 2’s “no GT pose” baselines use comparable auxiliary information and registration procedures? Please specify their pose estimation, input preprocessing, and model versions.
  3. Can you separately evaluate auxiliary updates to object geometry, direct auxiliary-cache additions, and grayscale versus depth conditioning under matched settings?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.