acceptodds
Under review as a conference paper at ICLR 2027

Feedforward Novel View Synthesis using Latent Representations for In-the-Wild Images

Abstract

Recent feed-forward models have shown promise for novel view synthesis (NVS) from unconstrained photo collections. However, existing training setups offer limited coverage of real-world appearance variation, transient occlusion, or diverse source-view configurations, while differences in evaluation protocols complicate comparisons. We introduce the Wild-NVS dataset and benchmark, curated from internet photo collections, with a sampling strategy that constructs source–target view sets using bidirectional covisibility, joint target coverage, and viewpoint diversity. The benchmark provides scene-level splits and fixed evaluation protocols across multiple source-view counts. We also propose Wild-NVS, a feed-forward model that synthesizes target views from latent representations. It combines bottlenecked appearance conditioning with a learned transient prediction head that guides source-feature suppression before decoding. Experiments on Wild-NVS and PhotoTourism demonstrate state-of-the-art performance among deterministic feed-forward methods for in-the-wild NVS, with improved appearance modeling and reduced transient artifacts. Our models and dataset will be made public.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Novel View Synthesis, Feedforward models, in-the-wild images, unconstrained images

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_8XJkOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper addresses feed-forward novel view synthesis (NVS) from unconstrained internet photo collections. While existing datasets or training settings consider little of the effects of real-world appearance variation, transient occlusion, or diverse source-view configurations, this work builds the Wild-NVS dataset and benchmark to serve as a more comprehensive evaluation protocol for real-world applicability. The dataset is built on MegaDepth-X reconstructions. Each target is paired with an ordered list of up to 64 source views, chosen by a bidirectional depth-warping covisibility test, joint target coverage, and viewpoint spread.

Then, this work introduces a Wild-NVS model that extends the latent NVS model LagerNVS with an appearance encoder and a transient head to handle real-world appearance variation and transient occlusion, respectively. Extensive experiments and ablations have demonstrated the effectiveness of the proposed method over baselines under real-world conditions.

Critical strengths
  1. It builds a reproducible benchmark that fills a real gap. Prior feed-forward in-the-wild methods are evaluated on unnamed scenes and unlisted images, so their numbers cannot be reproduced or compared. Wild-NVS fixes this with landmark-level splits (the PhotoTourism landmarks held out of training), source sets checked against depth with a bidirectional covisibility test, and full curation details. The authors also promise to release the dataset, which could be a lasting, reusable contribution, independent of the specific model.

  2. Experimental results demonstrate large, consistent empirical gains, with a clean data ablation. LagerNVS-FT (the same architecture, only fine-tuned on the new data) outperforms the original LagerNVS at every source count. On Wild-NVS, it also outperforms both GS baselines on every metric. Qualitative results also demonstrate that the appearance transfer works well. Conditioning images from other scenes change tone, brightness, and sky while keeping the structure (Fig. 1), which suggests the 8-dimensional bottleneck works as intended. Even if part of the margin comes from the protocol (see Weaknesses), gaps of this size are unlikely to disappear.

Critical weaknesses

Major

  1. The quantitative evaluation does not specify which appearance-conditioning image is used. The paper states that the target image supplies appearance during training; Figure 1 includes a target-conditioned result, Figure 3 discusses matching the target appearance, and Figure 4 uses target-image appearance codes. If the ground-truth target is also used for conditioning in Tables 1–2, Wild-NVS receives target color information unavailable to LagerNVS-FT.

  2. The GS baselines appear not to be retrained on Wild-NVS, run at different resolutions, and GenWildSplat cannot use the provided cameras. "SOTA" mixes data, pose and appearance-access effects.

Minor

  1. Another concern is the little quantitative support for the transient module, the main new modeling component. It brings little PSNR gains and SSIM can even degrade (Table 6).
  2. As for the VLM judgment, the full model is always "A", the number of pairs is unstated. There is no human check, and baselines are not judged. Providing some details can be more informative, e.g., how many pairs were judged, and was the A/B order randomized? How well do the VLM labels agree with human labels on a subset? What do the same judgments give for the baselines?
  3. The standalone training protocol for LagerNVS-FT is not explicitly specified. Tables 3 and 6 appear to imply 100K steps, but the schedule and source-view sampling remain unclear.
Questions for authors
  1. Section 6.1 appears to use predicted target poses for GenWildSplat. Does this apply only to GenWildSplat, and how are its predicted poses aligned with the ground-truth evaluation cameras?
  2. How many steps was LagerNVS-FT trained for, and with what schedule and source sampling? Is it compared at the same step count as Wild-NVS?
6 comments
AuthorsOpenReview ↗

Response to reviewer 8XJk

Major W1: Concerns on target color information available to Wild-NVS but not to LagerNVS-FT.

In order to evaluate the method quantitatively in Tables 1 and 2 of the main paper, we provide the baselines, and Wild-NVS the target image to extract the appearance embedding. However, LagerNVS-FT cannot be provided this information due to its architectural limitations. We have included this in Section 6.1 of the paper covering the evaluation protocol.

To isolate the target color information available to Wild-NVS, We train a variant, Wild-NVS†, whose appearance encoder sees only a randomly chosen left or right half of the target, mirrored to full width. The loss is applied only to the other half, and training runs for 100K steps. At test time, the encoder receives the left half and metrics are computed on the right half.

For comparison of LagerNVS-FT, we map the rendered image to the target image using an affine color correction following RawNeRF [1]. For each test image we fit a per-channel scale and shift parameter (6 parameters) using least squares, mapping the left half of the render to the global color and exposure in the left half of the target image. We apply it to the rendered image and compute the metrics on the right half of the image. This establishes an oracle baseline (LagerNVS-FT$^\mathcal{O})usesthetargetimagedirectlytofitthecolortransform,unlikeabottleneckedconvolutionalappearanceencoderinourmethod.Wereportcomparisonstoourmodeltrainedwiththeadaptedappearanceencoder(Wild−NVS) uses the target image directly to fit the color transform, unlike a bottlenecked convolutional appearance encoder in our method. We report comparisons to our model trained with the adapted appearance encoder (Wild-NVS^\dagger$), and also after applying the affine color correction to our model output (Wild-NVS$^{{\dagger}\mathcal{O}}$) in Table R1 below.

Table R1: Appearance input and available color information (100K steps, averaged over 4–12 views).

MethodPSNR ↑SSIM ↑LPIPS ↓
LagerNVS-FT14.990.5030.488
LagerNVS-FT$^\mathcal{O}$16.140.5030.516
Wild-NVS$^\dagger$17.030.5220.450
Wild-NVS$^{{\dagger}\mathcal{O}}$17.030.5230.455

Wild-NVS† outperforms the oracle LagerNVS-FT$^\mathcal{O}$ on all three metrics (17.03 vs 16.14 dB PSNR). Because the oracle’s global tone and exposure are already fitted to the target, this 0.89 dB gap cannot be explained by global color matching. It reflects improvements beyond a per-channel affine correction, such as local illumination and transient handling. Applying the same affine correction to Wild-NVS$^\dagger$ brings no further gain, which indicates that our appearance encoder already recovers the global tone and exposure from half of the target image.

Major W2: Differences in data, pose and appearance knowledge across methods.

We acknowledge the differences amongst the baselines in the architecture. We have revised our claims of state-of-the-art accordingly. We separate the three factors as follows.

  • Data. Fine-tuning LagerNVS on our dataset (LagerNVS-FT) improves PSNR by 1.7 dB with the architecture unchanged.
  • Appearance. Restricted to half of the target as its appearance input, Wild-NVS$^\dagger$ still outperforms both LagerNVS-FT and the color-corrected oracle LagerNVS-FT$^\mathcal{O}$ (Table R1 above).
  • Pose. Table R2 evaluates Wild-NVS without source cameras.

Table R2: Effect of source-camera information (averaged over 4–12 views).

MethodPSNR ↑SSIM ↑LPIPS ↓
GenWildSplat13.350.3940.619
WildSplatter13.370.3850.599
Wild-NVS pose-free15.360.4720.601
Wild-NVS17.750.5350.438

Without source cameras, Wild-NVS still outperforms both baselines by about 2.0 dB PSNR. Its LPIPS is better than GenWildSplat’s and on par with WildSplatter’s. Since Wild-NVS can leverage the pose information, it improves further by 2.39 dB PSNR and lowers the LPIPS score by 0.163. This pose-free rendering capability is achieved due to the dropout of source camera tokens at a fixed probability during training. However, we have not tuned this dropout hyperparameter for a pose-free vs posed rendering performance trade-off.

[1] Mildenhall, Ben, et al. "Nerf in the dark: High dynamic range view synthesis from noisy raw images." 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2022.

AuthorsOpenReview ↗

Response to reviewer 8XJk contd.

Minor W1: Quantitative evidence for the transient module.

We conducted a blind human study comparing the transient-aware Wild-NVS with the appearance-encoder-only Wild-NVS-A. We sampled 60 test targets containing visible transients in the source views from the Wild-NVS test set. Evaluators saw the ground-truth target and four renderings, in randomized order and without method names: GenWildSplat, WildSplatter, Wild-NVS-A (appearance encoder only), and Wild-NVS. Both of our models are trained to 100K steps. Evaluators were asked to choose the best render without the transient objects. Eleven evaluators rated as many samples as they chose through an online form, for 207 judgments in total. We report the share of judgments selecting each method and the 95% CI from a bootstrap over test cases with 4,000 resamples.

The human evaluation shows a significant preference for renderings from the transient-aware model, highlighting the improvement in rendering quality from transient feature suppression over the appearance-encoder only model.

Table R3: Human preference for transient-free renderings

MethodChosen by evaluators95% CI
Wild-NVS91.3%86.9--95.3
Wild-NVS-A5.3%2.4--8.3
GenWildSplat1.4%0.0--3.2
WildSplatter1.9%0.5--3.9

Minor W2: Details about VLM judge and agreement to human labels.

GPT-6 Astra compared the appearance-encoder-only model (Wild-NVS-A) with the full Wild-NVS on 1,626 rendering pairs. The order of the two renderings was not randomized, which could introduce an order bias. We have added these details to the supplementary material.

To validate the VLM labels, we ran a human evaluation on 120 of the 1,626 pairs from Table 8 of the appendix. We sampled these to over-represent pairs flagged by GPT-6 and weighted the rates back to the full set. Raters saw the two renderings side by side, in random order and without method names, and answered the same question as GPT-6. Eight raters gave 244 judgments in total.

Table R4: Agreement between GPT-6 and human labels.

Wild-NVS-AWild-NVS
GPT-6–human agreement (artifact / no artifact)66%81%
Human–human agreement (artifact / no artifact)73%81%
Artifact rate estimated from people (all 1626 pairs)18%7.7%
Artifact rate from GPT-610.2%1.6%

GPT-6 agrees with people nearly as often as people agree with each other. Both GPT-6 and people find artifacts more often in the appearance-encoder only model. The reweighted rate based on human evaluation is 18.0% flagging the appearance-encoder only model vs 7.7% for the full Wild-NVS model (a difference of 10.3 points, 95% CI 4.1 to 16.9). The evaluation using GPT-6 was 10.2% flagging the appearance-encoder only model vs 1.6% for the full Wild-NVS model (8.6 points difference). The preference for the transient-aware model therefore holds under both judges, but people mark artifacts more often than GPT-6 for both models. We therefore report the human-estimated rates in the main paper.

The baselines were not included in the VLM comparison because their renderings are substantially lower in quality (about 4 dB PSNR below Wild-NVS), and the study targets transient mitigation within our model. All four methods are compared in the human preference study (Minor W1, Table R3).

Minor W3/Q2: Training protocol for LagerNVS-FT not stated.

The original paper reported LagerNVS-FT metrics at 100K steps, trained under the same schedule and source-view sampling settings due to resource limitations. We have updated the numbers of the LagerNVS-FT model trained with the same compute budget as Wild-NVS (125K steps) under the same training schedules, learning rates, and source-view sampling in the main paper. We also provide them below. On the Wild-NVS dataset the PSNR remains unchanged (14.99 dB) with a slightly worse SSIM and LPIPS score. On PhotoTourism, it lowers the PSNR by 0.37 dB, which we attribute to the smaller evaluation set on 3 scenes. Our method Wild-NVS outperforms LagerNVS-FT at both checkpoints on both datasets leaving the conclusions from the main paper unchanged.

Table R5: LagerNVS-FT at 100K and 125K steps in the Wild-NVS dataset.

MethodPSNR ↑SSIM ↑LPIPS ↓
LagerNVS-FT (100K)14.990.5030.488
LagerNVS-FT (125K)14.990.5010.492
Wild-NVS17.750.5350.438

Table R6: LagerNVS-FT at 100K and 125K steps in the PhotoTourism dataset.

MethodPSNR ↑SSIM ↑LPIPS ↓
LagerNVS-FT (100K)16.180.5990.369
LagerNVS-FT (125K)15.810.5900.376
Wild-NVS19.980.6250.328
AuthorsOpenReview ↗

Response to reviewer 8XJk contd.

Q1: Pose information for baseline methods.

Among the baselines this applies only to GenWildSplat. Each method is evaluated with its own published inference protocol:

  • GenWildSplat cannot take cameras as input and its evaluation predicts the target camera with VGGT, jointly with the source views, and renders the target from that prediction. The Gaussians are built in the same predicted frame, so no alignment to the ground-truth cameras is needed. Metrics are computed on the rendered image and the ground-truth image directly.
  • WildSplatter uses Depth-Anything-3 to predict the source cameras and geometry, so the Gaussians live in that predicted frame, with source view 0 at the origin. The ground-truth COLMAP target pose is re-expressed relative to ground-truth source view 0. Its translation is then scaled by the mean ratio of predicted to ground-truth distances between source-camera centers. The target is rendered from this rescaled ground-truth pose.
  • Wild-NVS is given ground-truth COLMAP source and target cameras.

To remove the asymmetry, we report evaluations for Wild-NVS with no source cameras at all, in two settings that match the baselines' camera information in the Table below.

Table R7: Evaluation without source cameras (averaged over 4–12 views).

MethodTarget cameraSource camerasPSNR ↑SSIM ↑LPIPS ↓
GenWildSplatVGGT-predictednone13.350.3940.619
Wild-NVSVGGT-predictednone15.280.4700.604
WildSplatterCOLMAPnone13.370.3850.599
Wild-NVSCOLMAPnone15.360.4720.601

In the VGGT-predicted setting, the target pose is obtained exactly as in GenWildSplat. With the same camera information, Wild-NVS outperforms both baselines by about 1.9–2.0 dB PSNR. Its LPIPS is better than GenWildSplat’s and on par with WildSplatter’s.

We also applied WildSplatter's alignment to GenWildSplat, rendering from the rescaled ground-truth target pose and intrinsics instead of its prediction. This lowers its PSNR by 0.9–1.4 dB. The published predicted-pose default protocol is thus the more favorable setting for GenWildSplat.

Reviewer_kVuxOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces Wild-NVS, both a large-scale benchmark and a feed-forward model for novel-view synthesis from unconstrained internet photographs. The benchmark is constructed from underlying MegaDepth-X reconstructions. This is a useful attempt to establish a reproducible benchmark for feed-forward NVS under real illumination variation, camera-processing differences, and transient occlusions.

Architecturally, Wild-NVS is an extension of LagerNVS rather than a fundamentally new NVS backbone. It adds two main components: an appearance encoder that compresses a conditioning image to an eight-dimensional code and injects the expanded embedding into source tokens, and a transient-prediction head that estimates transient probabilities from multi-view latent features. Tokens associated with likely transient regions are interpolated with a learned transient embedding before decoding. The model is trained progressively, first adapting the LagerNVS components with appearance conditioning, then learning transient prediction, and finally activating transient suppression and masked photometric supervision.

Critical strengths
  1. The paper studies a relevant and insufficiently studied problem setting. The paper targets feed-forward NVS from real internet photo collections, where illumination, exposure, weather, transient objects, and camera processing vary substantially. This setting is more realistic and difficult than the photometrically consistent datasets frequently used for generalizable NVS.

  2. The dataset contribution is arguably the strongest part of the work. The construction accounts for bidirectional covisibility, complementary target coverage, camera and viewing-direction diversity, near-duplicate removal, scene-level, rather than image-level, data splitting, and fixed targets with nested source sets across different view counts.

  3. Suppressing transients in latent source tokens is well matched to a latent NVS architecture. Compared with masking source pixels outright, replacing uncertain tokens with a learned embedding may allow the decoder to combine static evidence from other views while avoiding obvious missing-data artifacts. The use of multi-view-aggregated features may also enable the transient head to identify objects beyond the semantic categories used to generate its supervision.

Critical weaknesses
  1. Limited methodological novelty beyond LagerNVS. The proposed model is explicitly an extension of LagerNVS. Both high-level ideas, compact appearance conditioning and transient prediction/suppression, are well established in prior in-the-wild NVS literature, even if their implementation at the latent-token level is new. Therefore, the model contribution appears incremental. The paper’s strongest novelty is likely the benchmark and associated data-curation pipeline, rather than the architecture itself.

  2. Most of the improvement comes from appearance conditioning, not transient suppression. In the component ablation, adding appearance conditioning improves average PSNR from 14.98 to 17.48 dB, whereas adding transient suppression changes it only from 17.48 to 17.49 dB. LPIPS improves modestly, while SSIM decreases. Thus, the evidence supporting transient suppression is considerably weaker than the headline results may suggest.

Questions for authors
  1. What image is used for appearance conditioning at test time? Is it the held-out target image, one of the source images, or a separately selected reference? Please specify the protocol for every benchmark and baseline, and report performance when no target-associated appearance reference is available.

  2. How accurate are the transient labels and predictions? Please report transient-segmentation precision, recall, or IoU on a manually annotated subset, including results for persistent movable objects and transient categories absent from the initial COCO candidate list.

  3. Can the authors disentangle source-feature suppression from target-loss masking? Please provide a factorial ablation of appearance conditioning, source-token suppression, and target-side photometric masking. This would clarify which component is responsible for any reduction in transient artifacts.

8 comments
AuthorsOpenReview ↗

Response to reviewer kVux

W1: Limited methodological novelty.

We agree that the proposed method is an adaptation of LagerNVS towards in-the-wild images as stated in the main paper. Prior feedforward in-the-wild methods either do not handle transients or exclude them with masks from an external segmentation network that the model itself never learns from. Wild-NVS instead learns to predict transients from multi-view features and suppresses them at the token level. We ablate the effect of our learned transient embedding compared to a naive transient suppression method by zeroing the transient features in Table R1, R2. We observe that the naive transient suppression slightly degrades in performance with the drop increasing in the PhotoTourism dataset. Our learned transient embedding allows the model to combine information from static regions while discounting the uncertain transient features, rather than discarding them entirely.

Table R1: Naive transient suppression on Wild-NVS dataset averaged across 4-12 views.

MethodPSNR ↑SSIM ↑LPIPS ↓
Wild-NVS17.750.5350.438
Transients zeroed17.710.5340.442

Table R2: Naive transient suppression on PhotoTourism dataset.

MethodPSNR ↑SSIM ↑LPIPS ↓
Wild-NVS19.980.6250.328
Transients zeroed19.840.6190.331

W2: Marginal improvements from transient suppression.

In order to quantify the improvement from the transient aware model we conducted a blind human study. We sampled 60 test targets containing visible transients in the source views from the Wild-NVS test set. Evaluators saw the ground-truth target and four renderings, in randomized order and without method names; GenWildSplat, WildSplatter, Wild-NVS-A (appearance-encoder-only), and Wild-NVS. Both of our models are trained to 100K steps. Evaluators were asked to choose the best render without the transient objects. Eleven evaluators rated as many samples as they chose through an online form, for 207 judgments in total. We report the share of judgments selecting each method and the 95% CI from a bootstrap over test cases with 4,000 resamples.

Table R3: Human preference for transient-free renderings.

MethodChosen by evaluators95% CI
Wild-NVS91.3%86.9--95.3
Wild-NVS-A5.3%2.4--8.3
GenWildSplat1.4%0.0--3.2
WildSplatter1.9%0.5--3.9

The human evaluation shows a preference for renderings from the transient-aware model by a wide-margin, highlighting the improvement in rendering quality from transient feature suppression over the appearance-encoder only model.

AuthorsOpenReview ↗

Response to reviewer kVux contd.

Q1: Clarification on evaluation protocol.

To evaluate the methods quantitatively in Tables 1 and 2 of the main paper, both baselines (GenWildSplat, WildSplatter), and Wild-NVS use the held-out target image to extract the appearance conditioning embedding. LagerNVS and LagerNVS-FT cannot be provided this information since the architecture does not facilitate this. We have included this in Section 6.1 of the paper covering evaluation protocols.

Both baselines require an appearance conditioning image, making the evaluation without an appearance conditioning image out of their training regime. However, during the training of Wild-NVS, we randomly set the appearance conditioning embedding to zero (p=0.3), which enables the model to render the target view even without an appearance conditioning image. In Table R4 we report the performance of the methods evaluated with the first source view used as appearance conditioning for a fair comparison with baselines, and also report our method without an appearance conditioning image (Wild-NVS-zero). Wild-NVS outperforms both baselines under the same reference. Conditioning on a mismatched source view scores below using no reference, because the model then renders the source view’s illumination rather than an average one. Without any reference, Wild-NVS-zero improves further, to 14.86 dB averaged over 4-8 views, about 3.1 dB above the best baseline.

Table R4: Appearance conditioning from the first source view on Wild-NVS. Wild-NVS-zero uses no appearance image.

Method4 Views6 Views8 Views
PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓PSNR ↑SSIM ↑LPIPS ↓
GenWildSplat11.360.3530.65311.760.3630.64612.070.3700.642
WildSplatter11.480.3410.64611.750.3550.64111.670.3630.638
Wild-NVS13.230.4640.52513.370.4710.50913.470.4750.498
Wild-NVS-zero14.640.4810.50714.890.4900.48915.050.4960.477

To limit the target color information available to Wild-NVS, we train a variant, Wild-NVS$^\dagger$, whose appearance encoder sees only a randomly chosen left or right half of the target, mirrored to full width. The model is trained for 100K steps supervised only on the held-out half of the image. During evaluation of this model Wild-NVS$^\dagger$ we input the left half of the target view to the appearance encoder and compute the metrics on the right half as in NeRF-W [1]. Table R5 reports the performance of Wild-NVS$^\dagger$, showing that this variant still outperforms the baselines (at only 100K steps). We note that restricting the appearance input to half of the target lowers PSNR by 0.46 dB, while still outperforming the best baseline by 3.66 dB.

Table R5: Half-target appearance conditioning on Wild-NVS (averaged over 4–12 views). Both Wild-NVS rows are at 100K steps.

MethodPSNR ↑SSIM ↑LPIPS ↓
GenWildSplat13.350.3940.619
WildSplatter13.370.3850.599
Wild-NVS$^\dagger$17.030.5220.450
Wild-NVS17.490.5180.450

[1] Martin-Brualla, Ricardo, et al. "Nerf in the wild: Neural radiance fields for unconstrained photo collections." 2021 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2021.

AuthorsOpenReview ↗

Response to reviewer kVux contd.

Q2: How accurate are the transient labels and predictions?

We manually annotated 100 test source views with SAM-assisted masks. The annotator saw the same 8 source views as the model, so persistence could be judged across views. Each object was tagged as a transient from the COCO candidate list, a transient outside it, or a persistent movable object. Table R6 reports the results with 95% bootstrap confidence intervals over images for the labels, and our transient head.

Table R6: Evaluation of transient Label quality and transient head prediction on 100 manually labeled images.

MethodRegionPrecisionRecallIoUAP
LabelsAll transients0.83 [0.77, 0.88]0.47 [0.34, 0.63]0.43 [0.32, 0.57]0.41 [0.30, 0.55]
Head (ours)All transients0.56 [0.48, 0.62]0.80 [0.64, 0.90]0.49 [0.40, 0.56]0.68 [0.57, 0.77]
LabelsOutside COCO list0.0040.0010.0010.007
Head (ours)Outside COCO list0.19 [0.04, 0.32]0.49 [0.15, 0.78]0.16 [0.03, 0.28]0.13 [0.03, 0.28]

The learned transient head recovers substantially more transient area than the labels it was trained on, improving the recall from 0.47 to 0.80, with higher AP and IoU. Because the head predicts from multi-view features, we believe it learns transience largely as inconsistency across views rather than from object semantics. As a side effect, static regions visible in only one source view also receive slightly higher transient scores. These false positives, together with the head’s lower-resolution soft map, which extends past object boundaries, account for its lower precision compared with the sharp instance masks of the labels. Its threshold-free AP is nevertheless clearly higher, showing that it separates transient from static pixels better overall. Moreover, suppression is soft, so a static region flagged with moderate probability is only partially replaced by the transient embedding rather than discarded.

Transients outside the COCO candidate list are absent from the labels by construction. The head recovers 49% of their area, in objects such as tents, market stalls, banners and temporary structures. Whether such an object is transient often cannot be decided from a single image, because the same category can be permanent in another scene. A closed-vocabulary, single-image detector therefore cannot reliably capture them, while the head can use the other views.

Persistent movable objects appear in only 10 of the 100 annotated images, covering about 4% of each such image, which limits this estimate. On these pixels, the labels and the transient head flag 2.1% and 3.5% as transient, respectively.

Q3: Can the authors disentangle source feature suppression from target loss masking?

We ablate the effect of transient feature suppression with a naive suppression method by zeroing the transient tokens in Table R1 and R2 above. This demonstrates the effect of a learned transient embedding over a naive zeroing of transient features.

Given the compute constraints for training we are unable to run a factorial ablation of source-feature suppression and target-side loss masking, which requires two additional full training runs that did not fit within the rebuttal period.

Reviewer_KZLJOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a dataset for training and evaluating sparse-view NVS under in-the-wild settings, and a method that achieves SOTA performance on this dataset. They build the dataset from the large-scale MegaDepth-X dataset using a co-visibility check, resulting in 1.8K datasets and 171K images. They also propose a feedforward method for NVS under this in-the-wild setting, which is based on lagerNVS with additional appearance encoding and transient modeling. The method outperforms previous methods on both the PhotoTourism datasets and the Wild-NVS dataset.

Critical strengths
  • The proposed training dataset and benchmark are interesting and could lead to more development ot the field. The covisibility check is practical for establishing valid source and target views, and a unified evaluation protocol helps ensure fair comparisons.

  • The NVS results are promising, with notable improvements upon previous methods in both qualitative and quantitative results.

Critical weaknesses
  • The technical contribution of the method is limited. The NVS model is largely based on LagerNVS; the appearance encoding and conditioning are also present in previous work like WildCAT3D. The contribution of transient feature suppression is a novel idea, but it lacks sufficient study. Comparisons with simpler alternatives are missing. For example, how does the feature suppression compare against simply zeroing out transient features?

  • The ablation study on "Transient feature suppression" is not convincing. Although visible in the qualitative results, the quantitative results remain mixed. Also, will the results be better if the network does not have to infer transients itself? By using the transient map estimated from the pre-trained segmentation network as an input along with source images.

  • Can the model generalize to more than 12 images? It would be helpful to understand if more images will help NVS and to what extent.

  • Concern about comparison fairness. GenWildSplat does not use input pose, while Wild-NVS does. How much improvement is coming from using GT camera pose?

  • Lack of visualization of the dataset. Some examples of selected source and target samples would be helpful for understanding the dataset.

Questions for authors
  • How much training is conducted for LagerNVS-FT in Table 1? Is it a fair comparison with Wild-NVS?

  • How is evaluation conducted under different resolutions? Since different methods use different resolutions: 448×448 for GenWildSplat, 504×504 for WildSplatter, and 512×512 for Wild-NVS.

5 comments
AuthorsOpenReview ↗

Response to reviewer KZLJ

W1: Limited technical contribution, and ablation with simpler transient suppression methods.

Wild-NVS replaces likely transient tokens with a learnable transient embedding rather than discarding them. This lets the decoder draw on static content from other views while discounting uncertain transient tokens. We compare this with simply zeroing the transient features. On both datasets, zeroing slightly degrades performance, with a larger drop on PhotoTourism (Tables R1 and R2). This suggests that the learned embedding helps the model rely on static regions in other views rather than losing that information.

Table R1: Naive transient suppression on Wild-NVS (4–12 views average).

MethodPSNR ↑SSIM ↑LPIPS ↓
Wild-NVS17.750.5350.438
Transients zeroed17.710.5340.442

Table R2: Naive transient suppression on PhotoTourism.

MethodPSNR ↑SSIM ↑LPIPS ↓
Wild-NVS19.980.6250.328
Transients zeroed19.840.6190.331

W2.1: Quantitative support for transient suppression.

We ran a blind human study comparing the transient-aware Wild-NVS with the appearance-encoder-only Wild-NVS-A. We sampled 60 Wild-NVS test targets whose source views contain visible transients. For each target, evaluators saw the ground-truth image and four unlabeled renderings in random order: GenWildSplat, WildSplatter, Wild-NVS-A and Wild-NVS. Both of our models are trained for 100K steps. Evaluators chose the best rendering free of transient objects. Eleven evaluators gave 207 judgments in total through an online form, rating as many samples as they wished. Table R3 reports the share of judgments per method, with 95% confidence intervals from a bootstrap over test targets (4,000 resamples).

Table R3: Human preference for transient-free renderings.

MethodChosen by evaluators95% CI
Wild-NVS91.3%86.9--95.3
Wild-NVS-A5.3%2.4--8.3
GenWildSplat1.4%0.0--3.2
WildSplatter1.9%0.5--3.9

Evaluators preferred Wild-NVS by a wide margin, and its confidence interval does not overlap with any other method’s. This indicates that transient feature suppression clearly improves transient handling over the appearance-only model.

W2.2: Using segmentation masks from pre-trained segmentation model as input.

We first measure how well a pretrained segmentation model identifies transients. We manually annotated 100 test source views with SAM-assisted masks where the annotator saw the same 8 source views as the model, so persistence could be judged across views. Table R4 compares raw YOLOv8 masks, our curated labels (YOLOv8 with the SfM persistence filter), and our transient head, with 95% bootstrap confidence intervals over images.

Table R4: Evaluation of transient Label quality and transient head prediction.

MethodRegionPrecisionRecallIoUAP
YOLOCOCO list0.75 [0.67, 0.82]0.54 [0.40, 0.72]0.46 [0.34, 0.59]0.48 [0.35, 0.64]
LabelsCOCO list0.83 [0.77, 0.88]0.47 [0.34, 0.63]0.43 [0.32, 0.57]0.41 [0.30, 0.55]
Transient HeadAll transients0.56 [0.48, 0.62]0.80 [0.64, 0.90]0.49 [0.40, 0.56]0.68 [0.57, 0.77]
Transient HeadOutside COCO list0.19 [0.04, 0.32]0.49 [0.15, 0.78]0.16 [0.03, 0.28]0.13 [0.03, 0.28]

Because the head predicts from multi-view features, we believe it learns transience largely as inconsistency across views rather than from object semantics. As a side effect, static regions visible in only one source view also receive slightly higher transient scores. These false positives, together with the head’s lower-resolution soft map, which extends past object boundaries, account for its lower precision compared with the sharp instance masks of the labels. Its threshold-free AP is nevertheless clearly higher, showing that it separates transient from static pixels better overall. Moreover, suppression is soft, so a static region flagged with moderate probability is only partially replaced by the transient embedding rather than discarded.

A YOLOv8 mask recovers only 54% of the transient area (AP 0.48), compared with 80% for our transient head (AP 0.68). It also cannot detect transients outside the COCO classes, such as tents, market stalls and banners, while the head recovers 49% of their area. Whether such an object is transient often cannot be decided from a single image, because the same category can be permanent in another scene, whereas our head predicts from multi-view aggregated features and can draw on the other views. A segmentation mask alone therefore provides a less complete transient signal than the learned head, and it would require running the segmentation network at inference time. Combining a mask with the learned head requires training a mask-conditioned variant which did not fit within the rebuttal period given the compute requirements.

AuthorsOpenReview ↗

Response to reviewer KZLJ contd.

W3: Generalization to more than 12 images.

Yes. Wild-NVS continues to improve up to 18 source views, although it was trained with only 2–8. We extend the benchmark to 14, 16 and 18 source views. To keep the source sets nested (each 14-view set is a subset of the 16-view set, and so on), we use only test targets with at least 18 source views, which reduces the test set from 1,048 to 946 targets. The values therefore differ slightly from Table 1 of the paper (e.g. 17.34 vs 17.37 dB at 4 views). All three metrics improve steadily with more views. The gains shrink beyond 12 views because each additional view adds less new coverage of the target.

Table R5: Wild-NVS with increasing source views (946 targets).

ViewsPSNR ↑SSIM ↑LPIPS ↓
417.340.5250.464
617.600.5330.447
817.760.5380.436
1017.860.5420.428
1217.960.5440.422
1418.000.5460.418
1618.040.5470.415
1818.060.5480.413

W4: Concerns about comparison fairness with pose information.

Each baseline is evaluated with its own published inference protocol:

  • GenWildSplat cannot take cameras as input and its evaluation predicts the target camera with VGGT, jointly with the source views, and renders the target from that prediction.
  • WildSplatter uses Depth-Anything-3 to predict the source cameras and geometry. The ground-truth COLMAP target pose is re-expressed relative to ground-truth source view 0. Its translation is then scaled by the mean ratio of predicted to ground-truth distances between source-camera centers. The target is rendered from this rescaled ground-truth (COLMAP) pose.
  • Wild-NVS is given ground-truth (COLMAP) source and target cameras.

We report evaluations for Wild-NVS with no source cameras at all, in two settings that match the camera information available to the baselines in the Table below.

Table R6: Effect of camera information (averaged over 4–12 views).

MethodTarget cameraSource camerasPSNR ↑SSIM ↑LPIPS ↓
GenWildSplatVGGT-predictednone13.350.3940.619
Wild-NVSVGGT-predictednone15.280.4700.604
WildSplatterCOLMAPnone13.370.3850.599
Wild-NVSCOLMAPnone15.360.4720.601
Wild-NVSCOLMAPCOLMAP17.750.5350.438

With the same camera information, Wild-NVS outperforms both baselines by 1.9–2.0 dB PSNR. Its LPIPS is better than GenWildSplat’s and on par with WildSplatter’s. With COLMAP source cameras, Wild-NVS improves by a further 2.39 dB. It can render without source cameras because source-camera tokens are dropped with a fixed probability during training. We have not tuned this probability to trade off pose-free against posed performance.

W5: Lack of visualization of the dataset.

We have included examples of the view-sets (source views and target view) from our dataset in supplementary section E.6 for a better understanding of the dataset.

Q1: Clarify training details of LagerNVS-FT.

Table 1 in the original paper reported numbers at 100K steps with all other settings fixed, due to resource limitations. We have updated the numbers of the LagerNVS-FT model trained with the same compute budget as Wild-NVS (125K steps) under the same training schedules, learning rates, and source-view sampling in the main paper. We also provide them below. On the Wild-NVS dataset the PSNR remains unchanged (14.99 dB) with a slightly worse SSIM and LPIPS score. On PhotoTourism, it lowers the PSNR by 0.37 dB, which we attribute to the smaller evaluation set on 3 scenes. Our method Wild-NVS outperforms LagerNVS-FT at both checkpoints on both datasets leaving the conclusions from the main paper unchanged.

Table R7: LagerNVS-FT at 100K and 125K steps in the Wild-NVS dataset.

MethodPSNR ↑SSIM ↑LPIPS ↓
LagerNVS-FT (100K)14.990.5030.488
LagerNVS-FT (125K)14.990.5010.492
Wild-NVS17.750.5350.438

Table R8: LagerNVS-FT at 100K and 125K steps in the PhotoTourism dataset.

MethodPSNR ↑SSIM ↑LPIPS ↓
LagerNVS-FT (100K)16.180.5990.369
LagerNVS-FT (125K)15.810.5900.376
Wild-NVS19.980.6250.328

Q2: Evaluation details for different method resolutions.

Since the baseline methods use different resolutions, each method gets the ground truth image from the original photo at its own resolution. The ground truth image is a square center crop, which is resized to 448×448448 \times 448 for GenWildSplat, 504×504504 \times 504 for WildSplatter, and 512×512512 \times 512 for Wild-NVS, ensuring the scores are calculated on the same region.

Thank the authors for the detailed response. The additional experiments using more input views, which resulted in better NVS quality, are promising. The clarification on experimental protocols and the visualization of the dataset are helpful as well. Therefore, I will maintain my positive score.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Review response

We thank all reviewers (KZLJ, kVux, 8XJk) for their valuable feedback and suggestions. The reviewers find that the Wild-NVS dataset and benchmark fill a real gap and could drive further progress in the field (KZLJ, kVux, 8XJk), that the problem setting is relevant and insufficiently studied (kVux), and that the method achieves notable, consistent improvements over prior methods both quantitatively and qualitatively (KZLJ, 8XJk). Latent transient-token suppression is considered a novel idea (KZLJ) that is well matched to latent NVS architectures (kVux), and the appearance transfer is found to work well (8XJk). The paper is found clear and easy to follow (kVux, 8XJk).

We have provided detailed responses to each concern and revised the manuscript accordingly. We summarize the key updates below:

  1. Quantitative evidence for transient suppression: A blind human study shows a strong preference for the transient-aware model over the appearance-only model (91.3% vs 5.3%), and a human-validated artifact analysis confirms fewer ghost-like transient artifacts (7.7% vs 18.0%). We also compare our learned transient embedding against naive zeroing of transient tokens.
  2. Accuracy of transient labels and predictions: On 100 manually annotated source views, the transient head recovers substantially more transient area than both its training labels and a pretrained segmentation model (recall 0.80 vs 0.47 and 0.54; AP 0.68 vs 0.41 and 0.48), including transients outside the COCO classes.
  3. Fair comparison with the same color and pose information: We evaluate Wild-NVS without source cameras under each baseline’s camera setting, and with restricted appearance information (a source view, no reference, or half of the target), together with an oracle color-corrected LagerNVS-FT. Wild-NVS outperforms the baselines in PSNR across all of these settings. We also clarify the appearance and camera protocol in Sec. 6.1 and revise our state-of-the-art claims.
  4. Training budget and generalization to increasing views: LagerNVS-FT is retrained with the same 125K-step budget as Wild-NVS, and Wild-NVS is evaluated with up to 18 source views, demonstrating generalization beyond its training range of 2–8 views.

In addition, the rebuttal provides further clarifications on evaluation resolutions, the VLM judge, and dataset examples. Changes made so far are highlighted in the revised paper, and the remaining results will be included in the final version.

We hope that these revisions and our responses address the reviewers’ concerns, and we welcome further discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.