- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper addresses feed-forward novel view synthesis (NVS) from unconstrained internet photo collections. While existing datasets or training settings consider little of the effects of real-world appearance variation, transient occlusion, or diverse source-view configurations, this work builds the Wild-NVS dataset and benchmark to serve as a more comprehensive evaluation protocol for real-world applicability. The dataset is built on MegaDepth-X reconstructions. Each target is paired with an ordered list of up to 64 source views, chosen by a bidirectional depth-warping covisibility test, joint target coverage, and viewpoint spread.
Then, this work introduces a Wild-NVS model that extends the latent NVS model LagerNVS with an appearance encoder and a transient head to handle real-world appearance variation and transient occlusion, respectively. Extensive experiments and ablations have demonstrated the effectiveness of the proposed method over baselines under real-world conditions.
Critical strengths
-
It builds a reproducible benchmark that fills a real gap. Prior feed-forward in-the-wild methods are evaluated on unnamed scenes and unlisted images, so their numbers cannot be reproduced or compared. Wild-NVS fixes this with landmark-level splits (the PhotoTourism landmarks held out of training), source sets checked against depth with a bidirectional covisibility test, and full curation details. The authors also promise to release the dataset, which could be a lasting, reusable contribution, independent of the specific model.
-
Experimental results demonstrate large, consistent empirical gains, with a clean data ablation. LagerNVS-FT (the same architecture, only fine-tuned on the new data) outperforms the original LagerNVS at every source count. On Wild-NVS, it also outperforms both GS baselines on every metric. Qualitative results also demonstrate that the appearance transfer works well. Conditioning images from other scenes change tone, brightness, and sky while keeping the structure (Fig. 1), which suggests the 8-dimensional bottleneck works as intended. Even if part of the margin comes from the protocol (see Weaknesses), gaps of this size are unlikely to disappear.
Critical weaknesses
Major
-
The quantitative evaluation does not specify which appearance-conditioning image is used. The paper states that the target image supplies appearance during training; Figure 1 includes a target-conditioned result, Figure 3 discusses matching the target appearance, and Figure 4 uses target-image appearance codes. If the ground-truth target is also used for conditioning in Tables 1–2, Wild-NVS receives target color information unavailable to LagerNVS-FT.
-
The GS baselines appear not to be retrained on Wild-NVS, run at different resolutions, and GenWildSplat cannot use the provided cameras. "SOTA" mixes data, pose and appearance-access effects.
Minor
- Another concern is the little quantitative support for the transient module, the main new modeling component. It brings little PSNR gains and SSIM can even degrade (Table 6).
- As for the VLM judgment, the full model is always "A", the number of pairs is unstated. There is no human check, and baselines are not judged. Providing some details can be more informative, e.g., how many pairs were judged, and was the A/B order randomized? How well do the VLM labels agree with human labels on a subset? What do the same judgments give for the baselines?
- The standalone training protocol for LagerNVS-FT is not explicitly specified. Tables 3 and 6 appear to imply 100K steps, but the schedule and source-view sampling remain unclear.
Questions for authors
- Section 6.1 appears to use predicted target poses for GenWildSplat. Does this apply only to GenWildSplat, and how are its predicted poses aligned with the ground-truth evaluation cameras?
- How many steps was LagerNVS-FT trained for, and with what schedule and source sampling? Is it compared at the same step count as Wild-NVS?
6 comments
Response to reviewer 8XJk contd.
Minor W1: Quantitative evidence for the transient module.
We conducted a blind human study comparing the transient-aware Wild-NVS with the appearance-encoder-only Wild-NVS-A. We sampled 60 test targets containing visible transients in the source views from the Wild-NVS test set. Evaluators saw the ground-truth target and four renderings, in randomized order and without method names: GenWildSplat, WildSplatter, Wild-NVS-A (appearance encoder only), and Wild-NVS. Both of our models are trained to 100K steps. Evaluators were asked to choose the best render without the transient objects. Eleven evaluators rated as many samples as they chose through an online form, for 207 judgments in total. We report the share of judgments selecting each method and the 95% CI from a bootstrap over test cases with 4,000 resamples.
The human evaluation shows a significant preference for renderings from the transient-aware model, highlighting the improvement in rendering quality from transient feature suppression over the appearance-encoder only model.
Table R3: Human preference for transient-free renderings
| Method | Chosen by evaluators | 95% CI |
|---|---|---|
| Wild-NVS | 91.3% | 86.9--95.3 |
| Wild-NVS-A | 5.3% | 2.4--8.3 |
| GenWildSplat | 1.4% | 0.0--3.2 |
| WildSplatter | 1.9% | 0.5--3.9 |
Minor W2: Details about VLM judge and agreement to human labels.
GPT-6 Astra compared the appearance-encoder-only model (Wild-NVS-A) with the full Wild-NVS on 1,626 rendering pairs. The order of the two renderings was not randomized, which could introduce an order bias. We have added these details to the supplementary material.
To validate the VLM labels, we ran a human evaluation on 120 of the 1,626 pairs from Table 8 of the appendix. We sampled these to over-represent pairs flagged by GPT-6 and weighted the rates back to the full set. Raters saw the two renderings side by side, in random order and without method names, and answered the same question as GPT-6. Eight raters gave 244 judgments in total.
Table R4: Agreement between GPT-6 and human labels.
| Wild-NVS-A | Wild-NVS | |
|---|---|---|
| GPT-6–human agreement (artifact / no artifact) | 66% | 81% |
| Human–human agreement (artifact / no artifact) | 73% | 81% |
| Artifact rate estimated from people (all 1626 pairs) | 18% | 7.7% |
| Artifact rate from GPT-6 | 10.2% | 1.6% |
GPT-6 agrees with people nearly as often as people agree with each other. Both GPT-6 and people find artifacts more often in the appearance-encoder only model. The reweighted rate based on human evaluation is 18.0% flagging the appearance-encoder only model vs 7.7% for the full Wild-NVS model (a difference of 10.3 points, 95% CI 4.1 to 16.9). The evaluation using GPT-6 was 10.2% flagging the appearance-encoder only model vs 1.6% for the full Wild-NVS model (8.6 points difference). The preference for the transient-aware model therefore holds under both judges, but people mark artifacts more often than GPT-6 for both models. We therefore report the human-estimated rates in the main paper.
The baselines were not included in the VLM comparison because their renderings are substantially lower in quality (about 4 dB PSNR below Wild-NVS), and the study targets transient mitigation within our model. All four methods are compared in the human preference study (Minor W1, Table R3).
Minor W3/Q2: Training protocol for LagerNVS-FT not stated.
The original paper reported LagerNVS-FT metrics at 100K steps, trained under the same schedule and source-view sampling settings due to resource limitations. We have updated the numbers of the LagerNVS-FT model trained with the same compute budget as Wild-NVS (125K steps) under the same training schedules, learning rates, and source-view sampling in the main paper. We also provide them below. On the Wild-NVS dataset the PSNR remains unchanged (14.99 dB) with a slightly worse SSIM and LPIPS score. On PhotoTourism, it lowers the PSNR by 0.37 dB, which we attribute to the smaller evaluation set on 3 scenes. Our method Wild-NVS outperforms LagerNVS-FT at both checkpoints on both datasets leaving the conclusions from the main paper unchanged.
Table R5: LagerNVS-FT at 100K and 125K steps in the Wild-NVS dataset.
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|
| LagerNVS-FT (100K) | 14.99 | 0.503 | 0.488 |
| LagerNVS-FT (125K) | 14.99 | 0.501 | 0.492 |
| Wild-NVS | 17.75 | 0.535 | 0.438 |
Table R6: LagerNVS-FT at 100K and 125K steps in the PhotoTourism dataset.
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|
| LagerNVS-FT (100K) | 16.18 | 0.599 | 0.369 |
| LagerNVS-FT (125K) | 15.81 | 0.590 | 0.376 |
| Wild-NVS | 19.98 | 0.625 | 0.328 |
Response to reviewer 8XJk contd.
Q1: Pose information for baseline methods.
Among the baselines this applies only to GenWildSplat. Each method is evaluated with its own published inference protocol:
- GenWildSplat cannot take cameras as input and its evaluation predicts the target camera with VGGT, jointly with the source views, and renders the target from that prediction. The Gaussians are built in the same predicted frame, so no alignment to the ground-truth cameras is needed. Metrics are computed on the rendered image and the ground-truth image directly.
- WildSplatter uses Depth-Anything-3 to predict the source cameras and geometry, so the Gaussians live in that predicted frame, with source view 0 at the origin. The ground-truth COLMAP target pose is re-expressed relative to ground-truth source view 0. Its translation is then scaled by the mean ratio of predicted to ground-truth distances between source-camera centers. The target is rendered from this rescaled ground-truth pose.
- Wild-NVS is given ground-truth COLMAP source and target cameras.
To remove the asymmetry, we report evaluations for Wild-NVS with no source cameras at all, in two settings that match the baselines' camera information in the Table below.
Table R7: Evaluation without source cameras (averaged over 4–12 views).
| Method | Target camera | Source cameras | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|---|---|
| GenWildSplat | VGGT-predicted | none | 13.35 | 0.394 | 0.619 |
| Wild-NVS | VGGT-predicted | none | 15.28 | 0.470 | 0.604 |
| WildSplatter | COLMAP | none | 13.37 | 0.385 | 0.599 |
| Wild-NVS | COLMAP | none | 15.36 | 0.472 | 0.601 |
In the VGGT-predicted setting, the target pose is obtained exactly as in GenWildSplat. With the same camera information, Wild-NVS outperforms both baselines by about 1.9–2.0 dB PSNR. Its LPIPS is better than GenWildSplat’s and on par with WildSplatter’s.
We also applied WildSplatter's alignment to GenWildSplat, rendering from the rescaled ground-truth target pose and intrinsics instead of its prediction. This lowers its PSNR by 0.9–1.4 dB. The published predicted-pose default protocol is thus the more favorable setting for GenWildSplat.
Response to reviewer 8XJk
In order to evaluate the method quantitatively in Tables 1 and 2 of the main paper, we provide the baselines, and Wild-NVS the target image to extract the appearance embedding. However, LagerNVS-FT cannot be provided this information due to its architectural limitations. We have included this in Section 6.1 of the paper covering the evaluation protocol.
To isolate the target color information available to Wild-NVS, We train a variant, Wild-NVS†, whose appearance encoder sees only a randomly chosen left or right half of the target, mirrored to full width. The loss is applied only to the other half, and training runs for 100K steps. At test time, the encoder receives the left half and metrics are computed on the right half.
For comparison of LagerNVS-FT, we map the rendered image to the target image using an affine color correction following RawNeRF [1]. For each test image we fit a per-channel scale and shift parameter (6 parameters) using least squares, mapping the left half of the render to the global color and exposure in the left half of the target image. We apply it to the rendered image and compute the metrics on the right half of the image. This establishes an oracle baseline (LagerNVS-FT$^\mathcal{O})usesthetargetimagedirectlytofitthecolortransform,unlikeabottleneckedconvolutionalappearanceencoderinourmethod.Wereportcomparisonstoourmodeltrainedwiththeadaptedappearanceencoder(Wild−NVS^\dagger$), and also after applying the affine color correction to our model output (Wild-NVS$^{{\dagger}\mathcal{O}}$) in Table R1 below.
Table R1: Appearance input and available color information (100K steps, averaged over 4–12 views).
Wild-NVS† outperforms the oracle LagerNVS-FT$^\mathcal{O}$ on all three metrics (17.03 vs 16.14 dB PSNR). Because the oracle’s global tone and exposure are already fitted to the target, this 0.89 dB gap cannot be explained by global color matching. It reflects improvements beyond a per-channel affine correction, such as local illumination and transient handling. Applying the same affine correction to Wild-NVS$^\dagger$ brings no further gain, which indicates that our appearance encoder already recovers the global tone and exposure from half of the target image.
We acknowledge the differences amongst the baselines in the architecture. We have revised our claims of state-of-the-art accordingly. We separate the three factors as follows.
Table R2: Effect of source-camera information (averaged over 4–12 views).
Without source cameras, Wild-NVS still outperforms both baselines by about 2.0 dB PSNR. Its LPIPS is better than GenWildSplat’s and on par with WildSplatter’s. Since Wild-NVS can leverage the pose information, it improves further by 2.39 dB PSNR and lowers the LPIPS score by 0.163. This pose-free rendering capability is achieved due to the dropout of source camera tokens at a fixed probability during training. However, we have not tuned this dropout hyperparameter for a pose-free vs posed rendering performance trade-off.
[1] Mildenhall, Ben, et al. "Nerf in the dark: High dynamic range view synthesis from noisy raw images." 2022 IEEE/CVF conference on computer vision and pattern recognition (CVPR). IEEE, 2022.