acceptodds
Under review as a conference paper at ICLR 2027

HiFiNVS: High-Fidelity Novel View Synthesis from Unposed Images

Abstract

Feed-Forward Novel View Synthesis from unposed images has recently benefited from pretrained visual geometry models, which provide strong camera and dense geometry priors for Gaussian reconstruction. Nevertheless, the resulting Gaussian predictions can remain inconsistent across views, while errors in the estimated cameras often lead to ghosting and blurred details. We introduce HiFiNVS, a backbone-agnostic NVS framework that exploits predicted geometry not only for reconstruction, but also to establish explicit cross-view constraints for feature adaptation and camera refinement. Specifically, our Reprojection-Grounded Graph Attention constructs a patch graph by reprojecting 3D points across images and propagates information along geometrically grounded correspondence edges, thereby improving the cross-view consistency of Gaussian predictions. We further introduce a feed-forward camera refiner that updates the predicted cameras using rendering feedback from the initial Gaussian scene. Rerunning our reconstruction once with the refined cameras effectively suppresses rendering artifacts. HiFiNVS keeps the visual geometry backbone frozen and can be instantiated with DA3, π³, and VGGT. Across all three backbones, camera refinement improves both camera accuracy and rendering quality. Experiments on unposed-view benchmarks show that HiFiNVS achieves state-of-the-art NVS performance.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward Reconstruction, Novel View Synthesis, Gaussian Splatting, 3D Reconstruction

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_EtJpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors attempted to solve the inaccurate camera prediction problem in the current feed-forward GS reconstruction methods. They leveraged DA3 as their geometry backbone and proposed an RGGA module, which is claimed to obtain better 3DGS reconstruction. Then they proposed a render-aware camera refiner for better camera prediction. Overall, it is good. But I have some concerns about the contribution of RGGA and the motivation and objective of this paper. For example, it seems the goal is to get more accurate camera parameter estimation. Then, have related works like VGGT-Omega and Pi^3 already predicted accurate camera parameters? In the experiments, the authors mostly show the rendering results, but lack camera parameter prediction experiments.

Critical strengths
  1. The introduction of Render-Aware Camera Refiner is interesting. It replaces standard Test-Time Training (TTO).
  2. The discussion in the paper is very clear.
  3. The experiments are mostly concrete.
  4. Ablations on different backbones are good.
Critical weaknesses
  1. The authors should show comparisons of the reconstructed GS scenes after Gaussian Head with RGGA and those predicted by the existing feed-forward 3DGS reconstruction work to demonstrate the contribution of the introduced RGGA. If the ones after RGGA are worse than the existing SOTA feed-forward 3DGS reconstruction works. Then why not skip this module and jump to the camera refiner?
  2. The authors should do ablation studies on the middle module shown in Figure 2 to demonstrate that this module is important.
  3. After reading, I feel like the camera refiner is a feed-forward alternative to TTO. If so, the authors should show comparisons with TTO methods, especially the latency. I believe feed-forward methods should be more efficient than TTO methods.
  4. Why don't you compare with AnySplat under posed settings? Also, E-Rayzer might be a strong related baseline?
  5. Lack of camera parameter comparisons.
Questions for authors
  1. The authors claimed in the paper that the existing methods have errors in the estimated cameras, leading to ghosting and blurred details. Then why do the authors still use the predicted camera and depth to obtain the correspondence?
  2. In Line159 - 161. How do you sample pixels? Like what the authors said in Line 218 - 219? If so, I suggest moving this part to the front.
  3. In Line 187- 189. Did you use thresholds? If so, how do you deal with the scene scale differences?
  4. How do you inject GG into graph attention? An inductive bias in global attention?
  5. After Gaussian Decoding, how do you supervise the outputs? If using a rendering loss, which camera did you use? The ones from the Geometry Backbone? Or GT cameras?
  6. Can Render-Aware Camera Refiner work without the RGGA module? Can a more powerful GS prediction backbone eliminate the requirement of the RGGA module?
2 comments
AuthorsOpenReview ↗

Response to Reviewer EtJp

Thank you for the detailed feedback and concrete suggestions for improving the paper. To address the questions in your summary, we clarify that our objective is high-fidelity novel view synthesis from unposed images, with rendering quality as the primary focus. Within HiFiNVS, RGGA is part of the Gaussian head and improves cross-view feature aggregation for reconstruction. The camera refiner is a separate stage that uses rendering feedback to improve NVS quality by correcting source-camera errors. Camera accuracy therefore serves the NVS objective, rather than being a separate goal. We address your concerns below.

1. Reconstruction comparisons and Gaussian head ablations (Weaknesses 1–2)

Our main experiments compare HiFiNVS with existing feed-forward 3DGS methods. Tables 1–2 show that HiFiNVS achieves state-of-the-art NVS quality with GT cameras (ReSplat-re uses multi-step refinement). For unposed inputs, the separate camera refiner improves rendering quality by reducing artifacts caused by errors in the backbone’s predicted cameras.

To isolate RGGA’s contribution within the Gaussian head (the middle module in Figure 2), we compare two ablated variants with the full model. We apologize for the unclear ablation labels: “w/o graph attention” in Table 6(a) corresponds to removing RGGA's reprojection-guided aggregation while retaining frame attention; “Global + frame attention” replaces RGGA's graph attention with standard global self-attention. The full model achieves better NVS quality under GT cameras, establishing the benefit of RGGA.

2. Quality and latency versus test-time optimization (Weakness 3)

Thank you for suggesting this important comparison. We use Adam with a learning rate of 10⁻⁴ to minimize a source-view rendering loss with respect to camera intrinsics and extrinsics. TTO-10 and TTO-20 use 10 and 20 camera optimization iterations, respectively. We keep the Gaussians fixed during optimization and rebuild the scene once with the updated cameras. We evaluate 140 DL3DV scenes per backbone at 480P with 16 input views. Each quality entry is PSNR (dB) / corrected AUC@5° (%) (both ↑):

MethodDA3π³VGGTTime cost (s) ↓
w/o refiner26.40 / 94.0426.08 / 88.0624.08 / 79.040.00
TTO-1026.65 / 94.4926.38 / 88.9024.25 / 79.4329.34
TTO-2026.66 / 94.6626.42 / 89.3424.28 / 79.7058.51
Our refiner27.29 / 96.0626.70 / 93.4925.91 / 87.150.24

In this comparison, our refiner yields higher PSNR and pose AUC at lower measured latency than both TTO variants. All variants use the same target-camera evaluation procedure. Time cost is the average runtime for camera correction and rebuilding on an RTX 5090. Our refiner has only 18.3M parameters, including an 11.7M frozen image encoder.

3. Posed AnySplat comparison and E-RayZer baseline (Weakness 4)

AnySplat's official model uses unposed inputs and has no GT-source-camera inference interface; Table 2 evaluates this native setting. E-RayZer’s released DL3DV evaluation uses 256×256 images. We found no released high-resolution checkpoint, so we exclude E-RayZer from our high-resolution comparison. We will expand Table 7 (Appendix C.1) to clarify these configurations and comparison choices.

4. Camera parameter comparisons (Weakness 5)

Table 3 reports camera accuracy alongside NVS quality; Table 8 (Appendix D.1) extends the reported comparison to 8/16/32 views across DA3, π³, and VGGT. Our additional comparison above confirms improvements in both NVS quality and camera pose accuracy.

5. Additional questions

(Due to the character limit, please see Part 2 for our responses.)

AuthorsOpenReview ↗

Response to Reviewer EtJp

Due to the character limit, we address your additional questions in this second comment.

5. Additional questions

(a) Great question. NVS requires precise pixel-level alignment and is sensitive to camera/depth errors. However, for patch-level correspondences, the cameras and depth predicted by large visual geometry models provide useful geometric priors (visualized in Figure 8, Appendix E.2). Table 6(a) shows that using these coarse correspondences to guide attention improves NVS quality.

(b) Yes, we use the same sampling scheme. We will move the sampling details to their first occurrence in §3.2.

(c) Appendix A.2 specifies a relative-depth threshold of 10%, so rescaling the scene does not change the depth-consistency test.

(d) RGGA performs local self-attention on G: each patch provides the query, and its graph neighbors provide keys and values. The graph edges determine which neighbors each query attends to (§3.2; Appendix A.2).

(e) Reconstruction uses rendering loss on 8 additional target views (§3.5; Appendix B.2), following YoNoSplat's mix-forcing of GT and predicted source poses. Targets use aligned GT-based poses and the same intrinsics as the source branch (Appendix B.1).

(f) Yes, the camera refiner is independent of RGGA. It uses source images, predicted cameras, and rendered RGBD to refine cameras. Our frozen camera refiner improves other GS methods' unposed NVS quality without fine-tuning (Tables 4 and 10, Appendix D.3). These gains demonstrate that the refiner generalizes across Gaussian reconstruction methods.

We hope these clarifications and additional results address your concerns. We welcome any further questions.

Reviewer_kA3COpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper aims to resolve feed-forward NVS from unposed images. The proposed HiFiNVS resolves the inconsistency, ghosting, and blurred details by boosting the attention of the Gaussian prediction head with a patch correspondence graph. And it introduces a separate render-aware camera refiner that observes source images, initial RGB/depth renderings, and residual features, and predicts bounded camera updates. Across three unposed-view benchmarks, HiFiNVS shows competitive performance under sparse settings.

Critical strengths
  1. HiFiNVS introduces a novel paradigm to enhance pose accuracy and multi-view consistency of predicted 3DGS. The explicit 3D correspondences prove to be an effective guide for enhancing feature aggregation.
  2. Strong, consistent reconstruction quality with meaningful component evidence. Table 2 and Figure 4 show clear superiority over existing methods for unposed 8/16/32 views. Table 6 sufficiently supports the geometry-grounded module design rather than merely added capacity.
Critical weaknesses
  1. A side effect of the refiner is the additional cost of graph rebuild and Gaussian head rerun. It would be helpful to analyze the cost of this process. It's also interesting to see whether the performance would improve with multiple rounds of refinement and how it trades off with computational overhead.
  2. DA3, Pi3, and VGGT can infer over hundreds of images, yet the validation is limited to a sparse setting with at most 32 views. Considering HiFiNVS as a general feed-forward NVS method for unposed images, it's suggested to supplement the results and prove generalization ability under dense view settings, such as 64/128/256 images.
Questions for authors

See weakness.

2 comments
AuthorsOpenReview ↗

Response to Reviewer kA3C

Thank you for recognizing our geometry-guided feature aggregation and the consistent reconstruction quality supported by our component analysis. We address your two concerns below.

1. Runtime cost and the effect of additional refinement rounds (Weakness 1)

We agree that measuring the extra cost of camera refinement is important. Table 5 breaks down the runtime, including the cost of rebuilding the graph and rerunning the Gaussian head. Rebuilding reuses cached backbone features and depth, so camera refinement does not require another backbone pass.

Great point about testing multiple refinement rounds! We additionally fine-tuned the refiner with mixed one- and two-round refinement for 5k iterations. The table below reports results on the DL3DV benchmark.

RefinementAUC@5° (%) ↑PSNR (dB) ↑
Before92.2127.33
One round95.8728.26
Two rounds95.0428.18

In this evaluation, the second round required another feedback and rebuild pass and yielded no further gain in either PSNR or AUC. This supports our choice of a single refinement round.

2. Generalization to dense input views (Weakness 2)

Your suggestion to test denser inputs addresses an important question about scalability. We evaluated our existing HiFiNVS model at 960P with 64/128/256 input views:

Input viewsGT-posed PSNR (dB) ↑Unposed PSNR (dB) ↑
6426.7925.68
12826.6425.31
25625.3623.94

These results demonstrate HiFiNVS’s generalization to dense-view inputs. Furthermore, RGGA’s local graph attention allows query chunking to reduce GPU memory usage for attention, enabling HiFiNVS to extend backbone predictions to NVS at larger view counts.

We hope the responses above address your concerns. We would be happy to clarify any remaining points and welcome your further feedback.

Thanks to the authors' effort in the additional experiment and analysis. My concern has been well resolved, and I would maintain my recommendation for weak acceptance.

Reviewer_GqTKOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

Authors propose a novel view synthesis module built upon a frozen, pretrained feed-forward geometry model. The proposed module uses predicted depth and camera parameters to construct a reprojection graph, then applies the proposed RGGA module to aggregate cross-view features for Gaussian prediction. Rendering loss is used to predict camera corrections, and the scene is reconstructed with the refined cameras.

The module is injected into some state-of-the-art feed-forward geometry model, reporting improvements in both novel view synthesis and camera pose prediction.

Critical strengths
  1. The method achieves strong novel view synthesis performance.

  2. RGGA has a clear motivation, using reprojection-based correspondences to guide cross-view feature aggregation.

Critical weaknesses
  1. Rendering error can arise from both camera pose errors and depth estimation error. Although GT camera supervision and bounded updates constrain the refiner, it remains unclear whether it may compensate for inaccurate geometry by moving cameras away from their correct poses. Could the authors independently vary geometry and camera pose errors to evaluate this behavior?

  2. The quality improvements come with additional computation for rendering feedback and scene reconstruction. The runtime breakdown does not establish whether this computation is more beneficial than allocating a similar budget to a larger backbone. Could the authors compare a smaller-backbone system equipped with the proposed modules against a larger-backbone baseline under matched end-to-end inference latency? Does the proposed approach achieve better NVS quality at the same time budget, or comparable quality with lower latency?

Questions for authors

See Weakness

7 comments
AuthorsOpenReview ↗

Response to Reviewer GqTK

Thank you for the thoughtful review and for highlighting the strong NVS performance and the motivation behind RGGA. To clarify the training setup, the camera refiner is supervised only by GT camera parameters, without a rendering loss (§3.5). We address your two concerns below.

1. Impact of inaccurate geometry on camera refinement (Weakness 1)

We appreciate your question about whether geometry errors could lead the refiner to move cameras away from their correct poses. Tables 3 and 8 (Appendix D.1) report improvements in both camera pose accuracy and NVS quality across DA3, π³, and VGGT. (Pose AUC measures relative camera pose accuracy against ground truth, with higher values indicating better accuracy.) These results show that the observed NVS gains are accompanied by improved camera accuracy.

Following your suggestion, we independently perturb DA3-G’s predicted source poses and depth on 140 DL3DV scenes (480P, 16 input views). For small, medium, and large perturbations (S/M/L), we apply either 0.5°/1°/2° camera rotations with center shifts of 0.5%/1%/2% of the median adjacent-view baseline or smooth multiplicative depth noise with relative root-mean-square (RMS) amplitudes of 2.5%/5%/10%.

The table reports source pose AUC and held-out PSNR before → after camera refinement and rebuilding.

Metricw/o perturbationCam-SCam-MCam-LDepth-SDepth-MDepth-L
AUC@5° (% ↑)85.14 → 88.7279.61 → 84.2868.31 → 73.9942.40 → 46.2385.14 → 88.3385.14 → 87.8985.14 → 87.19
PSNR (dB ↑)26.10 → 26.9322.93 → 26.0120.65 → 22.8619.01 → 19.5925.67 → 26.3724.69 → 25.1922.65 → 22.94

Refinement improves both metrics on average in all seven conditions. As depth noise increases, the gains shrink, but the refiner’s rendering improvements are still accompanied by more accurate relative poses.

2. Backbone size and NVS quality at matched end-to-end latency (Weakness 2)

Thanks for suggesting a comparison of backbone scaling and camera refinement at matched inference latency. We already evaluate HiFiNVS with DA3-Giant (≈1.1B parameters, the largest DA3 backbone), π³ (≈1B), and VGGT (≈1B) as geometry backbones. Tables 3 and 8 (Appendix D.1) demonstrate the sensitivity of NVS quality to camera accuracy: even DA3-Giant's predicted cameras produce substantially lower NVS quality than the GT-camera reference. This motivates us to develop a camera refiner using rendering feedback, which consistently narrows this gap across all three backbones. These results support a separate camera-refinement stage for high-quality unposed NVS, even with large backbones.

To address your question about backbone size, we have started training the NVS head with DA3-L (≈0.35B parameters) and will train the camera refiner for this backbone. Changing the backbone simultaneously changes the features, depth estimates, and initial camera predictions used by HiFiNVS, so this experiment evaluates overall NVS quality and inference cost across different model sizes within the same backbone family. Whether the two systems can be compared at matched end-to-end latency also remains to be established. Training and evaluation on our two available H100 GPUs may take several days; we will provide an update in a follow-up comment.

For a quick unposed test on the DL3DV benchmark, we replace DA3-G's initial poses with DA3-L poses and apply the refiner, keeping DA3-G's cached features and depth.

SettingTime (s) ↓PSNR (dB) ↑AUC@5° (%) ↑
DA3-G alone0.36 + 0.14 + 0.00 = 0.50 s26.1085.14
DA3-L poses + refiner0.10 + 0.13 + 0.35 = 0.58 s25.57 → 26.6578.99 → 82.07

Time is backbone inference + first reconstruction + second reconstruction with refinement. Arrows show before → after refinement. The quick test shows that refining poses from a smaller backbone can yield better NVS quality at a similar runtime.

We hope these clarifications address your concerns about camera accuracy and refinement cost. We welcome any further questions.

Thank you for the response and the additional experiments, and for clarifying that the refiner is supervised by GT cameras and takes rendering feedback as input, rather than being trained with a rendering loss.

W1. The perturbation study addresses my concern.

W2 remains my main concern. The main issue bother me is: why train a separate refiner instead of scaling up the model? The refiner adds a second training stage, a render–evidence–rebuild loop at inference, and 45–73% more latency (Table 5). The same budget could instead go to a larger or stronger reconstruction model, and Table 3 already suggests the backbone can matter more than refinement (DA3 without refinement, 26.40 dB, vs. VGGT with refinement, 25.91 dB). The quick test does not answer this. It reports DA3-L latency but the quality of a pipeline that still uses DA3-G's features and depth, so its real cost (≈0.94 s) exceeds both DA3-G alone (0.50 s) and DA3-G with the refiner (≈0.85 s). To justify the added complexity, I would like to see that, at matched end-to-end latency, adding the refiner beats scaling up the model.

For these reasons, I maintain my rating.

AuthorsOpenReview ↗

Follow-up Response to Reviewer GqTK

Thank you for your follow-up and for noting that the perturbation study addresses W1.

For W2, we trained HiFiNVS with DA3-L and evaluated it on 140 DL3DV benchmark scenes at 480P with 16 input views. To further reduce inference time, we use refinerfast\mathrm{refiner}_{\mathrm{fast}}, which applies the refined cameras directly to the per-view Gaussians from the initial reconstruction, avoiding a second round of graph construction and Gaussian decoding. The table reports inference time, PSNR, and camera AUC@5° (computed using the same official DA3 benchmark evaluator as in Tables 3 and 8).

SettingTime (s) ↓PSNR (dB) ↑AUC@5° (%) ↑
DA3-L + refiner0.135 + 0.149 + 0.231 = 0.51525.60 → 26.5069.05 → 72.37
DA3-L + refinerfast\mathrm{refiner}_{\mathrm{fast}}0.135 + 0.149 + 0.133 = 0.41725.60 → 26.4369.05 → 72.37
DA3-G + refiner0.342 + 0.149 + 0.231 = 0.72226.40 → 27.3070.33 → 72.48

On an RTX 5090, total time is backbone inference + first reconstruction + second reconstruction with refinement. Arrows indicate before → after refinement.

DA3-G alone takes 0.491 s, while DA3-L takes 0.515 s with our refiner and 0.417 s with refinerfast\mathrm{refiner}_{\mathrm{fast}}. At similar end-to-end reconstruction latency, DA3-L (≈0.35B) with our refiner (≈18M) achieves comparable PSNR and higher camera AUC@5° relative to DA3-G alone (≈1.1B). This demonstrates the effectiveness of the independent camera refiner. We will include this comparison in the final manuscript.

Backbone scaling and camera refinement provide complementary improvements to unposed NVS. Scaling increases model capacity to improve camera and depth prediction, whereas refinement exploits rendering evidence to improve camera accuracy. The two approaches can be combined; they need not replace one another. Our gains across different visual geometry backbones support this complementary role (Tables 3 and 8). Once the largest publicly available DA3 backbone is already in use, further scaling it requires developing and training a larger model, with substantial computational and data requirements. In comparison, our refiner has only ≈18M parameters and requires no backbone retraining, providing a lightweight, effective way to improve unposed NVS with strong pretrained backbones. Adopting a large backbone therefore does not remove the value of camera refinement.

We hope these additional results and clarifications address your concerns, and we look forward to your feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.