acceptodds
Under review as a conference paper at ICLR 2027

PBE: A Physics-Guided Beta-Primitive Extension for Feed-Forward 3D Reconstruction

Abstract

Feed-forward Gaussian reconstruction predicts renderable 3D scenes in a single pass, but its Gaussian kernels and low-order spherical harmonics offer limited flexibility in spatial shape and view-dependent appearance. Existing Beta kernels and Spherical Beta (SB) functions offer greater flexibility through adjustable spatial decay and localized angular responses. However, predicting Beta/SB parameters in a feed-forward manner is challenging because geometry and appearance require different cues, and sparse observations do not fully constrain view-dependent appearance. We propose PBE, a Physics-guided Beta-primitive Extension that addresses this problem through input-dependent base construction and residual prediction. PBE constructs single-view Beta/SB bases from predicted depth and camera parameters, and multi-view bases from depth-consistent support-view observations. Attribute-specific gates combine these bases, while learned residuals correct inaccurate estimates. PBE integrates into the primitive-parameter prediction stage of existing reconstruction models while preserving their feature backbones and single-pass inference. Trained solely on DL3DV, it consistently improves rendering quality on Depth Anything 3, VGGT-, YoNoSplat, and AnySplat under matched epoch budgets. Notably, Depth Anything 3 on Mip-NeRF 360 improves from to dB after five epochs without additional fine-tuning. Ablations further show that physics-informed Beta bases outperform random bases under the same residual refinement. These results jointly validate the effectiveness of PBE.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Feed-Forward, 3D Reconstruction, Gaussion Splatting, Physics-Guided Neural Rendering, Spherical Beta Functions

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_xyiQOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PBE augments the prediction head of a frozen feed-forward reconstruction backbone with spatial Beta and three-lobe Spherical Beta parameters. It constructs heuristic geometry-based single-view bases from depth and normals, combines multi-view evidence with attribute-specific gates, and predicts learned residual corrections. The authors train the augmented heads on DL3DV and evaluate Depth Anything 3, VGGT-Ω, AnySplat and YoNoSplat on DL3DV, Mip-NeRF360 and Tanks and Temples. Table 1 reports PSNR gains across the tested backbone–dataset combinations.

Critical strengths

The approach operates at the primitive-parameter prediction stage, retaining frozen feature backbones and feed-forward inference. Demonstrating the extension across four architectures supports its potential practical reuse rather than dependence on one particular reconstruction backbone. Table 1 reports improved PSNR for all four architectures on all three benchmarks, often with the 50K-step extension exceeding a 100K-step original-head baseline. This is useful evidence of improved reconstruction performance under the tested settings.

Critical weaknesses
  1. The spatial Beta base relies on a hand-weighted combination of normalized depth gradients and a curvature proxy (Eq. 2), while SB directions and initial sharpness use fixed angle and scale choices. These may be useful inductive biases, but are not clearly derived from physical image formation. Table 2 shows just 0.23 dB over a random-prior variant, while the full comparison also changes kernels, appearance basis, prediction head and multi-view evidence.

  2. Table 3 shows very similar scores for Single and Full at a fixed checkpoint; Single still retains multi-view evidence in the residual branch, so this is not a clean independently trained fusion ablation. In Table 1, YoNoSplat gains PSNR but substantially worsens LPIPS on all three benchmarks, an important caveat for the claimed improvements in specular appearance.

Questions for authors

Q1: Would a Gaussian/SH head with a similarly sized residual prediction module and the same geometry/multi-view inputs reproduce much of the gain? Please separate benefits of richer Beta/SB expressiveness from extra head capacity.

Q2: Can independently retrained ablations isolate spatial Beta, SB appearance, geometric initial bases and multi-view base fusion? A frozen-checkpoint intervention does not determine how a different model would perform after retraining.

Q3: What causes the repeatable YoNoSplat LPIPS degradation despite improved PSNR? Also, the abstract states a DA3 Mip-NeRF360 baseline of 24.25 dB, but Table 1 lists 24.55 dB; are these different experimental settings or a reporting error?

2 comments

For W1: Thank you for this observation. As for these manual settings, the explicit basis provides input-dependent parameter references, enabling the network to learn corrections to the reference values. Setting an initial configuration facilitates parameter optimization. For the SB direction and initial sharpness, we use fixed angle and ratio choices: three directions at different orientations provide dispersed initial angular responses, and 2, 4, and 8 provide initial responses from wide to narrow. These are initialization choices, not constants derived from physical laws. Through the initial settings and the constraints of the loss, we expect the SB directions to cover the surface of the primitives rather than collapse entirely into a single direction. The random prior results indicate that the manual settings have little effect on the overall performance, although they may cause larger errors in training with only a few epochs. In Fig. 5 of Appendix B, we provide a loss comparison between PBE and the random prior; the PBE loss is lower than that of the random prior in every epoch, and the loss gap is very clear in the first 5 epochs, with a maximum gap of about 0.1. We provide a more detailed ablation in Table 4, which demonstrates that our PBE method is effective and that the multi-view evidence, kernels, and prediction head are all useful.

For W2 and Q3:Thank you for this suggestion. We provide a more detailed ablation in Table 4, which demonstrates the effectiveness of the multi-view evidence. We ablated the multi-view features from the features, and all metrics on the three datasets were lower than those of PBE. Regarding the LPIPS degradation issue, in the current Table 2 we further added comparative experiments on Re10K (68): YONOSPLAT's LPIPS did not degrade, whereas AnySplat's LPIPS degraded. The reason is that our loss consists of an L1 loss and a gradient loss; during training, the model may focus too much on pixel-wise error and lack supervision of some local structures. Second, we did not tune different parameters for different methods during training; different architectures may respond differently to the same training configuration, which can also lead to LPIPS degradation. The specular reflections we mention actually refer to cases in some scenes where sudden highlights may appear. Low-order SH coefficients are limited in representing local, sharp view-dependent angular responses, and our PBE method can compensate for this deficiency. For example, in the top-left image of Fig. 1, the PBE method recovers most of the black regions of the water-surface reflection, while the original DA3 is clearly worse than PBE. The abstract states that the DA3 Mip-NeRF360 baseline is 24.25 dB, whereas Table 1 lists 24.55 dB. This is because in the abstract we report the result after 5 epochs of training, which is 24.25 dB, while Table 1 is after 10 epochs, which is 24.55 dB.

For Q1:Thank you for this suggestion. In Table 4, on DA3 we used exactly the same residual and geometric construction as PBE, as well as a residual-only module; the module parameter count, geometric construction method, and multi-view components are exactly the same as PBE. We then trained them separately on DL3DV for 5 epochs. As shown in the table, the metrics on all three test sets drop substantially: PSNR drops by about 11 dB, and SSIM and LPIPS drop by about 0.3 and 0.5, respectively, compared with the baseline. This indicates that the additional residual prediction does not provide a gain; the main gain comes from Beta/SB. Moreover, in this part of the experiment, the Gaussian attributes with the added geometric construction achieved better results than residual optimization alone, further demonstrating the effectiveness of the geometric construction.

For Q2:Thank you for this suggestion. In Table 4, we further discuss the expressiveness of Beta/SB. Combining either the beta parameter alone or SB alone with Gaussian primitives both achieves better results than the original Gaussian. w/o spatial Beta improves clearly over the DA3 baseline: PSNR increases by about 1.51–1.83 dB, SSIM increases by about 0.0389–0.0608, and LPIPS decreases by about 0.0345–0.0774. w/o SB improves less over the DA3 baseline: PSNR increases by 0.26/0.24 dB on DL3DV/Mip-NeRF 360, but is 0.08 dB lower on Tanks and Temples; LPIPS is slightly higher by 0.0008 on DL3DV and slightly lower on the other two datasets. Only SB alone with the original Gaussian primitives outperforms PBE on the Temple dataset in terms of PSNR and LPIPS; in most cases, the PBE method is optimal. In addition, after removing the fusion of multi-view features (not the multi-view in the geometric basis), all metrics decreased somewhat: PSNR drops by 0.02–0.24, while SSIM and LPIPS change only slightly, showing an overall decline. Together with Table 5, this shows that multi-view is effective in both training and inference.

Thank you for your suggestions on our work. For W2, in Fig. 7 of Appendix C, we further present the rendering comparison of YoNoSplat on an example view (000001), and we zoom in on some regions that are difficult to render. The results demonstrate the improvements of our PBE method for these three algorithms. In terms of PSNR, PBE brings improvements of 0.3510.351dB for YoNoSplat, $1.776 dBforVGGT−dB for VGGT-\Omega$, and 0.6920.692 dB for AnySplat, respectively.

Reviewer_KaQYOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper presents PBE, a lightweight plugin module for feed-forward 3D Gaussian Splatting. The method replaces the Gaussian kernel with a spatial Beta kernel and three-lobe spherical Beta appearance functions. Additionally, instead of directly regressing SB parameters, PBE first builds explicit parameter bases analytically and then refines them with learned residuals, where single-view bases and multi-view bases come from separate strategies. PBE is tested on several frameworks, including AnySplat, YoNoSplat, DA3 with GS head, and all shows improvements of rendering quality under the same (if not smaller) compute budget.

Critical strengths
  1. PBE is broadly applicable, and works as a drop-in extension to multiple feed-forward networks. By adding only a few channels to be predicted, the method improves the rendering quality by a considerable margin.

  2. The idea of splitting the prediction into explicit, geometry-derived bases plus learned residuals is a sensible inductive bias for feed-forward prediction, where the problem is hard under sparse views.

  3. The method is described in details, adding possibility to fully reproduce the method.

Critical weaknesses
  1. The claimed "physics-guided" approach seems to be dominated/determined by a set of hand-crafted hyperparameters. Examples include: a fixed 45-degree polar angle, initial sharpness values (2, 4, 8) and color scale of 0.03, and the 0.7/0.3 weights in the spatial beta base etc. Are these heuristics able to be fixed for different scenes, or do they need additional tuning to reach better results on other datasets?

  2. It's not clearly measured how the method compares to already-established Gaussian-based rendering method. How the rendering FPS vary in accordance to different rendering resolutions and the number of bases?

  3. Results on view-dependent scenes are missing, even though the claimed benefit is mainly on specular and reflective appearance. Some video results would be helpful in demonstrating such use cases & benefits.

Questions for authors
  1. Are Table 1 metrics computed on held-out novel views or on the input views? How are the input(context) and target views selected?

  2. Why is there a discrepancy between "24.25 -> 26.01 dB" (abstract) vs the 24.55 baseline in Table 1?

  3. How sensitive is PBE to the hand-set constants?

3 comments

For W1: Thank you for highlighting this distinction. These hyperparameters are subsequently refined by residual optimization in different scenes to obtain scene-specific corrections. No fine-tuning on other datasets is required. All our experiments keep the same hyperparameters and do not adjust them across different datasets.

For W2: Thank you for highlighting this distinction. We tested the rendering speed using an RTX PRO 5000 Blackwell, repeating the timing 5 times after warm-up, excluding model prediction, compilation, and image saving. The final results are as follows: on DA3, the speed decreased from the original 180 fps to 167 fps; on YONOSPLAT, we achieved 175 fps, whereas the original YONOSPLAT only reached 132 fps; on VGGT-Ω, it decreased from the original 249 fps to 198 fps; and on AnySplat, it decreased from the original 263 fps to 241 fps. The rendering frame rate, like the baseline, is affected by the rendering resolution and the number of primitives.

For W3: Thank you for this suggestion. In the left panel of Fig. 1 in the main text, we provide comparisons of rendered images for some scenes. For example, in the upper part of the left panel of Fig. 1, the PBE method recovers most of the black regions of the water-surface reflection, while the original DA3 is clearly worse than PBE. In the lower part of the left panel, in the bottom-left region of DA3 and Ours, our PBE method recovers richer railing details, whereas the railing rendered by DA3 is clearly blurrier than that of our PBE. We have also added new comparison images in Fig. 6 of Appendix C. In Fig. 6, for the red-boxed region of the glass in the upper rendered image, our PBE method renders the glass structure better and also renders some of the reflective parts, whereas DA3 renders the wall of the glass more blurrily, and in particular does not render some of the reflective spots on the glass. In the lower part of Fig. 6, first, in the black-dot region in the blue box, our PBE method renders it completely, whereas DA3 does not render this part. Second, for the reflected light in the upper half of the red box, our PBE method recovers it more comprehensively compared with the GT, while DA3's recovery of this part is less complete than PBE's. In addition, in the bottom-right part of the red box, DA3's rendering is somewhat blurrier compared with PBE.

For Q1: Thank you for highlighting this distinction. The metrics in Table 1 are computed based on input views, and the view selection method is consistent with that in the DA3 paper. We use FPS sampling on the DL3DV, NeRF 360, and Temple test sets as well as the DL3DV training set, sampling 16 views per scene for evaluation. In addition, in Table 2 we further test on 68 scenes from the RE10K dataset. To make our comparative experiments more thorough, we use three different view counts: 8, 12, and 20. The final results further demonstrate the effectiveness of the PBE method, which achieves the best result in 28 out of 36 cases.

For Q2: Thank you for highlighting this distinction. The abstract states that the DA3 Mip-NeRF360 baseline is 24.25 dB, whereas Table 1 lists 24.55 dB. This is because in the abstract we report the result after 5 epochs of training, which is 24.25 dB, while Table 1 reports the result after 10 epochs, where the PSNR is 24.55 dB. Overall, the difference in experimental settings affected your understanding, for which we sincerely apologize.

For Q3: Thank you for this suggestion. We appreciate the concern about the sensitivity to manually set constants. These constants are used to construct an initial parameter basis that can be corrected by the learned residuals. The model is not completely insensitive to these constants: intervention experiments on a fixed checkpoint show that substantially changing the geometric basis degrades reconstruction quality, and this experiment reflects the trained model's dependence on the basis. In Fig. 5 of Appendix B, we provide a loss comparison between PBE and the random prior; the PBE loss is lower than that of the random prior in every epoch, and the loss gap is very clear in the first 5 epochs, with a maximum gap of about 0.1. Different constants have limited impact on overall multi-epoch training, but they have a larger impact on training with fewer epochs.

Thanks for the response. I will keep my original rating.

Thank you very much for your reply. I have added the qualitative results for the other three algorithms, which are presented in Fig. 7 of the appendix to demonstrate the rendering improvements of PBE across different algorithms.

Reviewer_da8nOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes to aid 3D Gaussian parameters with Beta/SB in feed-forward models, as the Gaussian kernel and low-order SH provide limited flexibility. The paper compares PBE with existing methods, such as AnySplat and YonoSplat, and with backbones like DepthAnything v3 and VGGT-Omega. The methods achieve state-of-the-art results on DL3DV, MipNerf-360 and the Tanks & Temples dataset.

Critical strengths
  • [S1] The paper is rightly motivated, and using Beta and Spherical Beta attributes instead makes sense and is logical.
  • [S2] The results demonstrate that using Beta/SB parameters helps with reconstruction.
  • [S3] The authors demonstrate the efficacy of their approach using different backbones: VGGT-Omega and DA3.
Critical weaknesses
  • [W1] A major weakness is that the paper omits recent and strong baselines such as VolSplat[R1], C3G[R2], GlobalSplat[R3], AdaptiveSplat[R4], EcoSplat[R5], NoPoSplat[R6], SHARP[R7], etc and uses older methods. The evaluation with recent methods will give a clearer advantage of PBE.
  • [W2] The authors did not evaluate on the RE10K dataset, which is commonly used in this area.
  • [W3] Its not clear wether input image I are used in PBE or not, Eqn 1 suggest input image is used but L142-143 suggests input images are not used, similarly Fig 2 suggests that Image I are not used in PBE. The manuscript can improve with clearer writing and presentation.
  • [W4] An obvious baseline would be to directly replace 3D Gaussian parameters with beta/SB parameters and train using how AnySplat and YonoSplat were trained, which is omitted currently. This will support the claim made in L35-36.
  • [W5] Evaluations seem to be limited, and there are no additional qualitative or quantitative results for different numbers of context views (sparse views, dense views like in AnySplat, NoPoSplat). Similarly, for single-view and multi-view bases, fusion gates are required.

Refernces

  • [R1] Wang, Weijie, et al. "Volsplat: Rethinking feed-forward 3d gaussian splatting with voxel-aligned prediction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026
  • [R2] An, Honggyu, et al. "C3G: Learning compact 3d representations with 2k Gaussians." arXiv preprint arXiv:2512.04021 (2025).
  • [R3] Itkin, Roni, et al. "GlobalSplat: Efficient feed-forward 3D Gaussian splatting via global scene tokens." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
  • [R4] Singhal, Badrinath, et al. "AdaptiveSplat: Texture Aware Controllable 3D Gaussian Allocation for Feed-Forward Reconstruction." European Conference on Computer Vision. Cham: Springer Nature Switzerland, 2026.
  • [R5] Park, Jongmin, et al. "Ecosplat: Efficiency-controllable feed-forward 3d gaussian splatting from multi-view images." arXiv preprint arXiv:2512.18692 (2025).
  • [R6] Ye, Botao, et al. "No pose, no problem: Surprisingly simple 3d gaussian splats from sparse unposed images." International Conference on Learning Representations. Vol. 2025. 2025.
  • [R7] Mescheder, Lars, et al. "Sharp monocular view synthesis in less than a second." International Conference on Learning Representations. Vol. 2026. 2026. (edited)
Questions for authors
  • [Q1] In Figure 2, the top row shows that RGB is not input to the PBE+GS head. However, in the second row, RGB is an input to the PBE+GS head. Please clarify this discrepancy.
  • [Q2] Can the authors provide more qualitative results and also show zoomed-in regions where we observe clear improvement over the baseline methods?
3 comments

For W1: Thank you for this suggestion. We apologize that, due to time and computational resource constraints, we only selected four methods: Depth Anything 3, VGGT-Ω\Omega, YoNoSplat, and AnySplat. However, among these four methods, we selected two released within the past year and two earlier methods, and among the four selected methods, two have small parameter counts and two have large parameter counts. In this way, our evaluation of the effectiveness of the PBE method is also comprehensive, as we can improve the reconstruction performance of both small-parameter and large-parameter models.

For W2: Thank you for this suggestion. We apologize that, due to time and computational resource constraints, we did not have enough time to complete evaluation on the entire RE10K dataset. We randomly selected 68 scenes from the RE10K test set for testing, and the results are shown in Table 2 of the main text. Our evaluation in Table 1 uses 16 views. To make the comparative experiments more thorough, we evaluated 8, 12, and 20 views on the selected 68 RE10K test scenes, extending the evaluation to different numbers of views. The directory of the specific 68 scenes is provided in Part D of the appendix. The results in Table 2 further demonstrate the effectiveness of the PBE method, which achieves the best result in 28 out of 36 cases.

For W3 and Q1: We apologize for the misunderstanding caused by our wording. The input RGB image does indeed participate in the PBE computation; the original text and figure did not clearly distinguish the visual features provided by the host network from the directly used RGB observations. Specifically, PBE uses the RGB values of the anchor pixels to construct the single-view SB color initial basis, and associates support views via camera poses, sampling their RGB observations to construct the multi-view color evidence and the SB appearance basis. Therefore, the input image I\mathcal{I} in the formula is a necessary input rather than a redundant symbol. We have revised Fig. 2 to include the introduction of RGB, and we have revised the text in L142–L143 so that the text, formula, and figure are consistent.

For W4: Thank you for highlighting this distinction. Directly replacing the 3D Gaussian parameters with beta/SB parameters is inconsistent with our method. Direct prediction would require longer training to achieve the same effect as our PBE. For example, when we improved DA3 using the original GS-head to directly predict beta/SB parameters, the loss reached 0.10 at 10 epochs, whereas the original DA3 had a loss of 0.063 at 10 epochs. We further added an FFT module on top of this, and it still only achieved a loss of 0.068 at 10 epochs.

For W5: Thank you for this suggestion. The metrics in Table 1 are computed based on input views, and the view selection method is consistent with that in the DA3 paper. We use FPS sampling on the DL3DV, NeRF 360, and Temple test sets as well as the DL3DV training set, sampling 16 views per scene for evaluation. In addition, in Table 2 we further test on 68 scenes from the RE10K dataset. To make our comparative experiments more thorough, we use three different view counts: 8, 12, and 20. The final results further demonstrate the effectiveness of the PBE method, which achieves the best result in 28 out of 36 cases. We use DA3 for the qualitative results because DA3 is better on most metrics, so we use this method as an example.

For Q2: Thank you for this suggestion. In the left panel of Fig. 1 in the main text, we compare the rendering results of some scenes. For example, in the upper part of the left panel, the PBE method can recover most of the door and window regions reflected on the water surface, while the original DA3 method is clearly worse than PBE. In the lower part of the left panel, especially in the bottom-left region of DA3 and the PBE method, the railing details recovered by the PBE method are richer, whereas the railing rendered by DA3 is clearly blurrier than that of PBE. In addition, we added new comparison images in Fig. 6 of Appendix C. In the red-boxed region of the glass in the upper rendered image of Fig. 6, the PBE method renders the glass structure better, and some reflective parts can also be rendered, whereas DA3 renders the wall of the glass more blurrily, and in particular does not render some of the reflective spots on the glass. In the lower part of Fig. 6, for the black-dot region in the blue box, the PBE method can render it completely, whereas DA3 does not render this part. Second, for the reflected light in the upper half of the red box, the PBE method recovers it more comprehensively compared with the GT, while DA3's recovery of this part is less complete than PBE's. In addition, in the bottom-right part of the red box, DA3's rendering result still has some blurriness compared with PBE.

I acknowledge the author's response. I will make final decision after discussing with other reviewers.

Thank you for the suggestion regarding additional qualitative comparisons. In addition to the original DA3 results, we further added rendering comparisons of YoNoSplat, VGGT-Ω\Omega, and AnySplat and their PBE variants on an example view (000001). The comparison figures are shown in Fig. 7 of the appendix to demonstrate the reconstruction performance of PBE under different architectures. On this view, PBE brings a PSNR improvement of 0.351dB0.351 dB for YoNoSplat, 1.776dB1.776 dB for VGGT-Ω\Omega, and 0.692dB0.692 dB for AnySplat. This is especially evident for the glass and metal basin regions. These newly added rendering results and the corresponding per-view metrics complement the average performance comparison in the main table, providing a more intuitive basis for evaluating the effect of PBE on different host models.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.