acceptodds
Under review as a conference paper at ICLR 2027

SceneFoam: Sparse-View Reconstruction of Native Radiance Foams

Abstract

SceneFoam learns to reconstruct native Power Foam fields from sparse calibrated images, recovering bounded cells, internal surfaces, local texture, and view-dependent color. Its central design aligns appearance updates with the geometry that determines visibility and pixel-to-texture contributions. An image-conditioned frontend and native initializer are trained jointly to predict the scene. Four source-conditioned color-fitting steps then align directional radiance with the input photographs and independent reference images. A geometry-first refiner updates the surfaces, recomputes their image residuals and source-loss gradients, and predicts appearance using this refreshed evidence. Together, these stages couple a learned scene prior with observations evaluated on the current foam. On eight-view DL3DV140, SceneFoam obtains 27.97dB PSNR, 0.873 SSIM, and 0.153 LPIPS, outperforming ReSplat on all three metrics. On four native-PF comparison scenes, it improves all three metrics over both 30,000-update optimization baselines with complete inference in 8.94–9.44s. The recovered field directly supports rasterization, ray tracing, distorted cameras, and interaction through its tetrahedral dual. SceneFoam brings high-quality sparse-view acquisition to textured spatial partitions through geometry-consistent evidence updates.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
3D scene reconstruction, sparse-view reconstruction, neural rendering

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_UkE5OpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors propose using network-provided update directions for radiance foam optimization (with respect to sparse conditioning views) rather than direct gradient descent on the parameters. They show that these are much more efficient (~10s to beat ~60 minutes of direct optimization). Among others, they discuss creating artificial reference images (through gaussian splatting reconstruction), as well as interleaved appearance-geometry optimization.

Critical strengths
  • the approach makes intuitive sense to me: the gradients from radiance foam differentiable rendering are too chaotic for efficient gradient descent optimization, but scattering local error information through the foam allows a NN to learn effective update steps anyway.
  • the authors show improvements both on 8-view and 2-view sparse reconstruction, as well as generalization capability across datasets.
  • the most crucial ablation studies are present
Critical weaknesses
  • none of the improvements are motivated are discussed; they are all presented as "we randomly tried this and luckily it worked better than the baseline". Ditto with the baseline, and much of the math exposition: only someone already familiar with the PowerFoam optimization approach would be able to follow it.
  • existing work discussion is too succinct and impossible to follow for people not intimately familiar and its history with the field already. Radiance foam solutions are not even discussed in the first two paragraphs, nor is a clear discussion of its benefits/drawbacks given at any point.
  • the selection of 7 midpoint cameras for the pseudo imagery + 14 more (virtual) intermediate centerpoints (L226 and onwards) is not motivated at all and feels like optimizing for line-based sparse view captures (whose datasets often only evaluate on the full trajectory). A more holistic approach that would consider e.g. circular trajectories (where pseudo views within that circle would be more informative) is more appropriate.
  • the last section about rendering flexibility of radiance foam, while correct, is implicitly misrepresented as a benefit of this work, when it applies to all radiance foam / power foam methods.
Questions for authors
  • I have significant concerns about the use of AI for writing this paper: in many spots, counter-intuitive wording is used, or field-established terminology is actively avoided. I also spotted several instances of "AI filler" sentences, which add nothing to the local text except confusing the reader. They are too cryptic or actively irrelevant to that section of the paper --> independent references (L076, later as well): implies these are reference images, which they are NOT. These are artificially/heuristically created interpolated view positions, i.e. "pseudo" views. --> L127: "The distinction matters ..." that is true, but why does "a structured subproblem" give us any benefit whatsoever? --> L143: "The panels are analytical schematics." a pointless sentence. --> L150: "An oriented surface separates a side of density sigma from an empty side." Again true, but not revisited anywhere in the manuscript, nor is it relevant to the current paragraph. --> L247: the distinction between "image residuals" and "gradients of the source-image loss" is nonexistent and there is no point in trying to make one --> L284: "development scenes" = validation scenes, "confirmation scenes" = testing scenes

  • General (smaller) manuscript comments, which unfortunately do add up: --> L080: Transfer(s) --> L082: mention the (very positive!) comparison against ~1 hour optimization for PF. --> L118: referring to SceneFoam (the current manuscript) as "their" is a typographical error --> Fig2: unclear how the notations here map onto (g, a, C). Should only occur after the first mention in the text. --> In general, the way to select directions for directional color bases is too opaque and not reproducible from this manuscript on its own. --> How to select the texture site locations (L151) is not explained at all. How to map a sphere-topology onto a material patch is not obvious at all --> L158: where does the offset come from? How would any aspect of the appearance be independent of the color selection? This implies e.g. we could never make the scene fully dark? Eqn2 is further missing any directional dependency, when further on the manuscript discusses that each direction has a color attached. --> L202: I found the discussion around finetuning ReSplat for a now output modality confusing; I suggest formulating it as 'reinterpreting the ReSplat output as radiant foam parameters and finetuning from there' rather than talking about intermediate gaussian outputs --> L205: Unclear how P_seed fits into Fig 3. --> L208-209 was a bit too unclear, given that it reads like it applies only to the colouring optimization (which per Eqn2 and Sec4.2 is a "structured subproblem" which is easily solved) --> L221: observed colors instead of predicted colors? --> L241: is the decision whether or not to take an update step a local decision (cell-per-cell) or global? --> L250: why normalize the gradients? There is no motivation given, nor is this ablated --> L311: it is not clear to me whether RealEstate has a dedicated model trained for it, nor which model exactly is used for ACID. --> Sec 5.3: not clear from reading (only from number comparisons) on which datasets these ablation studies happened --> Sec 5.3 "color fitting before refinement": I'd also have much preferred to see the post-refinement impact of omitting the color fitting before refinement

2 comments

Dear Reviewer UkE5,

Thank you for your suggestions on camera selection and presentation. Following your feedback, we tested a coverage-based camera strategy, which improved PSNR by 0.22 dB on the eight partial-orbit scenes (Table U2b). We also outlined how we’ll clarify the Power Foam background and pseudo-view terminology. If you have a chance to look at our response, we’d welcome any feedback on these points.

Thank you for these additional details. I will be retaining my positive score.

Reviewer_vS9iOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

SceneFoam reconstructs native Power Foam fields from sparse calibrated images through learned initialization, color fitting with source images and frozen ReSplat-generated references, and geometry-first refinement. Its main contribution is a learned acquisition pipeline that refreshes rendering residuals and gradients after geometry updates to guide appearance prediction. Experiments on DL3DV140, RealEstate10K, and transfer to ACID report improved image quality over ReSplat. Ablations examine fitting, directional bases, reference sources, and refinement order. A four-scene comparison reports better quality and faster acquisition than native PF optimization, while demonstrations show reuse for multiple rendering modes and deformation.

Critical strengths
  1. The geometry-first refinement has a clear methodological motivation: updating cell geometry changes visibility and pixel-to-texture contributions, so appearance updates should use evidence recomputed on the updated geometry. Table 3B supports this design by comparing simultaneous updates, sequential updates with retained evidence, and sequential updates with refreshed evidence under matched training budgets. Refreshing evidence improves PSNR from 27.57 to 27.97 dB and LPIPS from 0.202 to 0.153 over the sequential variant without refresh. This is the paper’s strongest evidence for a meaningful methodological contribution beyond simply adapting an existing reconstruction network to a new representation.

  2. Learning to acquire native Power Foam fields from sparse images is a useful contribution because the resulting asset can directly reuse the representation’s rasterization, ray-tracing, and interaction interfaces. On the four reported native-PF comparison scenes, SceneFoam improves all three image-quality metrics over optimization with learned initialization and virtual references, while reducing acquisition time from tens of minutes to approximately nine seconds. Although the comparison has limited coverage, it demonstrates a promising practical benefit for making this representation easier to acquire.

Critical weaknesses
  1. The secondary-ray demonstrations do not establish that the reconstructed directional appearance remains accurate under reflection and refraction. Eq. (10) evaluates appearance using the direction from the current ray-segment origin to each texture site, rather than the actual propagation direction at the query point. Consequently, changing the segment origin can change the evaluated color even when the query point and propagation direction remain fixed. This approximation is particularly relevant to short reflected or refracted segments. Appendix H.4 provides an error bound, but it requires the site-to-query separation to be smaller than the segment length and the directional denominator floor to remain inactive. The experiments do not report whether these conditions hold or how large the resulting appearance error is. Agreement between two backends using the same approximation cannot resolve this concern.

  2. The paper evaluates reconstruction primarily through RGB image metrics, leaving the accuracy of the recovered surfaces unclear. Improvements in PSNR, SSIM, and LPIPS do not necessarily imply better geometry, since appearance can compensate for geometric errors. Given the emphasis on recovering internal surfaces and supporting interaction, a quantitative depth or normal evaluation on a suitable dataset would help establish the geometric quality of the reconstructed foams.

Questions for authors
  1. Could you clarify the discrepancy between Figure 6 and the corresponding tables? At four fitting steps, the figure shows post-fitting LPIPS of approximately 0.187 and post-refinement LPIPS of approximately 0.172, whereas Tables 3A and 3B report 0.211 and 0.153. Do these results use different configurations, checkpoints, or rendering settings, despite the stated shared evaluation protocol? Which values correspond to the submitted model?
  2. Do you have depth or normal evaluations supporting the accuracy of the recovered surfaces? If not, what evidence supports geometric fidelity beyond RGB reconstruction quality?
  3. In Table 3B, were the refinement variants matched in the number of rendering and gradient-evaluation passes, in addition to training budget? What is the inference time of each variant?
Reviewer_PsVVOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

SceneFoam reconstructs native Power Foam fields from sparse calibrated images through joint prediction, source-conditioned radiance fitting, and geometry-first refinement. Its key design refreshes texture-site residuals and gradients after geometry changes, aligning appearance updates with the current scene. The acquired field supports native rendering and interaction.

Critical strengths
  1. The paper presents a learned framework for reconstructing native Power Foam representations from sparse views, significantly reducing the per-scene optimization time required by the original Power Foam method.

  2. The geometry-first refinement with refreshed appearance evidence is technically well-motivated, and the ablation studies demonstrate its effectiveness.

  3. SceneFoam achieves consistent improvements over ReSplat across multiple benchmarks.

  4. The reconstructed Power Foam representation supports multiple downstream applications, including rasterization, ray tracing, and physical interaction, demonstrating its potential versatility beyond novel-view synthesis.

Critical weaknesses
  1. Regarding of motivation and what are the adavantages over alternatives: If the primary goal is feed-forward reconstruction of ray-traceable 3D scenes, a more straightforward approach would be to combine ReSplat with 3DGUT/3DGRT, which operate on the same Gaussian representation and already support secondary-ray rendering. The paper does not demonstrate clear advantages over this simpler alternative. Given its substantial dependence on ReSplat, the necessity of introducing a more complex Power Foam representation remains insufficiently justified. A direct feed-forward Power Foam predictor independent of ReSplat, or convincing advantages over ReSplat + 3DGUT/3DGRT, would provide a stronger motivation.

  2. SceneFoam achieves only a marginal improvement over ReSplat, while increasing reconstruction time by approximately 11×. Although Power Foam offers faster ray tracing than Gaussian-based alternatives, the substantial additional reconstruction overhead undermines this advantage.

  3. SceneFoam relies on an additional frozen ReSplat model to generate virtual-view references. Removing these references reduces PSNR from 27.97 to 27.77 dB, leaving only a 0.07 dB improvement over ReSplat. This raises concerns about how much of the improvement comes from the proposed reconstruction framework rather than additional supervision.

Also, SceneFoam is not a purely feedfoward predictor: It is Learned Prediction + Per-scene Numerical Fitting (color fitting, with supervision from resplat's result) + Learned refinement. It's unfair to compare Scenefoam with feed-forward baselines directly.

  1. The geometry-first evidence refresh improves reconstruction quality, but the paper does not compare its learned appearance refinement against simpler alternatives, such as directly repeating the existing color-fitting procedure after updating geometry. Consequently, the necessity of the additional learned refinement mechanism remains unclear.

  2. Ray tracing and physical interaction are presented as important advantages, yet these capabilities largely originate from the existing Power Foam representation. Moreover, geometric accuracy is not quantitatively evaluated.

Minor:

  1. Writing and Contribution Clarity. SceneFoam does not propose a new 3D representation, yet substantial portions of the paper describe Power Foam and its inherent benefits, including ray tracing and physical simulation. These are capabilities inherited from prior work rather than contributions of SceneFoam. The presentation should more clearly distinguish the proposed reconstruction method from the existing representation and devote greater attention to the actual methodological contributions.
Questions for authors
  1. What advantages does SceneFoam offer over the simpler ReSplat + 3DGUT/3DGRT pipeline in terms of reconstruction quality, ray-tracing efficiency, and computational cost?

  2. Could the authors provide a runtime breakdown of the 9.39-second reconstruction, particularly for reference generation, color fitting, and refinement?

  3. Would SceneFoam's improvements persist under supervision- and compute-matched comparisons with ReSplat?

  4. How does the learned appearance refinement compare with simply rerunning numerical color fitting or simple appearance online optimization after geometry updates?

  5. What quantitative evidence demonstrates the geometric accuracy and downstream benefits of SceneFoam beyond the capabilities inherited from Power Foam?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.