acceptodds
Withdrawn from ICLR 2027

From Particles to Fields: Reframing Photon Mapping with Continuous Gaussian Photon Fields

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Accurately modeling light transport is essential for realistic image synthesis. Photon mapping captures complex global illumination effects such as caustics and specular-diffuse interactions, but dense photon storage and repeated surface gathering remain expensive when rendering multiple views, even when the photon map is reused. To accelerate multi-view rendering, we reformulate surface-lighting estimation as a continuous and reusable field. We introduce the Gaussian Photon Field (GPF), a learnable representation of surface irradiance using anisotropic 3D Gaussian primitives, enabling efficient radiance evaluation across viewpoints. GPF is initialized with photon-guided surface coverage and optimized against sparse multi-view stochastic progressive photon mapping (SPPM) radiance references. Differentiable Gaussian interpolation allows the irradiance coefficients and their spatial support to be fitted jointly through the rendered image predictions. At rendering time, the field is combined with the terminal Lambertian response and explicitly traced camera-side transport, preserving view-dependent reflection and refraction without further photon tracing or progressive refinement. Experiments on scenes with caustics and specular-diffuse interactions demonstrate high-fidelity multi-view rendering with reduced per-view computation, connecting physically-based light transport with the efficiency of reusable learned representations.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Neural Rendering, Neural Scene Representation, Photon Mapping, Differentiable rendering, Neural Radiance Field

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_eMq8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes a Gaussian Photon Field to speed up physically based rendering. It uses anisotropic 3D Gaussians to represent irradiance on diffuse surfaces. The GPF is initialized from photon deposits and supplementary training-surface samples, and it optimizes full Gaussian parameters jointly. The GPF shows impressive rendering quality with only 8 or 16 SPP and demonstrates improvements on SPPM.

Critical strengths
  • The idea of the Gaussian Photon Field seems interesting and practical, enabling efficient rendering of scenes with complex global illumination.

  • The rendering quality and rendering efficiency are promising.

Critical weaknesses
  • The idea of caching spatially varying illumination has several existing works, like Neural Radiance Caching[1], InvRender[2], NeFII[3] and photon-based cache methods[4,5]. The difference and novelty among these related words requires further clarification.

  • Comparison with existing methods on caching global illumination is missing. Comparison with related methods, like Photon-Driven Neural Radiance Caching[4] or Photon‐driven Irradiance Cache[5], can help understand the advantage of the proposed method.

  • Ablation studies are missing for important design choices. The paper does not study the effect of the Gaussian initialization method: how much the photon deposition and supplementary camera rays help the optimization. And "Surface-aware Gaussian interpolation", as well as the loss terms in Eq. (8), have no ablation studies.

Reference:

  1. Müller, Thomas, et al. "Real-time neural radiance caching for path tracing." arXiv preprint arXiv:2106.12372 (2021).
  2. Zhang, Yuanqing, et al. "Modeling indirect illumination for inverse rendering." 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2022.
  3. Wu, Haoqian, et al. "Nefii: Inverse rendering for reflectance decomposition with near-field indirect illumination." 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). IEEE, 2023.
  4. Stamm, Julian C., Tom Kneiphof, and Reinhard Klein. "Photon-Driven Neural Radiance Caching." Proceedings of the 2026 Companion Proceedings of the Symposium on Interactive 3D Graphics and Games. 2026.
  5. Brouillat, Jonathan, Pascal Gautron, and Kadi Bouatouch. "Photon‐driven Irradiance Cache." Computer Graphics Forum. Vol. 27. No. 7. Oxford, UK: Blackwell Publishing Ltd, 2008.
Questions for authors
  • Questions regarding novelty, comparison, and ablation studies can be found in the weaknesses above.
  • I am confused about Table 2. What is the reason for including this unfair comparison, and what is the main message? A clarification is appreciated.
Reviewer_StzpOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This submission describes an approach for efficient physically based rendering of synthetic scenes. The central idea is to integrate a learnable Gaussian-based representation into the photon mapping algorithm. Specifically, instead of storing and gathering conventional photons, the approach uses a 3D Gaussian representation. The Gaussian parameters are optimized using a sparse set of rendered training views. The approach is shown to outperform several classical and modern renderers, as well as radiance field methods. Unfortunately, a crucial evaluation is missing, casting substantial doubt on the usefulness of the proposed approach.

Critical strengths

I appreciate the problem setting this paper addresses. Exploring synergies between physically based rendering and optimizable representations is an exciting research direction with significant potential for improving image quality and rendering speed. Scalable solutions could greatly impact the development of VR and video games, to name just a few applications.

Leaving aside the minor concerns outlined in the Weaknesses section, the paper generally does a good job of contextualizing and describing the research.

Critical weaknesses

Unfortunately, I have a major concern with this submission that prevents me from providing a positive recommendation:

The inputs to the proposed algorithm are 1) a synthetic scene with complete information about geometry, materials, and light sources, and 2) a sparse set of high-quality, multi-view renderings of that scene. As far as I understand, the baselines considered in this work use either input 1) through physically based rendering or input 2) through radiance field reconstruction. The authors acknowledge that the latter is not "an equal-input benchmark" (L417), but fail to acknowledge that the former is not either.

This lack of acknowledgment is unfortunate, but it also leads to another serious concern: The authors do not consider the most straightforward approach to utilizing both inputs. Given sparse renderings and complete scene information, in particular geometry, one could, in my view, simply optimize a lightmap, i.e., textures that directly store the appearance of the surface (potentially including view-dependent effects), from the rendered training images using differentiable path tracing. In this case, one would not need to trace any photons during the fitting stage, but could simply render the scene from the training views and optimize the surface texels to reproduce the desired appearance. In my view, this approach would perform very well on the scenes considered. As this baseline is neither mentioned nor compared against, I am concerned that several conclusions drawn in this paper regarding the benefits of combining physically based rendering and learnable representations in the propsed way are unsubstantiated. In particular, I cannot see a situation in which the proposed approach could achieve something that the baseline I just described could not.

A minor concern is that several central algorithmic details are only provided in the appendix. Section 4 therefore feels rather high-level and is not self-contained.

Minor comments:

  • L074: I do not see why the photon field is described as "compact" when it is represented using Gaussians.
  • L079: "Neural field paradigms" seems inappropriate here, as there is no neural component.
Questions for authors

I would hope the authors can address my main concern outlined in the Weaknesses section above.

Reviewer_c28yOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces a neural-field-inspired, 3DGS-like method for photon mapping, the so-called Gaussian Photon FIeld (GPF). The motivation comes from SPPM being time- and memory-intensive, so the authors propose to render a few sample images, cache the photon maps and fit a 3DGS-like representation to those, which can then be queried very efficiently (cheap lambertian query from camera + GPF eval) for novel views.

While it's a reasonably well motivated extension to SPPM, I feel there's no particularly strong fit for ICLR. The paper presents a technique for more efficient ammortization in a relatively nieche sub-field of computer graphics (PM). That alone is not a reason for a low score, but it shoud be mentioned regardless.

Critical strengths
  • I like the paper presentation. The writing flows nicely, sections are well-linked, the reasoning chain makes sense, and the general motivation and paper flow is easy to follow along - for a reader familiar with the field, at least. For unfamiliar readers, I do appreciate that the paper includes Sec 3 (preliminaries) for self-containment.
  • The extension from discrete Photon maps and KDE estimates to a neural field or Gaussian primitives is well motivated and fills a natural gap (modulo my comments on usefulness, see below).
  • The method performs best across all baselines and scenes, although some are admittedly unfit for the task (see below). The supplemental video shows nice multiview-consistency.
Critical weaknesses

Major:

  • While the method does work, I am not convinced that it is actually useful. The massive preparation times (Sec 5, in the order of 1.5-3 hrs per scene) will only pay off for many novel views, as the authors correctly mention in the ammortization discussion in section five. I can't help but wonder when such an ammortization will actually occur - it would need to be a case where we have to render many (synthetic!) caustic images, and I can't think of one that's remotely real-world applicable. Alternatively, in cases where we'd need efficient real-time caustic rendering such as games, this would simply be approximated by feeding a precomputed caustic map to the rasterizer.
  • Related to the above, my 2nd concern is evaluation scope, and it's twofold: first, the evaluated scenes are quite few (only five). Second, and more importantly, this technique only works for synthetic scenes, since we need reference geometry + cameras + material information for the initial SPPM gathering. The authors do not mention a way of extending this to real-world captures, and I can not see a straightforward way to do this either. This is fruther aggravated by the fact that, once trained, geometry + lighting are fixed, so for every small scene change, one would have to re-train the entire GPF.
  • Building on the above point: given that we can only use GPFs for synthetic scenes, there is an arguably unbiased solution: a simple path trace of the scene with a large number of samples. While path tracing might be costly, SPPM is only consistent, not unbiased. Any technique approximating SPPM, including the presented GPF, therefore is also biased. This bias, and its potential attributions (e.g., how much comes from SPPM, from the limited amount of Gaussians, from the limited amount of fitting iterations, etc) is unfortunately not evaluated in the paper. I would turn a blind eye here if it was real scenes since we do not yet have an analytic way of achieving perfect resonstructions for real images, but since GPF is only operating on synthetic scenes where an unbiased solution exists, this should be evaluated.
  • Finally, the eval that exists is not equal-time: it compares SPPM at 3 iterations, and path tracin at 10-50 spp, while GPF is learned from the full 2000-iter SPPM renders. I get that the method's selling point is ammortization, but these settings seem random regardless, and don't provide much insight.

Minor:

  • The evaluation set of 5 scenes is quite small.
  • Renderformer is not trained to replicate advanced effects like caustics at a global scale, which the authors adequately mention (L. 376 onwards). Its inclusion as baseline here is therefore not really adequate.
  • Similarly, the comparisons in Fig. 5 are a bit cosmetic: NeRF and 3DGS are not meant to be evaluted from 3 views, especially not in the presence of complex lighting effects like caustics. The comparison is skewed in another way too, since those are baselines learned from images, without camera+geometry+lighting info. From a storytelling perspective, I get why that figure exists, yet, it's placed a little too prominently for my liking, given its significance and the actual information it conveys.
  • Memory numbers in Fig.1 are a bit strange, the path tracer uses 67/61 GB of GPU memory, that seems excessive?

Writing:

  • Related work is missing an entire section on Neural Radiance Caching (e.g., Mueller), irradiance caching and lightmap baking, which seem very relevant. Should at least be cited, ideally, compared to.
Questions for authors

See above. Please try to address my major criticisms, at least. Specifically, I'm yet to be convinced of the real-world impact of this method, or its actual usefulness / applicability.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.