acceptodds
Under review as a conference paper at ICLR 2027

LeDenoiser: Exploring Latent World Models for Monte Carlo Rendering Denoising

Abstract

Monte Carlo rendering converges as samples accumulate, yet deep learning-based denoising rarely models that process. The field has largely settled on image-to-image regression: spatial 2D filtering over rigid multi-channel inputs, supervised only on output radiance. We ask whether light transport convergence can instead be formulated as an evolving physical state transition. Inspired by latent world models, which learn action-conditioned dynamics entirely within a regularized representation space, we propose *LeDenoiser*, a latent world model for Monte Carlo denoising. A noisy 1-SPP observation forms the initial state, encoded by a pre-trained vision transformer; auxiliary Arbitrary Output Variables (AOVs) act as discrete token-level action operators; and the predictor learns an action-conditioned transition that advances this state toward convergence. On SPP-World, our 22-scene multi-SPP path-tracing benchmark, unconditional latent dynamics lift severe 1-SPP noise (17.75 dB) to 32.61 dB, demonstrating that a latent world model can capture scene-level light transport without treating auxiliary geometry as a rigid dependency. Because AOVs form a discrete action space, that space can also be planned over. To our knowledge, this provides the first planning study inside a rendering world model: Model Predictive Control via the Cross-Entropy Method discovers sparse action subsets ( of 8 buffers) that outperform exhaustive all-buffer conditioning. The latent dynamics are controllable, and geometric redundancy varies non-trivially across scenes. These findings establish latent world models as a viable alternative to image regression in Monte Carlo rendering, bridging graphics pipelines with physical foundation models.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
world models, representation learning, model predictive control, Monte Carlo denoising, Monte Carlo rendering

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_m4vxOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper treats Monte Carlo denoising as a latent state transition, in the style of JEPA/LeWM.

  • A frozen DINOv2 ViT-B/14 encodes a noisy render into 24×24 patch tokens.
  • A 6-block transformer predicts the tokens of a 2048-SPP target. Each AOV (8 types) is encoded by the same backbone and injected via cross-attention as a "discrete action," so any subset can be supplied at test time.
  • A 111.9M-parameter DPT-style decoder maps the tokens back to RGB. Training uses token MSE, SIGReg, L1, and LPIPS.
  • A CEM planner searches over AOV subsets using a reference-free fixed-point residual.

On SPP-WORLD (22 Bitterli scenes in Mitsuba 3 at 336×336), with one model fitted per scene, it reports 32.61 dB at 1 SPP without AOVs, versus 21.54 dB for OIDN and 22.49 dB for OptiX. CEM selects about 1.6 of 8 AOVs and beats all-AOV conditioning by 0.14 dB within a 5-step rollout.

Critical strengths

An interesting formulation for MC denoising. The paper explores action-conditioned prediction in a frozen foundation-model latent space, drawing on JEPA and LeWM. It treats AOVs as independently selectable conditioning tokens rather than fixed input channels, and searches over their activation at test time using a reference-free latent self-consistency objective. This offers a different architectural and conceptual perspective on MC denoising and connects the denoising literature to recent self-supervised world-model work, although the benefits of this formulation remain insufficiently established.

Critical weaknesses

The evaluation protocol does not demonstrate generalization.

  • One predictor–decoder pair is fitted per scene, with every training pair sharing the same 2048-SPP target; no viewpoint variation or held-out views are reported.
  • Testing uses an independently sampled 4096-SPP reference, but no held-out scene content is reported.
  • With a 111.9M-parameter decoder, learned positional embeddings, and only 11 reported input SPP levels per scene, the model may simply memorize the shared training target rather than learn to denoise. The reported experiments do not rule out this possibility.
  • Adding all 8 AOVs changes PSNR by only 0.05 dB, which is consistent with this explanation, though not diagnostic. Separately, producing the training target already requires rendering a high-SPP image of the scene, raising a practical cost concern.
  • So the roughly 10 dB margin over general-purpose OIDN and OptiX, which get no scene-specific supervision, does not establish a better general-purpose denoiser. No held-out views or scenes, target-copy control, or matched per-scene baseline is reported.

The world-model contribution lacks empirical support.

  • The reported predictor is not conditioned on SPP or Δt; it always targets 2048 SPP. Repeated rollout feeds predicted latents back into a model trained on encoded rendered inputs, without reported rollout training. This introduces an unvalidated distribution shift rather than demonstrating learned convergence dynamics.
  • CEM planning requires about 45k candidate-level predictor evaluations for rollouts, plus approximately 9k for scoring, and reaches 30.30 dB. That is 2.36 dB below the single-step model (32.66 dB) and 0.09 dB above random search (30.21 dB), with no variability reported.
  • Appendix B shows that a running mean equals a batch mean. That does not establish that the mean is a sufficient statistic for estimating converged radiance, and it does not show that DINO features preserve the relevant information.
  • SIGReg's role is unclear. The encoder is frozen and the targets are fixed DINO tokens, so joint collapse through changes to the target representation is prevented, while feature regression already penalizes inaccurate predictor outputs. The paper does not establish what additional failure SIGReg prevents or whether it improves reconstruction. No ablation is given.
Questions for authors

Clarifications and experiments that could change my assessment:

  1. Is the 1-SPP evaluation input the same render used in training, or independently sampled?

  2. Generalization controls. What results do you obtain from: (a) Using the 2048-SPP training target itself as the output, without running the model, what PSNR/SSIM/FLIP does it achieve against the 4096-SPP evaluation reference? (b) With AOVs disabled, what happens if the input image is replaced by a uniform image or random noise? Successful scene reconstruction would suggest that the model can reproduce the training target without relying on scene information in the input. (c) a small pixel-space U-Net trained with the same per-scene supervision; (d) held-out views or scenes, without retraining?

  3. In what space are the losses and PSNR/SSIM/FLIP computed, and which FLIP mode is used? How is HDR radiance mapped into, and recovered from, the sigmoid output?

  4. Which OIDN and OptiX versions and settings were used, including cleanAux and the albedo/normal definitions at specular and glass hits? Please show baseline outputs in Figs. 3–4. Some OIDN SSIMs are unusually low (0.17 on kitchen, 0.06 on veach-ajar), yet OIDN exceeds your SSIM on dragon, lego, and rover.

  5. What quality does Dec(Enc(I_ref)) achieve, and do the missing highlight (veach-mis) and glass sphere (veach-bidir) persist? This would separate encoder–decoder limits from latent-prediction errors.

  6. What are the per-scene training time, the rendering cost of the training data, and the end-to-end inference latency with and without planning?

  7. How were the 22 scenes selected from the Bitterli collection?

Would also strengthen the paper: SIGReg and latent-loss ablations; comparisons with learned denoisers (e.g., NPPD, KPCN) under matched supervision.

Future work (not required for my assessment): cross-scene training with evaluation on Noisebase; a radiance-skip or kernel-prediction readout to preserve fine detail; an interactive A/B viewer with full-resolution outputs.

6 comments

Thanks for you reviews, which is quite helpful to make our manuscript better. Here are the discussion and responses for the questions:

  1. 1-spp evaluation is sampled independently with a different seed.
  2. We conducted the requested generalization controls across all 22 scenes against the 4096-SPP evaluation reference (in the tone-mapped SDR/LDR space):
  • (a) directly using the 2048-SPP training target as output yields 43.903 ± 7.085 dB PSNR, 0.9456 ± 0.0896 SSIM, and 0.0291 ± 0.0171 FLIP;
  • (b) with AOVs disabled (k=0), replacing the input with a uniform gray image (0.5) degrades performance to 16.220 ± 2.688 dB PSNR / 0.5856 ± 0.1180 SSIM / 0.4022 ± 0.0925 FLIP, and replacing it with random noise yields 16.433 ± 2.593 dB PSNR / 0.5914 ± 0.1164 SSIM / 0.4003 ± 0.0971 FLIP;
  • (c) Due to time constraints, we did not train the 22 per-scene pixel-space new U-Net baselines for (c).
  • (d) evaluating single-scene checkpoints on held-out scenes (spp_1, k=0) without retraining drops to 12.475 ± 2.880 dB PSNR / 0.4680 ± 0.1978 SSIM / 0.6023 ± 0.1179 FLIP. These results confirm that the model does not simply memorize or hallucinate the training target without input conditioning—it strictly relies on scene-specific information in the input—while also demonstrating that per-scene checkpoints do not generalize to unseen scenes without retraining. (a), (b), and (d) already directly address the core memorization and generalization concerns. Specifically, these controls confirm that the model does not simply memorize or hallucinate the training target without input conditioning—it strictly relies on scene-specific information in the input—while also demonstrating that per-scene checkpoints do not generalize to unseen scenes without retraining.
  1. We compute the all metrics in image space. And we mapped the color space from HDR, exr format, into a tone-mapped SDR / LDR space for the ground truth checking, Reinhard tone mapping with Gamma 2.2 correction.
  2. You provided a good check the decoder Dec(Enc(I_ref)) is still failed to reconstruction these two details, although PSNR: 34.9084 dB SSIM: 0.9513, and FLIP: 0.0560. The decoder model training seems not sufficient to reach a pixel-level accuracy.
  3. It spends 10-20min to train a fully end-to-end LeDenoiser at 16-bit floating points accuracy. To plan, it speeds in about 5 min per scene.
  4. We choose almost all scenes in Mitsuba 3 format from the official website.

Thank you so much for your patience and thoughtful review. We truly appreciate the constructive concerns, evaluation suggestions, and additional experiments you proposed—they are genuinely helpful for improving our work and have encouraged us to keep pushing this direction forward. We still believe exploring LeDenoiser has promising value, though we also recognize that many details currently have room for improvement and that we need to better demonstrate what makes the architecture and approach compelling.

Since we value your perspective, we would love to ask a few questions to learn from your insights:

  1. In your view, what would the "ideal" version of our method look like? What key capabilities should it have, and what distinct advantages over existing baselines would make it most convincing?
  2. Our current architecture uses a full-Transformer encoder/decoder with conditional action attention, yet we still struggle with losing fine details and highlights (such as in veach-mis). Do you have any architectural or methodological recommendations that could help address this?

Thank you again for your time and guidance!

Thank you for the response. Unfortunately it does not resolve my main concerns, and the clarification on the data split makes my soundness concern stronger, so I will keep my score. (1) Each scene is trained for only 700 steps on a single RTX 4090, so a per-scene U-Net baseline should be cheap to run. Without one, comparing against generic OIDN/OptiX models, which never see clean images of the test view, is not fair. (2) Saying that removing the latent loss "turns the model into a standard pixel-reconstruction network" is exactly what needs testing. If that network performs comparably, the world-model formulation is not what drives the results. For SIGReg, the encoder here is frozen and the targets are fixed, so the collapse argument from LeWM does not carry over (Sec. 3.1 itself says the frozen encoder prevents collapse). The concern about other pretrained encoders, or training from scratch, was not addressed. (3) Training and test share the same view, and each scene has only ~28 training pairs, all targeting a 2048-SPP render of that view. So the claim that "the 4096-SPP reference never appears in training" means little: the 2048-SPP target is nearly identical to it, and simply outputting that target would likely beat 32.61 dB by a wide margin. For a fixed view the AOVs are also essentially constant across training samples. This explains why K=0 ≈ K=8 and makes the +0.14 dB planning gain hard to interpret, especially since the 5-step rollout (30.30 dB) is worse than a single step (32.66 dB). The pairing scheme described in the response also differs from Appendix A. (4) The official DINOv2 positional embeddings are for 518×518, so 336×336 does not uniquely "match" the backbone, and tiled inference is standard for high-resolution denoising. No new results were provided.

Reviewer_ruyeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper introduces LeDenoiser, which formulates Monte Carlo rendering denoising as conditional state prediction in latent space. The method uses a frozen DINOv2 encoder to extract image features, a Transformer to predict the features of high-sample-count images, and a decoder to reconstruct the final images. The authors train and evaluate their method on a per-scene basis using SPP-WORLD, a benchmark comprising 22 scenes. The experiment results are satisfactory.

Critical strengths
  1. Leveraging pretrained visual features is a promising direction for Monte Carlo rendering denoising. Under the pape per-scene training setup, the model is able to recover reasonably good overall image structure from heavily corrupted 1-SPP inputs. The variant without AOVs achieves an average PSNR of 32.61 dB across 22 scenes (Table 1), with corresponding qualitative results shown in Figure 3.
  2. The paper presents a relatively straightforward idea that is clear and easy to follow.
Critical weaknesses
  1. I don't quite see how this paper relates to world models. Using DINOv2 as a feature extractor is already a common practice across various computer vision tasks. While Monte Carlo rendering denoising could certainly benefit from pretrained visual features, I don't think that necessarily makes it a world model. Without a conventional denoising baseline trained under the same per-scene setup, it is unclear whether the gains come from dynamics modeling or simply scene-specific fitting.
  2. The contribution of pretrained representations has not been sufficiently validated. The method relies on a frozen DINOv2 encoder, but the paper does not provide comparisons with other pretrained encoders or a baseline trained from scratch. Appendix C and Table 3 only analyze the cosine similarity between different DINOv2 features, without evaluating their impact on the final denoising performance. As a result, it remains unclear to what extent the method's performance depends on DINOv2's pretrained representations and whether the approach can generalize to other encoders.
Questions for authors
  1. As mentioned in the weaknesses, I would like to see more evidence explaining where the performance gains come from. Could the authors provide a comparison with a standard denoising network trained under the same per-scene setup, as well as ablation studies removing the latent-space prediction loss and SIGReg, respectively?
  2. Could the authors clarify how the dataset is split? Do the training and test sets share the same camera configurations, lighting conditions, and material settings? How much training data is available for each scene?
  3. Could the authors also provide results at higher resolutions? Currently, all experiments are conducted at relatively low resolutions. Although Appendix E acknowledges this limitation, in practice, Monte Carlo rendering denoising is rarely trained or applied at resolutions as low as 336 × 336. Evaluating the method at more realistic resolutions would help demonstrate its practical applicability.
2 comments

Thank you very much for your careful review and constructive feedback. Your questions point directly to the key design choices and limitations of our current manuscript. We genuinely appreciate your review. Here are some response to your questions.

  1. Sources of performance gains and ablations
  • Due to time and compute constraints during the rebuttal period, we could not train a separate pixel-space denoising baseline across all 22 scenes under the same per-scene setup. We agree this is a useful reference point. We plan to include it in a future revision.
  • In our framework, the latent-space prediction loss (MSE between predicted tokens and target tokens) is the core objective of the JEPA predictor. It supervises state transitions along the SPP trajectory in latent space. Removing this loss turns the model into a standard pixel-reconstruction network. That breaks the latent world model formulation we aim to study.
  • Similarly, SIGReg (lambda_sigreg = 0.09) prevents representation collapse. It keeps the latent distribution isotropic Gaussian. Its effectiveness has already been established in prior LeWM (LeWorldModel) work, which we directly build upon.
  1. Dataset details
  • We evaluate on 22 scenes. For training, each SPP sequence corresponds to one random seed across 12 SPP levels: 1, 2, 4, 8, 16, 32, 64, 128, 256, 512, 1024, and 2048. Multiple seed sequences can be sampled for training.
  • Every training sample pairs a lower SPP level with a higher SPP level from these 12 levels. All training samples use 0 to 8 AOV channels (albedo, depth, geo_normal, position, sh_normal, uv, prim_index, and shape_index) as conditioning actions.
  • At test time, both the 1-SPP input and the 4096-SPP reference are newly and independently sampled renders. Crucially, the highest 4096-SPP ground truth level never appears during training.
  • Under the current per-scene setup, training and test images for a given scene share the same camera, lighting, and material settings. Our goal here is to test whether a per-scene latent world model can capture Monte Carlo convergence dynamics, rather than cross-view or cross-scene generalization.
  1. Higher-resolution evaluation
  • You raise a fair point that 336 x 336 is much lower than practical rendering resolutions. In this work, we focus on exploring the feasibility of a JEPA-based latent world model for rendering denoising.
  • We chose 336 x 336 (a 24 x 24 grid of 576 tokens with patch size 14) to match the pretrained DINOv2 ViT-B/14 backbone. This alignment reduces feature bottlenecks and compression loss from the frozen encoder.
  • Increasing resolution quadratically grows sequence length and attention memory. That would require multi-GPU distributed training. Still, our architecture is modular. It can be upgraded directly to scalable high-resolution Transformer backbones such as FiT or SiT.

Since your feedback is very helpful for our next steps, we would also like to ask three questions for your advice:

  1. For the per-scene standard denoising baseline you suggested, which architecture (such as a lightweight U-Net, DnCNN, or Restormer-style network) would make the fairest comparison against our latent predictor?

  2. When scaling beyond 336 x 336 to practical resolutions while keeping sharp highlights and fine edges, would you suggest moving to flexible-resolution architectures like FiT or SiT, or adding multi-scale skip connections between pixel space and latent tokens?

  3. More broadly, what would an ideal model of this kind look like in your view, and what other rendering or graphics tasks do you think a JEPA-style latent world model conditioned on AOVs would be well suited for?

Thank you again for your time and insightful comments. We warmly welcome any further suggestions that could help improve our work.

Reviewer_QJxCOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes LeDenoiser, a latent-space framework for Monte Carlo rendering denoising that formulates denoising as an action-conditioned latent-state transition. The method uses a frozen DINOv2 encoder to extract image representations, a Transformer predictor to map representations of low-samples-per-pixel (SPP) inputs toward high-SPP target representations, and a pixel decoder to reconstruct images. Training combines latent-space and image-space supervision. Auxiliary Arbitrary Output Variables (AOVs) are encoded as selectable token groups, allowing different combinations of auxiliary inputs, while the Cross-Entropy Method (CEM) searches for multi-step AOV selection sequences. The authors train a separate predictor and decoder for each of 22 benchmark scenes and evaluate reconstruction quality from 1-SPP inputs against independently rendered 4096-SPP references. The experiments examine single-step denoising, different AOV combinations, and multi-step planning strategies.

Critical strengths

Flexible auxiliary input interface. The paper encodes different AOVs as selectable token groups and trains the model with randomly sampled AOV subsets, so a single model can accept different combinations of auxiliary inputs(including none) without retraining or changing the network architecture. This design has practical value for rendering scenarios where availability of auxiliary buffers varies.

Critical weaknesses
  1. The evaluation does not adequately distinguish input-dependent denoising from scene-specific target-image fitting. According to Section 4.1 and Appendix A, a separate model is trained for each scene, with inputs at different SPP levels paired with a 2048-SPP target. However, the training/test split with respect to noise realizations, camera views, and scene states is not clearly specified. Using an independently rendered 4096-SPP reference does not rule out the possibility that the model primarily fits a fixed-view target image; the reported experiments lack controls that distinguish this explanation from input-dependent denoising. Furthermore, OIDN and OptiX do not undergo the same scene-specific training, so the reported performance gap does not adequately separate gains from the proposed method from those attributable to scene-specific adaptation. This limits the support that the main experiments provide for the central claim of learning light-transport dynamics.

  2. The additional benefits of the world-model formulation and planning are not sufficiently established. The paper lacks a conventional conditional latent-space regression baseline with matched training data, network capacity, and pixel-space supervision, as well as key ablations of the latent prediction loss and SIGReg. It therefore remains unclear what benefits the world-model formulation provides over ordinary conditional regression. The planning results offer limited additional support: CEM achieves an average PSNR of 30.30 dB, while the random policy already achieves 30.21 dB (Table 4), an average improvement of only 0.09 dB. Single-step reconstruction using all AOVs is separately reported to achieve 32.66 dB (Table 1), leaving the practical advantage of multi-step planning over single-step inference unclear. Results across different search seeds and actual runtimes are not reported, so the consistency of the planning gains and their relationship to computational cost remain unestablished.

Questions for authors
  1. Please clarify the number of camera views, target images, and independent noise realizations available for each scene, as well as the exact training/test split. Were the 1-SPP inputs used for evaluation included in the training data? Could you provide results on held-out views or camera poses within the same scene, or other controls that distinguish fixed-target fitting from input-dependent denoising, such as a baseline that directly outputs the 2048-SPP training target?

  2. With scene-specific training data, network capacity, and pixel-space supervision matched, how much does the proposed method improve over conventional conditional latent-space regression? What are the individual contributions of the latent prediction loss and SIGReg? Could you also provide a conventional denoising baseline trained or adapted using the same scene data, to distinguish gains from the proposed method from those attributable to scene-specific adaptation?

  3. Is CEM’s average improvement of 0.09 dB over the random policy (Table 4) consistent across different search seeds? What is its actual runtime, including the full search procedure, compared with single-step inference using all AOVs? Is there a clear relationship between the latent fixed-point residual used for planning and the actual reconstruction error?

4 comments
AuthorsOpenReview ↗

What if this were another story?

Thanks for your review and patience.

To be completely honest, when we read your concern about per-scene fitting versus a real world model, it hit close to home—we felt some of that same worry while working on this project. You are right that our model does not generalize across unseen scenes yet, so it is definitely not a universal world-model denoiser. At the same time, following LeWM and LeJEPA, it really does act as a single-scene latent world model that you can interact with and control: picking different AOVs works as the action that guides how the model updates the scene's rendering state.

Since reviewers naturally worry that per-scene training might just be memorizing a target image, we ran a few quick sanity checks across all 22 scenes (evaluated against the held-out 4096-SPP reference with AOVs turned off, k=0):

  • Giving the model a plain gray image (0.5) instead of the 1-SPP input drops the score to 16.22 dB PSNR / 0.5856 SSIM / 0.4022 FLIP.
  • Giving it random noise drops the score to 16.43 dB PSNR / 0.5914 SSIM / 0.4003 FLIP.
  • Running a checkpoint trained on one scene on a different scene (spp_1, k=0) drops to 12.48 dB PSNR / 0.4680 SSIM / 0.6023 FLIP. So the model is clearly not just spitting back a memorized picture; it really depends on what is in the input image.

We also wanted to share another way to look at the numbers already in the paper, which might make our goal clearer:

  • Keep the data, training, and test setup exactly the same. You can think of our single-step Enc + Predictor + Dec forward pass (32.66 dB PSNR, 0.8955 SSIM, 0.0735 FLIP at K=8) as the supervised image-regression baseline, and Enc + Predictor + CEM latent planning (30.30 dB PSNR, 0.8745 SSIM, 0.0894 FLIP, using only 1.6 AOVs on average) as the exploratory world-model planning version.
  • Across all 22 scenes, the average gap between 5-step CEM planning and direct one-step image regression is fairly small: 2.36 dB in PSNR (32.66 vs. 30.30), 0.0210 in SSIM (0.8955 vs. 0.8745), and 0.0159 in FLIP (0.0735 vs. 0.0894). Inside the 5-step rollout comparison, CEM active search even beats using all 8 AOVs at every step (+0.14 dB PSNR) while picking only about 20% of them.
  • Put simply, LeDenoiser works like a small active agent: it looks at the current latent state and distance, chooses which AOV actions to take, and plans a path that gets close to a standard supervised image-regression denoiser.

As far as we know, turning Monte Carlo denoising into an active AOV-planning problem inside a JEPA-style latent world model has not been tried before. Even compared to RL-based denoisers that might act directly on pixels, predicting and planning state transitions in latent space is a very different way to approach the problem.

Thank you again for taking the time to read our paper and share such honest feedback. We know our work is still early and far from perfect, and we truly hope you might take another look at the effort, experiments, and results we put together with these new controls in mind. More than anything, we really want to make this method genuinely better and more useful, so any straightforward advice or ideas you could share on where we should improve next would mean a lot to us.

Thank you for the detailed response and additional results. CEM (30.30 dB) still underperforms direct single-step regression (32.66 dB). Although it improves over the non-selective five-step baseline while using fewer AOVs, the current results do not demonstrate that multi-step planning improves reconstruction quality over direct single-step regression. We therefore encourage the authors to narrow the corresponding claims accordingly.

Thank you for your patience and helpful feedback. We hear your point clearly about multi-step CEM planning versus direct single-step regression, and we will make sure our claims match what the experiments actually support.

Could you please bear with us for a bit longer? While double-checking our planning code, we noticed a planning parameter that had not been fully tested yet, which may relate to this rollout gap. We are running those additional planning experiments right now to check it properly, and we will share a clear summary of the results with you as soon as they finish.

Thank you for waiting. Here are the results from our follow-up CEM experiments. On our original 1-step weights, multi-step planning still does not beat 1-step inference across every scene, so we will narrow the paper claims to focus on latent control and sparse AOV selection. At the same time, the new runs explain why the earlier 5-step score dropped to 30.30 dB and show how to fix it.

  1. Why 30.30 dB dropped, across all 22 scenes with no retraining

Our earlier 5-step rollout used full step size 1.0, feeding a 1-step predictor back into itself 5 times. That caused -2.50 dB of over-smoothing drift, dropping from 32.66 dB to 30.16 dB. Lowering the step size to 0.25 across all 22 scenes raises CEM by +2.20 dB from 30.30 dB to 32.49 dB, cuts latent MSE by 21.0%, and uses 66% fewer AOVs, 2.71 instead of 8. Even without retraining, damped CEM already beats 1-step inference on the hardest multi-bounce and refractive scenes: veach-ajar improves by +0.47 dB at step size 0.25 and +0.80 dB at step size 0.35, veach-bidir by +0.30 dB, glass-of-water by +0.19 dB and +0.40 dB, and volumetric-caustic by +0.12 dB.

Original 1-Step Weights, 22-Scene MeanPSNR ↑SSIM ↑FLIP ↓Avg AOVsLatent MSE
1-step Fixed, 8 AOVs32.6630.89550.07358.00148,938
5-step Undamped Fixed, step 1.0, 8 AOVs30.1610.87250.09228.00111,312
5-step Undamped CEM, step 1.0, target-guided30.2980.87450.08941.57101,496
5-step Damped Fixed, step 0.25, 8 AOVs32.4840.89340.07668.00121,691
5-step Damped CEM, step 0.25, target-guided32.4940.89350.07612.71117,647
3-step Damped CEM, step 0.35, target-guided32.4700.89330.07732.71128,396

We also measured seed variance, runtime, and AOV budget limits on 6 diverse scenes: coffee, cornell-box, dragon, glass-of-water, veach-mis, and veach-ajar. Across 3 search seeds, 5-step damped CEM at step size 0.25 gives 32.015 / 32.020 / 32.022 dB, with a mean per-scene std of 0.0058 dB and a max of 0.0126 dB. Capping each step to at most 2 AOVs keeps 32.013 dB with 1.87 AOVs in 18.2 s. Using 128 candidates for 12 search steps reduces runtime from ~716 s to 15.4–28.1 s, while a single forward pass takes ~0.04 s on average across all 22 scenes.

  1. Multi-step and fixed-point fine-tuning on 6 scenes

Two training gaps held back both the multi-step scores and the reference-free search: training only supervised 1-step outputs, and only used strictly lower-to-higher sample pairs, so clean states were never trained as fixed points. On the 6 scenes above, we ran 40 extra training epochs comparing a matched 1-step-only control against multi-step rollout training with a 1-step loss, a 5-step damped rollout loss at step size 0.35, and a clean fixed-point loss.

6-Scene Study, 40 Extra EpochsPSNR ↑SSIM ↑FLIP ↓AOVsFixed-Point Residual
Matched 1-step Control, 1-step, 8 AOVs33.5620.91680.06568.00—
Matched 1-step Control + 5-step CEM, step 0.3533.4560.91470.06421.83—
Multi-Step Trained, 1-step, 8 AOVs33.3750.91570.07198.0022,709
Multi-Step Trained, 5-step fixed, 8 AOVs33.5820.91740.06778.001,655
Multi-Step Trained + 5-step CEM, target-guided33.6140.91740.06712.371,674
Multi-Step Trained + 5-step CEM, reference-free33.5910.91740.06754.771,607

With multi-step and fixed-point training, 5-step CEM reaches 33.614 dB with target guidance and 33.591 dB without any reference target. Both beat the same model's 1-step pass by +0.239 / +0.216 dB and also beat the matched 1-step-only control of 33.562 dB, while using 2.37 AOVs instead of 8 and cutting the fixed-point residual by ~14x from 22,709 to 1,607.

We will narrow the claims on the original 1-step weights as requested and add these 22-scene damping and 6-scene multi-step results to the revision.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.