- Rating
- 1
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
The paper treats Monte Carlo denoising as a latent state transition, in the style of JEPA/LeWM.
- A frozen DINOv2 ViT-B/14 encodes a noisy render into 24×24 patch tokens.
- A 6-block transformer predicts the tokens of a 2048-SPP target. Each AOV (8 types) is encoded by the same backbone and injected via cross-attention as a "discrete action," so any subset can be supplied at test time.
- A 111.9M-parameter DPT-style decoder maps the tokens back to RGB. Training uses token MSE, SIGReg, L1, and LPIPS.
- A CEM planner searches over AOV subsets using a reference-free fixed-point residual.
On SPP-WORLD (22 Bitterli scenes in Mitsuba 3 at 336×336), with one model fitted per scene, it reports 32.61 dB at 1 SPP without AOVs, versus 21.54 dB for OIDN and 22.49 dB for OptiX. CEM selects about 1.6 of 8 AOVs and beats all-AOV conditioning by 0.14 dB within a 5-step rollout.
Critical strengths
An interesting formulation for MC denoising. The paper explores action-conditioned prediction in a frozen foundation-model latent space, drawing on JEPA and LeWM. It treats AOVs as independently selectable conditioning tokens rather than fixed input channels, and searches over their activation at test time using a reference-free latent self-consistency objective. This offers a different architectural and conceptual perspective on MC denoising and connects the denoising literature to recent self-supervised world-model work, although the benefits of this formulation remain insufficiently established.
Critical weaknesses
The evaluation protocol does not demonstrate generalization.
- One predictor–decoder pair is fitted per scene, with every training pair sharing the same 2048-SPP target; no viewpoint variation or held-out views are reported.
- Testing uses an independently sampled 4096-SPP reference, but no held-out scene content is reported.
- With a 111.9M-parameter decoder, learned positional embeddings, and only 11 reported input SPP levels per scene, the model may simply memorize the shared training target rather than learn to denoise. The reported experiments do not rule out this possibility.
- Adding all 8 AOVs changes PSNR by only 0.05 dB, which is consistent with this explanation, though not diagnostic. Separately, producing the training target already requires rendering a high-SPP image of the scene, raising a practical cost concern.
- So the roughly 10 dB margin over general-purpose OIDN and OptiX, which get no scene-specific supervision, does not establish a better general-purpose denoiser. No held-out views or scenes, target-copy control, or matched per-scene baseline is reported.
The world-model contribution lacks empirical support.
- The reported predictor is not conditioned on SPP or Δt; it always targets 2048 SPP. Repeated rollout feeds predicted latents back into a model trained on encoded rendered inputs, without reported rollout training. This introduces an unvalidated distribution shift rather than demonstrating learned convergence dynamics.
- CEM planning requires about 45k candidate-level predictor evaluations for rollouts, plus approximately 9k for scoring, and reaches 30.30 dB. That is 2.36 dB below the single-step model (32.66 dB) and 0.09 dB above random search (30.21 dB), with no variability reported.
- Appendix B shows that a running mean equals a batch mean. That does not establish that the mean is a sufficient statistic for estimating converged radiance, and it does not show that DINO features preserve the relevant information.
- SIGReg's role is unclear. The encoder is frozen and the targets are fixed DINO tokens, so joint collapse through changes to the target representation is prevented, while feature regression already penalizes inaccurate predictor outputs. The paper does not establish what additional failure SIGReg prevents or whether it improves reconstruction. No ablation is given.
Questions for authors
Clarifications and experiments that could change my assessment:
-
Is the 1-SPP evaluation input the same render used in training, or independently sampled?
-
Generalization controls. What results do you obtain from: (a) Using the 2048-SPP training target itself as the output, without running the model, what PSNR/SSIM/FLIP does it achieve against the 4096-SPP evaluation reference? (b) With AOVs disabled, what happens if the input image is replaced by a uniform image or random noise? Successful scene reconstruction would suggest that the model can reproduce the training target without relying on scene information in the input. (c) a small pixel-space U-Net trained with the same per-scene supervision; (d) held-out views or scenes, without retraining?
-
In what space are the losses and PSNR/SSIM/FLIP computed, and which FLIP mode is used? How is HDR radiance mapped into, and recovered from, the sigmoid output?
-
Which OIDN and OptiX versions and settings were used, including
cleanAuxand the albedo/normal definitions at specular and glass hits? Please show baseline outputs in Figs. 3–4. Some OIDN SSIMs are unusually low (0.17 on kitchen, 0.06 on veach-ajar), yet OIDN exceeds your SSIM on dragon, lego, and rover. -
What quality does Dec(Enc(I_ref)) achieve, and do the missing highlight (veach-mis) and glass sphere (veach-bidir) persist? This would separate encoder–decoder limits from latent-prediction errors.
-
What are the per-scene training time, the rendering cost of the training data, and the end-to-end inference latency with and without planning?
-
How were the 22 scenes selected from the Bitterli collection?
Would also strengthen the paper: SIGReg and latent-loss ablations; comparisons with learned denoisers (e.g., NPPD, KPCN) under matched supervision.
Future work (not required for my assessment): cross-scene training with evaluation on Noisebase; a radiance-skip or kernel-prediction readout to preserve fine detail; an interactive A/B viewer with full-resolution outputs.
6 comments
Thank you so much for your patience and thoughtful review. We truly appreciate the constructive concerns, evaluation suggestions, and additional experiments you proposed—they are genuinely helpful for improving our work and have encouraged us to keep pushing this direction forward. We still believe exploring LeDenoiser has promising value, though we also recognize that many details currently have room for improvement and that we need to better demonstrate what makes the architecture and approach compelling.
Since we value your perspective, we would love to ask a few questions to learn from your insights:
- In your view, what would the "ideal" version of our method look like? What key capabilities should it have, and what distinct advantages over existing baselines would make it most convincing?
- Our current architecture uses a full-Transformer encoder/decoder with conditional action attention, yet we still struggle with losing fine details and highlights (such as in
veach-mis). Do you have any architectural or methodological recommendations that could help address this?
Thank you again for your time and guidance!
Thank you for the response. Unfortunately it does not resolve my main concerns, and the clarification on the data split makes my soundness concern stronger, so I will keep my score. (1) Each scene is trained for only 700 steps on a single RTX 4090, so a per-scene U-Net baseline should be cheap to run. Without one, comparing against generic OIDN/OptiX models, which never see clean images of the test view, is not fair. (2) Saying that removing the latent loss "turns the model into a standard pixel-reconstruction network" is exactly what needs testing. If that network performs comparably, the world-model formulation is not what drives the results. For SIGReg, the encoder here is frozen and the targets are fixed, so the collapse argument from LeWM does not carry over (Sec. 3.1 itself says the frozen encoder prevents collapse). The concern about other pretrained encoders, or training from scratch, was not addressed. (3) Training and test share the same view, and each scene has only ~28 training pairs, all targeting a 2048-SPP render of that view. So the claim that "the 4096-SPP reference never appears in training" means little: the 2048-SPP target is nearly identical to it, and simply outputting that target would likely beat 32.61 dB by a wide margin. For a fixed view the AOVs are also essentially constant across training samples. This explains why K=0 ≈ K=8 and makes the +0.14 dB planning gain hard to interpret, especially since the 5-step rollout (30.30 dB) is worse than a single step (32.66 dB). The pairing scheme described in the response also differs from Appendix A. (4) The official DINOv2 positional embeddings are for 518×518, so 336×336 does not uniquely "match" the backbone, and tiled inference is standard for high-resolution denoising. No new results were provided.
Thanks for you reviews, which is quite helpful to make our manuscript better. Here are the discussion and responses for the questions: