- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
PAM-ToD formulates cross-time scene transfer as adapting a frozen road-scene 3D Gaussian Splatting model to a target time of day using a small set of posed target-time anchor images. This paper applies a multiplicative log-gain and nonnegative additive emission correction to Gaussian colors, using a global term, multiscale spatial grids, and per-Gaussian residuals with several regularizers. The paper also introduces CARLA-ToD, a controlled static and dynamic benchmark with matched trajectories and camera poses across morning, noon, and night.
Critical strengths
- Freezing geometry and base colors while attaching a relatively small appearance module is a sensible design for repeated simulation use.
- CARLA-ToD makes per-pixel cross-time evaluation much more meaningful than evaluation over independently captured drives with changing objects and poses.
- Visualization shows PAM-ToD better retains broad road and façade structure under large morning/noon/night shifts than the displayed competing methods.
Critical weaknesses
- The appearance transport prior matches source-Gaussian and target-anchor RGB distributions by within-class luminance quantiles. It does not use correspondence, and therefore assumes that rank order within a semantic class is preserved across potentially different spatial regions, materials, occlusion patterns, and shadow configurations.
- The comparison protocol between different models is not consistent. See Question 1
Questions for authors
- PAM-ToD uses sparse-anchor adaptation setting. While GS-W is trained with eight frames from different times of day, inverse-rendering methods are trained separately for each time and evaluated through environment-map swapping and the diffusion methods are applied to StreetGS renderings. Would each diffusion baseline receive target-time image, anchor, reference, or conditioning information? The evaluation would be more convincing with at least one strong frozen-3DGS adaptation baseline trained on exactly the same target anchors, such as a global affine RGB transform, a global-plus-grid color field without semantic priors, and a regularized per-Gaussian correction baseline.
- The real-world validation is limited. Could the authors to evaluate cross-time appearance transfer using real captures under different illumination conditions, for example using repeated traversals from Oxford RobotCar [1].
3 comments
[Part 2/3]
W2 / Q1. Fairness of the baseline comparison: do the diffusion baselines receive target-time information?
We agree that the paper did not state the inputs to each baseline clearly enough. The full setup is as follows:
| Method | Target-time data | Base 3DGS |
|---|---|---|
| GS-IR / GI-GS / LumiGauss | Full sequence | Own reconstruction |
| GS-W | () | Own reconstruction |
| LumiNet / DiffusionRenderer / UniRelight | () | StreetGS (frozen) |
| PAM-ToD | () | StreetGS (frozen) |
Every in the table represents exactly the same set of anchor images used by PAM-ToD. The diffusion baselines are conditioned on these anchor images with . LumiNet uses them directly as reference images. DiffusionRenderer and UniRelight take environment lighting as input, so they are fed an equirectangular HDR map built from these images. All three methods are applied to the same StreetGS renderings as PAM-ToD. GS-W is trained on the source sequence, with the frames at the anchor timesteps replaced by . The inverse rendering baselines train a separate model on the full sequence for each time of day and swap the environment maps at test time; thus, they utilize significantly more target-time information than PAM-ToD. Under the Novel View Synthesis protocol, the held-out views are strictly unseen by all methods.
We fully agree that a frozen-3DGS adaptation baseline trained with exactly the same anchors is the most informative comparison. Following your suggestion, we have added four simple baselines:
- A global affine RGB transform shared by all Gaussians ( matrix and bias).
- A per-Gaussian affine transform.
- A per-Gaussian affine transform with regularized
- A global gain + spatial field correction without semantic priors.
All baselines utilize the same frozen StreetGS and exactly the same anchor images as PAM-ToD, for both and .
Table R1. Appearance corrections trained on the same frozen StreetGS with the same anchors (same setting as Tables 3–5)
| Method | SSIM ↑ () | PSNR ↑ () | LPIPS ↓ () | SSIM ↑ () | PSNR ↑ () | LPIPS ↓ () |
|---|---|---|---|---|---|---|
| Base (no adaptation) | 0.471 | 10.17 | 0.487 | 0.471 | 10.17 | 0.487 |
| Global affine | 0.691 | 18.51 | 0.356 | 0.697 | 19.09 | 0.350 |
| Per-Gaussian affine | 0.595 | 14.49 | 0.415 | 0.769 | 23.16 | 0.341 |
| Regularized per-Gaussian affine | 0.605 | 14.91 | 0.411 | 0.773 | 23.61 | 0.341 |
| Global gain + spatial field (w/o priors) | 0.647 | 17.80 | 0.404 | 0.767 | 22.06 | 0.341 |
| PAM-ToD | 0.720 | 21.11 | 0.327 | 0.808 | 24.72 | 0.285 |
PAM-ToD outperforms all baselines across every metric at both and .
- Global affine cannot represent spatially varying illumination, meaning more anchors barely improve performance (18.51 → 19.09 dB).
- Per-Gaussian affine reaches 23.16 dB at but drops sharply to 14.49 dB at , falling below the performance of the global transform. With few anchors, its numerous free parameters overfit the anchor images, and Gaussians outside the anchor views are never updated. Adding regularized a marginal improvement of about 0.4 dB (14.49 → 14.91 dB at ).
- Global gain + spatial field (w/o priors) reaches 22.06 dB at but only 17.80 dB at . Even with the same correction structure, corrections for unobserved regions cannot be stably estimated from a few anchors without the semantic priors.
By contrast, PAM-ToD achieves an impressive 21.11 dB even at .
[Part 3/3]
Q2. Validation on real-world data
Thank you for this excellent suggestion. We agree that validation with real target-time images was the most significant gap in our original experiments. We therefore ran a new real-world experiment, and the results are very encouraging: PAM-ToD outperforms all baselines by a large margin. We believe this experiment makes the paper substantially stronger.
The Oxford RobotCar dataset was designed for visual odometry, which may introduce localization errors into novel view evaluation. Therefore, we used nuPlan multi-traversal data, where the exact same route is driven repeatedly under varying lighting conditions. We reconstruct StreetGS from a source traversal, adapt it using anchor images taken from a target traversal, and evaluate on held-out views from the target traversal across four scenes. Because dynamic elements like parked vehicles and pedestrians differ between traversals, metrics are computed strictly on static regions observed in both.
Table R2. Quantitative results on nuPlan (Novel View Traversal)
| Method | SSIM ↑ () | PSNR ↑ () | LPIPS ↓ () | SSIM ↑ () | PSNR ↑ () | LPIPS ↓ () |
|---|---|---|---|---|---|---|
| Base (StreetGS, no adaptation) | 0.549 | 15.21 | 0.279 | 0.549 | 15.21 | 0.279 |
| MTGS | 0.579 | 15.79 | 0.359 | 0.576 | 15.92 | 0.353 |
| WildGaussians | 0.513 | 14.28 | 0.401 | 0.519 | 14.49 | 0.401 |
| GS-W | 0.503 | 13.73 | 0.383 | 0.509 | 13.97 | 0.381 |
| PAM-ToD | 0.605 | 17.81 | 0.274 | 0.759 | 22.42 | 0.195 |
PAM-ToD proves highly effective on real target-time images as well. At , it improves PSNR by 7.2 dB over the base StreetGS (15.21 → 22.42 dB) and outperforms the best baseline, MTGS, by a margin of 6.5 dB. Even at , where anchors are sourced from a single timestep, it consistently outperforms all baselines across all metrics while maintaining real-time rendering above 70 FPS.
We have added these real-world results to the main paper (Table 6 and Figure 4 of the revised PDF). We have also clarified in the revised PDF that the Waymo experiment, which uses synthesized anchors, validates the adaptation of scenes reconstructed from real data rather than accuracy against real target-time images.
Authors rebuttal
[Part 1/3]
We thank the reviewer for recognizing our frozen-geometry design and the CARLA-ToD benchmark, as well as for the constructive suggestions. We address each point below.
W1. The appearance transport prior assumes that luminance ranks are preserved within a semantic class
We agree that when materials, shadows, or occlusions differ within a semantic class, matching luminance quantiles does not always yield the exact appearance change.
However, the appearance transport prior does not strictly assume the preservation of the luminance rank of each individual Gaussian. Rather, it provides a statistical prior on appearance changes, derived from per-class luminance distributions.
The final appearance is determined by optimization against the target anchor images. In regions observed by the anchors, this optimization explicitly corrects the mismatch between the prior and the actual illumination change.
Our ablation study supports this. In Table 4, removing the appearance transport prior lowers PSNR from 24.72 to 24.54 dB, drops SSIM from 0.808 to 0.803, and worsens LPIPS from 0.285 to 0.289. Although the scenes contain regions where the rank assumption does not hold, the prior does not degrade accuracy and serves as an effective overall prior.
In unobserved regions, however, errors from the prior may remain. We have stated this as a limitation in the Conclusion of the revised PDF.