- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3
Read review
Paper summary
PhySmoke reconstructs clear images at observed camera views from smoky RGB images, registered thermal images, and known cameras. Static surfaces are represented by native 2D Gaussian surfels and dynamic smoke by deformable 3D Gaussians. A mixed renderer jointly sorts the primitives and composites them through one transmittance chain. A final reliability gate uses the estimated smoke contribution to control how much image residual is restored. On nine synthetic SmokeSeer scenes, the complete method improves the macro-average PSNR, SSIM, and LPIPS over SmokeSeer. The evaluation is restricted to observed views; the method uses the smoky RGB image at the target pose.
Critical strengths
-
The surface/volume split matches the distinct roles of the static scene and moving smoke. The controlled mixed-renderer comparison reports a 1.17 dB PSNR gain over layer-wise compositing with the same components. This supports the claim that shared depth ordering matters, rather than merely swapping a 3D surface representation for a 2D one.
-
The detail-recovery ablation identifies a real tradeoff. Ungated residual recovery improves PSNR but worsens SSIM and LPIPS, whereas the proposed gate retains the PSNR gain and improves those two macro metrics. The added recovery improves PSNR over the physical representation in all nine scenes, although gains in the other metrics vary.
Critical weaknesses
-
The evidence for generalization is narrow. All nine evaluated scenes are synthetic scenes from one benchmark, and the Dense experiment changes initialization on the same scene identities. The paper reports one seed. Because the method depends on registered thermal boundaries and a smoke-based reliability estimate, I would like evidence on real RGB/thermal captures or, at least, sensitivity to registration error, thermal noise, and weaker thermal boundaries. Without it, the practical motivation remains less well supported than the in-benchmark reconstruction result.
-
The paper gives a compactness ratio for the surface component, not cost for the complete pipeline. The method also trains a dynamic smoke representation and an additional volumetric branch for recovery. Training time, rendering time, memory, and total primitive count would clarify the practical cost of its roughly 1 dB average PSNR gain. This is an evidence gap, not a demonstrated flaw in the renderer.
Questions for authors
-
How sensitive are the observed-view results to RGB/thermal misregistration and noisy or weak thermal boundaries? Even a controlled perturbation on the current benchmark would help establish the usable capture range.
-
Could the authors provide complete-pipeline training time, rendering time, peak memory, and total surface-plus-smoke primitive count, using the same hardware and protocol as SmokeSeer?
-
Do the authors have any results on real registered RGB/thermal smoke captures? If not, which capture assumptions are most likely to fail outside the synthetic benchmark?
3 comments
Thank you for the detailed response and additional experiments. The reported results substantially address my original concerns about real-data validation, sensitivity to thermal inputs, and complete-pipeline cost.
The self-collected real scene provides quantitative evidence beyond the synthetic benchmark: PSNR/SSIM/LPIPS improve from 20.10/0.7895/0.2190 to 20.48/0.8206/0.1773 across 77 observed views. The two public real captures add qualitative support. The controlled thermal-input perturbations directly address my robustness question, and the three-seed repeats on three selected scenes strengthen confidence in the gains.
The revision also clarifies important limitations. Halving thermal contrast reduces mean PSNR from 22.32 to 17.22 dB and narrows the advantage over SmokeSeer from 0.99 to 0.15 dB. Complete-pipeline profiling shows approximately 2.1-2.4 times the baseline training time and 7.3-9.8 times its output latency. Thus, surface compactness should not be interpreted as lower full-system cost.
One remaining clarification concerns the additional thermal-derived depth-support constraint used on the self-collected scene. A matched ablation with and without this constraint would help distinguish the original configuration's contribution from that of the real-data adaptation. I regard this as a clarification of the real-data gains rather than a prerequisite for my updated recommendation.
Overall, the added evidence, together with the joint-renderer and recovery ablations, supports the technical contribution for the stated observed-view reconstruction task. I am revising my recommendation from weak rejection (2) to weak acceptance (3), while retaining the limitations concerning real-scene coverage, thermal-input quality, and computational cost.
Response to Reviewer 2puG for the depth-support adaptation
Thank you for the careful reassessment and for updating your recommendation. We appreciate your recognition of the added real-data validation, thermal-input tests, and complete-pipeline profiling. We agree that surface compactness and full-system cost are separate measurements.
We have already completed the matched ablation you suggest. Both runs start from the same 15k thermal-stage checkpoint and use the same inputs, camera calibration, initialization procedure, seed, 15k joint-training budget, and recovery settings. The comparison switches the depth-support adaptation off or on, including its associated suppression of unsupported surface densification. Evaluation follows Table 4: all 77 observed training views, identical object bounding boxes, and the same independently reconstructed smoke-free reference renderings.
The matched ablation results are:
| Method | PSNR ↑ | SSIM ↑ | LPIPS ↓ |
|---|---|---|---|
| SmokeSeer | 20.10 | 0.7895 | 0.2190 |
| Ours without depth support | 20.52 | 0.8199 | 0.1785 |
| Ours with depth support | 20.48 | 0.8206 | 0.1773 |
These results directly distinguish the contributions. Without the additional constraint, our method already improves all three metrics over SmokeSeer, including a 0.42 dB PSNR gain and an 18.5% LPIPS reduction. Adding depth support further improves SSIM and LPIPS, with a small 0.04 dB PSNR decrease. The substantial improvement over SmokeSeer is therefore already present without depth support; the adaptation adds an incremental structural and perceptual improvement.
The visual motivation is that a higher PSNR does not by itself establish cleaner surface–smoke separation. Our full-image photometric objective rewards background fidelity alongside object fidelity, while some smoky appearance can still be absorbed into the surface representation. We introduced depth support to address the residual smoke and local structural blur seen in the surface renderings, explicitly guiding optimization toward depth-consistent surfaces. The SSIM and LPIPS improvements support this emphasis on structural and perceptual quality despite the small PSNR tradeoff. The object-region evaluation above quantifies the effect on the reconstruction target directly.
For real captures, whose smoke distribution need not match the simulator, this constraint uses the geometric support already available in our representation. Specifically, we use the native 2DGS depth-rendering capability to obtain a frozen surface-depth reference from the thermal stage. During joint optimization, opacity-weighted penalties discourage surface primitives from floating in front of reliable depth support, while the volumetric smoke branch remains free to explain the intervening medium. This uses the existing thermal observations and requires no external depth sensor, clear-image supervision, or smoke labels. The synthetic experiments and the two public real-capture examples use the configuration without this additional constraint.
This matched comparison clarifies the contribution of the depth-support adaptation. Thank you again for the constructive discussion.
Response to Reviewer 2puG
Thank you for requesting real-data validation, controlled input perturbations, and complete-pipeline profiling. We have addressed these requests with a substantial experimental expansion: our own physical RGB–thermal capture and calibration pipeline, public real captures, 42 complete two-stage training runs covering perturbations and three seeds, and full system cost measurements. The revision reports the resulting evidence in Section 4.7 and Appendices H–I.
We assembled a capture setup using a DJI Matrice 600 carrying a Zenmuse XT2, operated along the ground. We acquired paired smoky RGB–thermal observations and separate smoke-free sequences, calibrated the fixed camera rig, reconstructed poses and initial geometry, and constructed independent smoke-free reference renderings. RGB and thermal observations use their own intrinsics and calibrated poses.
Table 4 evaluates all 77 observed training views of our self-collected scene using identical object bounding boxes. Ours improves PSNR/SSIM/LPIPS from SmokeSeer's 20.10/0.7895/0.2190 to 20.48/0.8206/0.1773: +0.38 dB PSNR, +0.0311 SSIM, and a 19.0% LPIPS reduction. Figure 6 shows four matched-view comparisons across our data and the public Bathroom and Red Container captures. The magnified regions demonstrate smoke suppression and clearer surface reconstruction. Appendix H documents the full acquisition and evaluation pipeline.
Appendix I.2, Table 10 retrains both stages of both methods on Hotdog, Room, and Forest, covering object, indoor, and outdoor content. We use the same initial cloud, full resolution, seed, and 15k thermal + 15k joint budget. Thermal-camera orientation, thermal noise, and thermal contrast are changed independently; recovery parameters retain the synthetic defaults. Every entry averages the same three scenes.
Mean PSNR in dB, reported as SmokeSeer / Ours: Clean: 21.33 / 22.32. Yaw 0.5°: 18.54 / 19.60. Yaw 1°: 17.43 / 18.00. Noise σ=0.01: 19.98 / 20.92. Contrast 0.5×: 17.07 / 17.22.
Ours achieves higher mean PSNR and SSIM and lower LPIPS in every tested condition. These results directly verify the reconstruction advantage under registration error, noisy thermal observations, and weakened thermal structure. The tables also make the dependence on input quality explicit. In real acquisition, our fixed-rig calibration and separate camera projections provide the geometric consistency needed to use both modalities effectively.
Appendix I.3, Table 11 profiles matched clean seed-zero runs on RTX A6000 GPUs. The values below are reported as SmokeSeer / Ours and include both learned branches for Ours.
Hotdog: training 25.4 / 56.1 min; output 5.7 / 55.9 ms/view; training peak memory 1.97 / 2.04 GiB; inference peak memory 0.16 / 0.36 GiB; total primitives 0.38 / 0.66 million.
Forest: training 31.5 / 66.7 min; output 9.2 / 67.3 ms/view; training peak memory 2.83 / 2.83 GiB; inference peak memory 0.51 / 0.72 GiB; total primitives 1.65 / 2.46 million.
Room: training 30.8 / 74.1 min; output 10.3 / 80.3 ms/view; training peak memory 2.74 / 2.74 GiB; inference peak memory 0.52 / 0.84 GiB; total primitives 1.54 / 2.60 million.
Training time sums the required stages, including loading and final checkpoint I/O, with validation/reporting disabled equally. Training memory is the maximum across sequential stages. Synchronized output latency includes both deformation networks, both surface/joint outputs, fusion, recovery, and required host-to-device transfers; disk writes are excluded and the scene-fit prepass is reported separately. Memory is peak PyTorch CUDA allocation, and counts include every required surface and smoke set.
This profiling quantifies the compute–quality tradeoff: the full reconstruction and recovery pipeline produces outputs in 55.9–80.3 ms and uses 2.1–2.4× baseline training time. The surface compactness measurement and the complete-pipeline profile now give explicit, separate accounts of representation size and system cost.
Appendix I.4, Table 12 reports fresh matched training with seeds 0, 1, and 2, including means and sample standard deviations. Ours improves all three metrics in every one of the nine scene/seed pairs. Mean paired PSNR gains are 2.62 dB for Hotdog, 1.02 dB for Room, and 0.23 dB for Forest. Together with the real acquisition study and controlled perturbations, these repetitions substantially strengthen the experimental validation of the method.