- Rating
- 3(+1)
- Confidence
- 1
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes a simulation-grounded VLM framework that turns 2D wildfire simulations into labeled synthetic videos, then uses video retrieval and a multi-agent system to estimate fire-related physical conditions and generate structured wildfire reports.
Critical strengths
- A practical and well-motivated problem.
The paper addresses the lack of real wildfire videos with synchronized physical annotations. Using 2D simulations, lightweight 3D proxies, and video generation to create labeled visual data is a reasonable and potentially scalable solution. This pipeline could be useful for other scientific applications where physically annotated visual data is scarce.
- Solid experimental improvements on synthetic benchmarks.
The proposed video-memory approach improves four-tag classification accuracy from 22.6% (direct VLM querying) to 51.5%, while the full multi-agent system reaches 77.3% accuracy on six downstream reporting tasks. The paper also includes component ablations, cross-generator tests, and some real-UAV evaluations, making the empirical results reasonably convincing within the synthetic-data setting.
Critical weaknesses
The paper claims to bridge physical wildfire simulation and real-world visual monitoring. However, its main evaluation only demonstrates the recovery of simulation-derived labels from synthetically generated videos, while the real-world experiments address different, simpler perception tasks. It remains unclear whether the proposed framework learns or retrieves physically meaningful cues that transfer to real wildfire observations.
Questions for authors
The paper’s main motivation is to bridge the gap between physical simulations and real-world wildfire monitoring. However, the current experiments mainly evaluate alignment with synthetic proxy renderings or performance on different real-world perception tasks.
Could the authors clarify whether they have quantitatively evaluated the sim-to-real gap between generated and real wildfire videos, for example, using established video-distribution metrics such as FVD?
More importantly, is there any evidence that the physical cues learned from synthetic videos generalize to real UAV footage? The current real-world experiments focus on smoke/fire detection and fire-area ranking rather than the proposed environmental-driver estimation tasks. Could the authors provide a same-task sim-to-real evaluation or other quantitative evidence supporting this transfer?
3 comments
Thank you for the detailed response and revisions. After reading the revised paper, I appreciate that the authors have clearly narrowed their claims and explicitly acknowledged the lack of real-world validation for physical-driver estimation.
While the sim-to-real gap remains an open issue, I agree that the current experiments provide reasonable support for the paper’s more focused contribution in simulation-grounded video generation and memory-based VLM inference. It is certainly an interesting and meaningful topics and I'd like to see it advancing.
I therefore consider my main concern sufficiently clarified and am willing to raise my score.
We thank reviewer for considering the real data lacking issue and consider this paper will help advancing this area of research and raise the rating.
We thank the reviewer for your recognition of the importance of such a problem, the contribution of our work and the questions for improving. We sincerely hope reviewers will consider the mentioned difficulties of acquiring real data with target settings and consider the experiments we did in order to partially compensate for this problem.
Evaluation on real UAV footage:
This actually is exactly the motivation and reason why we want to propose this framework, especially the first data collection stage and demonstrate it is usable to aid wildfire predictions using our memory system and agentic framework. A direct numerical comparison against arbitrary online UAV videos would also be difficult to interpret because the terrain, viewpoint, sensor characteristics, fire intensity, and temporal evolution would not be matched. Constructing a paired real reference would require recovering these latent conditions from real footage, which is itself a separate and ill-posed inverse problem. To the best of our knowledge, there’s wildfire image dataset and very few UAV wildfire video ones publicly available, e.g. FireMan-UAV-RGBT or Boreal Forest Fire dataset and FLAME dataset are the ones that’s widely used for wildfire evaluations. However, they do not provide real clips paired with matching simulator states and the complete set of environmental and fire-behavior labels required by our evaluation but are designed for smoke and fire detection with many experiment actually performed as controlled burn instead of real wildfire capturing. To be specific, FireMan-UAV-RGBT depicts controlled burns of piled vegetation rather than wildfires. Boreal Forest Fire dataset targets smoke detection and the fire front is often barely visible. FLAME primarily supports fire detection and segmentation. Nevertheless, this is the real-world experiment dataset that we can get to the best of our knowledge and that’s why we evaluated the dataset’s targeted task even they are not directly same as our work. Ideally, evaluation would use UAV footage captured at the physical location represented by our simulation, or an artist-created aerial environment approximating it with turned 2D simulation for propagation, but this is truly beyond our current capable resources.
Such data lacking condition not only hinders the evaluation of proposed framework but also suggest lack of environment for robotic training. This scarcity, particularly for academic teams without access to wildfire agencies, domain experts, or proprietary data, is precisely the motivation for our simulation-grounded framework, i.e. the 2D to 3D proxy to controlled generated video pipeline.
To partially address the problem of testing robustness of our framework under scarce data condition, we also provided an experiment where we switched out the video generator as shown in Table 5 and below:
We truly hope reviewer to consider the practical availability of such data, the scope of the paper, and the additional evaluations we provided, even if they did not fully address the reviewer’s expectations.
FVD with real footage:
As mentioned in the previous section, real footage collection of UAV for wildfire videos is really lacking and we don’t have direct access to those videos. However, since the 3D proxy is directly procedurally recovered from 2D simulation automatically, we consider it faithful with simulation. That’s why we choose to evaluate the physics alignment with 3D proxy video (in Appendix D) and added a realistic appearance by querying the existing LLM (GPT, Claude, Gemini) since LLMs should have seen enough realistic wildfire videos implicitly during their training. We provided the physics alignment with proxy video in Appendix D and the appearance evaluation using implicit knowledge of LLMs is shown in Table 14 and below where LLM scores the video with 0-100 for realism: