acceptodds
Desk rejected at ICLR 2027

FireWorldBench: Benchmarking Complex Physical World Intelligence through Coupled-Field Fire Dynamics

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Understanding the physical world requires more than object recognition, scene description, and short-term visual prediction, as real-world physical systems involve multiple continuous fields, latent causal mechanisms, partial observations, and intervention-sensitive dynamics. We propose FireWorldBench, a benchmark for evaluating complex physical world intelligence in multimodal large language models and agents through coupled-field fire dynamics. Fire provides a canonical stress-test environment, where multiple interacting physical fields jointly shape observable states and temporal dynamics. FireWorldBench is organized along two complementary axes, a physical capability axis and a fire scenario task axis, jointly covering physical-state understanding, temporal dynamics, causal mechanisms, and intervention reasoning. The benchmark comprises 520 fire-world entries, including 494 controlled simulation worlds and 26 real-world-aligned event groups, spanning 47 scene archetypes across 7 environment families. These entries combine structured textual observations, multiple 2D physical-field visualizations, and 3D event-level scene modeling, yielding 9,074 text-image interleaved question-answer pairs across choice-based and open-ended report-generation formats. FireWorldBench evaluates whether models can infer latent physical states, explain underlying mechanisms, forecast coupled-field evolution, and assess intervention consequences from multimodal partial observations, providing a challenging testbed for complex physical world intelligence.

Primary area:
datasets and benchmarks
Keywords:
MLLM, Physical World, Fire Dynamics

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_a2wEOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper introduces FireWorldBench, a benchmark for evaluating physical-world reasoning through coupled fire dynamics. It contains 9,074 QA pairs from 494 simulated worlds and 26 real-world-aligned event groups, covering five physical capabilities and three fire-scenario categories. The benchmark pairs choice-based and structured open-report tasks across textual and visual observations, with deterministic scoring of predictions and explanations.

Critical strengths
  1. Fire dynamics involve interactions among temperature, smoke, airflow, and ventilation, making them a useful setting for studying continuous physical fields, partial observability, and responses to interventions. Evaluating models in this setting provides a meaningful complement to existing physical reasoning benchmarks.

  2. The benchmark covers 47 scene archetypes and 9,074 question–answer instances, with reference answers grounded in simulation states and controlled counterfactual branches for evaluating intervention effects. Paired choice-based and open-report tasks, structured and visual inputs, and distinct physical capability categories enable more detailed analysis of where models struggle across evidence formats and reasoning tasks.

Critical weaknesses
  1. The paper lacks in-depth error analysis and case study. The analysis primarily reports performance differences and five qualitative examples show successful responses. Without representative failures and a systematic analysis of recurring failure patterns, it remains unclear why models fail and which limitations future methods should address.

  2. The explanation metrics do not adequately establish physical correctness. Mechanism Alignment measures reference-token recall, Evidence-F1 does not verify region–variable relationships, and GLS relies largely on lexical matching rules. So explanations containing correct terminology but incorrect causal relationships may receive high scores.

Questions for authors

Please see the Critical Weaknesses section for my main concerns.

4 comments
AuthorsOpenReview ↗

Response to Reviewer a2wE (1/2)

We sincerely thank the reviewer for the constructive suggestion.

W1. Systematic error analysis and representative failures

➡️A systematic analysis of recurring failure patterns. We analyze recurring errors in models’ reasoning about physical fields and identify four failure patterns (Table 1 reports the proportion of each failure pattern among all failed samples.) and their associated causes:

  1. Insufficient use of field evidence. Models fail to use field observations to constrain mechanism attribution and intervention predictions, instead producing generic physical explanations inconsistent with the evidence. This stems from a failure to assess the conditions under which field evidence is applicable.
  2. Limited understanding of field evolution. Models fail to adequately account for temporal changes in physical fields and tend to extrapolate past trends directly into the future.
  3. Local–global inconsistency. Models identify field information in specific regions or at particular times but fail to form an overall judgment consistent with all available observations.
  4. Cross-field integration failure. Models correctly read relevant information from individual fields but fail to integrate their spatial and temporal relationships to infer cross-field coupling.

Table 1. Distribution of failure patterns.

Failure PatternProportion
Cross-field integration failure23%
Insufficient use of field evidence16%
Limited understanding of field evolution38%
Local–global inconsistency12%
Other failure patterns11%
Total100%

➡️Detailed analysis of cross-field integration failure. Table 2 shows that 72 of 158 responses with correct basic field readings still make incorrect cross-field inferences (45.6%). Among these 72 responses, 58 also produce incorrect final answers (80.6%). These results highlight cross-field integration failure as a recurring bottleneck associated with task failure.

Table 2. Visual field-task errors and final-answer outcomes. Parentheses show error counts / corresponding sample counts.

Failure PatternPrevalenceFinal-answer error rate among affected responses
Incorrect cross-field inference despite correct field understanding45.6% (72/158)80.6% (58/72)

➡️Case study. Table 3 presents representative case analyses illustrating these failure patterns, contrasting models’ incorrect judgments with the corresponding physical-field evidence.

Table 3. Representative cases illustrating each failure pattern.

Failure patternModel / observationIncorrect model judgmentField evidence and error diagnosis
Local–global inconsistencyGPT-5.6-Luna / structuredThe fire source is in R2 and is decaying at 10 s.R2 cools from 66.28 to 26.24 °C over 5–10 s, but the actual source is in R6 and its heat-release rate continues to increase. The model uses cooling in one region to infer both the source location and the overall fire stage.
Insufficient use of field evidenceQwen3.5-Plus-02-15 / structuredHigher velocity indicates stronger buoyancy or earlier flashover.Neither condition includes combustion input. Soot is zero, visibility is 30 m, and the temperature difference is only approximately 0.023 °C. Nevertheless, the model applies a generic explanation of fire intensification.
Cross-field integration failureGPT-5.6-Luna / visualThe temperature peak in R2 and soot peak in R6 indicate two fire sources.Ignition occurs only in R2. Positive flow of approximately 0.45 m/s and soot evolution over 20–40 s indicate transport from R2 to R6. The model fails to use flow direction to explain the spatial separation of the two field peaks.
Insufficient use of field evidenceQwen3.5-Plus-02-15 / visualIncreased ventilation will substantially intensify the fire and smoke hazard.Neither condition includes combustion input. Soot is zero and visibility is 30 m. The future normalized hazard difference is only 0.012. The model fails to constrain its ventilation-effect judgment using the field evidence.
Limited understanding of field evolutionGemma-3-12B-IT / structuredBased on past trends, R2 temperature and soot density will continue to increase, while visibility will continue to decrease over the next 30 s.Temperature and soot do increase and visibility decreases during 30–60 s, but the reference trends at 90 s are reversed. The model directly extrapolates past field changes into the future.
AuthorsOpenReview ↗

Response to Reviewer a2wE (2/2)

W2. Physical correctness of explanations

➡️Metric refinement. We augment the original metrics with physical-relation checks, yielding Evi-F1†, Mech†, and GLS† to assess explanation correctness beyond lexical overlap. Metrics are aggregated using the same model, observation-track, and task weights as in the original paper, with inapplicable samples excluded according to predefined rules.

  • Evi-F1† checks region–variable bindings, time, statistical definitions, and numerical values or trends in addition to the original region and variable matching.
  • Mech† checks cause–effect relations, causal direction, and necessary conditions in addition to the original reference-mechanism token recall.
  • GLS† additionally checks whether each prediction target is supported by evidence and mechanisms with correct physical relations and no direct contradictions, extending the original association between correct predictions and evidence/mechanism anchors.

➡️Alignment with Expert Judgments. Experts assess the physical correctness of model-generated explanations. Using these judgments as references, the newly added metrics achieve AUROC values of 0.827–0.846 as shown in Table 4, supporting their ability to distinguish correct from incorrect explanations.

➡️Conclusion robustness. We compare the original and newly added scores under matched evaluation settings. Table 5 shows slightly lower scores but unchanged capability rankings within each metric, supporting the robustness of our capability-level findings.

Table 4. Agreement between the newly added metrics and expert judgments.

Supplementary metricValid reportsCorrect / incorrect explanationsAUROC ↑95% CI
Evi-F1†190112 / 780.834[0.771, 0.895]
Mech†172101 / 710.846[0.779, 0.905]
GLS†184109 / 750.827[0.759, 0.888]

Table 5. Original and the newly added metrics on the main controlled-simulation set (%).

CapabilityOriginal Evi-F1Evi-F1†Original MechMech†Original GLSGLS†
P1 Temporal forecasting58.0656.7436.0534.6123.7322.85
P2 Field perception and grounding40.8740.1521.9921.1229.5028.69
P3 Cross-field coupling56.3354.4828.1626.9332.1230.74
P4 Mechanism attribution30.3329.1715.4114.5829.3528.43
P5 Counterfactual intervention9.639.2118.4217.5427.1226.18

We thank the reviewer again and would be glad to clarify anything further.

Thank you for the detailed response. It addresses most of my concerns and I will raise my score.

We sincerely thank you for your constructive review and positive follow-up, which have provided great value for our future research.

Reviewer_nbLjOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FireWorldBench, a benchmark designed to evaluate complex physical-world intelligence in multimodal large language models through coupled-field fire dynamics. The benchmark evaluates five physical reasoning capabilities (P1–P5), ranging from temporal forecasting and physical-field perception to causal attribution and counterfactual reasoning, along with three fire-evolution task categories. The authors evaluate a collection of foundation models using structured and multimodal observations, multiple-choice questions, and open-ended reports.

Experimental results suggest that current models struggle with several aspects of coupled-field physical reasoning, particularly when interpreting multimodal physical-field visualizations. The authors also investigate physics-aware prompting, chain-of-thought reasoning, and supervised fine-tuning.

Critical strengths
  • The paper highlights an important limitation of existing physical reasoning benchmarks, which often emphasize object motion, collisions, and spatial relationships. Evaluating reasoning over interacting continuous physical fields is a potentially useful direction. The use of fire dynamics provides a reasonably complex environment involving temperature, airflow, smoke propagation, and interventions.

  • The authors construct a relatively large benchmark covering diverse environments, physical reasoning tasks, and observation modalities. The evaluation includes both structured and visual observations, as well as choice-based and open-ended questions.

Critical weaknesses
  1. Insufficient evidence that the benchmark genuinely measures complex physical-field understanding.
  • My primary concern is the construct validity of the benchmark. While the paper argues that understanding complex physical worlds requires reasoning about interacting physical fields, the experimental design does not convincingly demonstrate that the proposed questions actually require such capabilities.

Several qualitative examples appear answerable using general physical knowledge or contextual cues rather than detailed interpretation of physical-field observations (Figure 9 & 11). These outcomes may be inferred from relatively basic physical intuition without requiring the model to reconstruct or reason over the underlying coupled fields.

  • More importantly, the paper does not sufficiently demonstrate whether model performance depends on access to the physical-field information itself. A critical missing ablation would be to remove the field visualizations and sensor observations while retaining the scenario description and question.
  1. Insufficient justification of the capability taxonomy and uncertainty regarding ground-truth mechanism generation.
  • The paper proposes five physical reasoning capabilities (P1–P5), but the rationale for selecting these particular dimensions is not sufficiently developed. It is unclear whether the five capabilities are intended to constitute a complete characterization of complex physical-world intelligence. Why exactly these 5 dimensions, rather than 7or 10? The paper lacks a clear theoretical justification for why these dimensions are both necessary and sufficient.

  • Furthermore, these capabilities are not necessarily independent. Temporal forecasting (P1), for example, may inherently require understanding interactions between physical fields (P3) and causal mechanism attribution (P4). Similarly, P4 and counterfactual intervention reasoning (P5) are closely related, as predicting the effects of interventions often depends on identifying the underlying causal mechanisms. The paper would benefit from a clearer conceptual justification of the taxonomy and an explanation of how the dimensions are meaningfully distinguished during evaluation.

Questions for authors
  • How are ground-truth causal mechanisms and explanations generated from simulation outputs? Could the authors clarify what the specific mapping from numerical physical-field observations to causal explanations is?
  • What motivates the choice of theseP1-P5 as the evaluation framework for complex physical-world intelligence?
  • I am not fully convinced that the benchmark questions cannot be answered using general physical knowledge. An ablation study that removes the physical-field visualizations and sensor observations would be helpful.
3 comments
AuthorsOpenReview ↗

Response to Reviewer nbLj (1/2)

We sincerely thank the reviewer for the constructive comments.

W1. The construct validity of the benchmark

➡️The proposed tasks require explicit reasoning over physical fields. During dataset construction, we design questions that require reasoning about latent physical states or cross-field relationships. Solving them therefore requires explicit use of physical-field evidence; basic intuition or general knowledge alone is insufficient. Furthermore, Appendix Table 20 shows that our physics-aware prompting method improves answer accuracy by 0.7–7.8% through explicit incorporation of physical-field information into reasoning.

➡️Removing the field visualizations and sensor observations reduces task accuracy. We additionally report four models' task performance after removing the field visualizations and sensor observations. The results in Table 1 demonstrate the critical role of physical-field information in solving our benchmark tasks.

Table 1. Sensor and field observations ablations on 750 items, with 150 per capability. Cells report choice / open-report Acc (%).

Model / trackbest single fieldfull inputw/o sensor observationsw/o field visualizationsw/o both
GPT-5.6-Luna / S46.2 / 36.457.2 / 46.831.4 / 24.828.9 / 23.426.1 / 20.7
GPT-5.6-Luna / I44.7 / 35.854.3 / 45.131.4 / 24.828.9 / 23.426.1 / 20.7
Gemini-2.5-Flash / S42.6 / 35.751.6 / 43.728.6 / 23.126.7 / 22.424.8 / 19.6
Gemini-2.5-Flash / I41.5 / 35.049.2 / 42.328.6 / 23.126.7 / 22.424.8 / 19.6
Claude-Haiku-4.5 / S42.1 / 35.250.8 / 42.929.8 / 24.027.6 / 22.925.1 / 20.4
Claude-Haiku-4.5 / I39.6 / 34.146.7 / 40.629.8 / 24.027.6 / 22.925.1 / 20.4
Qwen3.5-Plus-02-15 / S41.8 / 35.450.1 / 43.527.3 / 22.625.1 / 21.722.9 / 18.8
Qwen3.5-Plus-02-15 / I39.5 / 34.646.5 / 41.227.3 / 22.625.1 / 21.722.9 / 18.8

W2/Q2. Justification and overlap of P1–P5

➡️Rationale and theoretical foundations for the P1–P5 taxonomy. We aim to evaluate complex physical-world intelligence through physical-field reasoning, with five tasks progressing from prediction without field maps to single-field understanding, joint multi-field reasoning, causal attribution of observed field changes, and counterfactual prediction of coupled-field responses to interventions. We intend these five capability dimensions to cover as broad a range of complex physical-world scenarios as possible while revealing capability-specific bottlenecks in model understanding that a single aggregate score may obscure.

Our design also draws on the insights and limitations of prior physical reasoning benchmarks [1][2][3], broadening the evaluation focus from object-centric dynamics and latent physical properties to coupled-field reasoning for evaluating complex physical-world intelligence. Moreover, task categories in most existing physical reasoning benchmarks [4][5][6] can be mapped onto our five capability dimensions, providing a taxonomy that is largely independent of scenario granularity.

➡️Capability-targeted construction and reasoning-specific evaluation. We address capability overlap at two levels. At the data-construction level, each question is designed so that answering it correctly depends primarily on the target capability, clearly defining what the question evaluates. At the response-evaluation level, we examine whether open-ended explanations explicitly show the corresponding reasoning, providing an additional check that the targeted capability is reflected in the model’s response.

➡️The capabilities assessed by P1–P5 are largely independent. For each capability dimension, we first provide models with correct outputs from earlier dimensions, then evaluate their performance on the current dimension. The results in Table 2 suggest that each dimension probes distinct reasoning demands beyond the capabilities assessed by preceding dimensions.

Table 2. Accuracy and change in error rate after providing correct answers to preceding dimensions. 300 matched tasks. Accuracy is reported as a percentage. Error-rate change is computed as after minus before.

CapabilityW/o Prior capabilitiesWith Prior capabilitiesError-rate change (pp)
P155.6360.83−5.20
P247.0852.08−5.00
P342.0847.92−5.84
P440.4243.75−3.33
P537.7140.21−2.50
AuthorsOpenReview ↗

Response to Reviewer nbLj (2/2)

Q1. Generation of Ground-Truth Causal Mechanisms

Controlled changes in physical conditions and simulator response differences jointly ground causal mechanisms and explanations. We configure or modify physical conditions in the simulator to produce observable effects of the relevant mechanisms. We then apply physical rules to the simulation outputs to determine mechanism labels and use predefined templates to generate corresponding explanations. In summary, Ground-truth causal mechanisms and explanations are derived by comparing simulator responses before and after controlled changes to physical conditions, which can be configured through simulator parameters. Real-world cases require an additional step: reconstructing the physical conditions in FDS from real-world observations.

Q2. Motivations of the choice of P1-P5

As discussed in our response to W2, we justify the choice of P1–P5 for evaluating complex physical-world intelligence from three perspectives. First, we define five progressive capability dimensions through the unified lens of physical-field reasoning. Second, the taxonomy accommodates tasks from most existing physical reasoning benchmarks while remaining largely independent of scenario granularity. Third, our experiments show that each dimension indeed assesses a distinct reasoning capability, helping identify bottlenecks in reasoning over physical fields.

Q3. General physical knowledge is insufficient to solve complex tasks

As shown in W1, removing either physical-field visualizations or sensor observations reduces task accuracy. We additionally evaluate the removal of both. Table 1 shows a further accuracy decline when field visualizations and structured field observations are removed together, indicating that general physical knowledge alone, without task-specific field evidence, is insufficient to solve these complex tasks. Meanwhile, during benchmark construction, we ensure that solving each task requires the physical-field reasoning capability targeted by its designated dimension.

We thank the reviewer again and would be glad to clarify anything further.

[1] Yi et al., 2020, CLEVRER: Collision Events for Video Representation and Reasoning, ICLR.

[2] Baradel et al., 2020, CoPhy: Counterfactual Learning of Physical Dynamics, ICLR.

[3] Bear et al., 2021, Physion: Evaluating Physical Prediction from Vision in Humans and Machines, NeurIPS.

[4] Tung et al., 2023, Physion++: Evaluating Physical Scene Understanding that Requires Online Inference of Different Physical Properties, NeurIPS.

[5] Chow et al., 2025, PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding, ICLR.

[6] Tang et al., 2026, Causal Scaffolding for Physical Reasoning: A Benchmark for Causally-Informed Physical World Understanding in VLMs, KDD.

Dear Reviewer nbLj,

We sincerely appreciate your constructive suggestions to clarify the motivation behind the P1–P5 taxonomy and add observation ablation studies. We fully agree with the importance of both points, which have helped us strengthen the conceptual justification and empirical support of our benchmark. We have revised the submission accordingly.

Please let us know whether any concerns remain or new questions have arisen following our responses and revisions. We would welcome the opportunity to discuss these further during the remaining discussion period.

Thank you again for your thoughtful and helpful feedback.

Best regards,
The Authors

Reviewer_ckZyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper introduces FireWorldBench, a benchmark for physical reasoning involving fire, smoke, temperature, airflow, and other coupled fields. It contains 520 fire-world entries and 9,074 QA pairs, covering physical-state understanding, prediction, causal reasoning, and interventions.

Critical strengths
  • The overall structure is clear, but some figures are quite crowded, especially Figures 1 and 2. I like the coupled-field setting. It's quite different from the usual object-centric physical reasoning benchmarks.

  • Using fire dynamics is a good choice. The benchmark covers interactions between several physical fields rather than only simple object motion.

  • The evaluation is fairly comprehensive, covering 24 models and multiple input modalities. The comparison between structured observations and visual field information is particularly useful.

Critical weaknesses
  • Only 26 of the 520 worlds are real-world-aligned, and their dense physical fields still come from FDS reconstruction. It's hard to tell how representative the results are of actual fire scenarios.

  • Figure 8 reports possible answer leakage in 8/300 audited questions and future-information leakage in 4/300. These are small numbers, but they matter for a benchmark intended to

Questions for authors

NA

2 comments
AuthorsOpenReview ↗

Response to Reviewer ckZy

We thank reviewer ckZy for the valuable comments.

W1. Real-data scale and effectiveness of FDS reconstruction

➡️More real-world fire scenarios that do not rely on FDS reconstruction. We expand the benchmark’s real-world data collection with new question–answer instances constructed directly from recorded sensor data and surveillance videos, without relying on FDS reconstruction [1]. Tables 1 and 2 show that the new data cover multiple physical reasoning tasks that current models struggle to solve, with performance declining further when sensor and video observations are removed.

Table 1. Data composition and capability coverage of MmodalFire-300 v1.0.0, with 54 native source identifiers and 338 PNG assets.

CapabilityAdded QAs
P1: Temporal forecasting36
P2: Field perception and grounding72
P3: Cross-field coupling120
P4: Mechanism attribution24
P5: Counterfactual intervention48
Overall300

Table 2. Four models' evaluation on the added 300 choice questions; Δ is observations plus question minus question-only, with 95% CIs in brackets.

ModelMatched QAsObservations + question Acc (%)Question-only Acc (%)ΔAcc (pp)Observations + question Macro-F1 (%)
GPT-5.6-Luna30049.028.021.0 [13.3, 29.0]49.0
Gemini-2.5-Flash30045.026.019.0 [11.3, 26.7]45.1
Claude-Haiku-4.530043.026.017.0 [8.7, 25.0]42.8
Qwen3.5-Plus-02-1530046.027.019.0 [11.3, 27.0]45.9

➡️Demonstration of the effectiveness of FDS reconstruction. We additionally compare FDS reconstruction results with real measurements to validate temperature and smoke response magnitudes, curve shapes, and onset times. The results in Tables 3 and 4 demonstrate that the reconstructed data reproduce recorded responses and their temporal changes at the measured locations, indicating strong consistency with real-world observations.

Table 3. Reconstruction validation on 17 held-out events and 34 response curves; values are mean ± SD.

Response channelHeld-out eventsCurvesNormalized MAE ↓Pearson r ↑
Temperature response17170.190 ± 0.0500.790 ± 0.090
Smoke response17170.205 ± 0.0550.820 ± 0.085
Both channels combined17340.198 ± 0.0520.805 ± 0.088

Table 4. Held-out event-onset timing validation under the fixed time mapping; MAE is in seconds.

Event endpointMatched eventsTiming MAE ↓95% CI
Temperature-rise onset175.8[4.0, 7.9]
Smoke-response onset177.1[5.0, 9.5]

W2. Answer leakage and future-information leakage

➡️Limited leakage scope. Guided by issues (answer leakage and future-information leakage) identified during expert review, we audit the full dataset and find that leakage affects only a small subset of questions as shown in Table 5.

Table 5. Leakage counts, rates, and task distribution in the original 9,074 questions, including 52 questions reached through shared-input/template risk paths; eight P5 questions have both leakage types.

TaskAudited QAsAnswer-exposure QAsFuture-information QAsUnique affected QAsAffected (%)
P1 Temporal forecasting2206044441.99
P2 Field perception and grounding21300000.00
P3 Cross-field coupling2708560562.07
P4 Mechanism attribution1130280282.48
P5 Counterfactual intervention9002812323.56
Overall, deduplicated9074112561601.76

➡️Excluding the affected questions leaves our main conclusions unchanged. We additionally conduct a full leakage audit and compare performance before and after excluding affected questions and repairing their inputs. The results in Table 6 show that excluding the affected questions has little impact on overall performance, while current models still struggle with these tasks.

Table 6. Maximum performance decreases after exclusion and repair, in percentage points. Exclusion evaluates overall performance after removing 160 affected questions. Repair evaluates performance on 144 repaired questions.

ModelMax Acc decrease after exclusionMax Macro-F1 decrease after exclusionMax Acc decrease after repairMax Macro-F1 decrease after repair
GPT-5.6-Luna0.3570.4785.74.4
Gemini-2.5-Flash0.3370.5496.06.4
Claude-Haiku-4.50.3470.7035.46.9
Qwen3.5-Plus-02-150.3020.5286.25.9

We thank the reviewer again and would be glad to clarify anything further.

[1] Jia et al., 2026, MmodalFire: A Continuous Multimodal Dataset Comprising Video and Physical Sensing Data for Detecting Indoor Fires, Scientific Data.

Dear Reviewer ckZy,

We have added evaluations on real fire data without FDS-based latent-state reconstruction, clarified the scope of the two expert-identified issues, and re-evaluated models after excluding the affected data. Your feedback has been particularly valuable in strengthening our benchmark’s coverage and quality.

Please let us know whether any concerns remain or new questions have arisen. We would be glad to discuss them in the remaining time.

Best regards,
The Authors

Comments on the paper

1 comment
AuthorsOpenReview ↗

Summary of Revisions

Dear Reviewers and Area Chair,

As the discussion period draws to a close, we have revised our submission to address the concerns raised during rebuttal.

  1. We added evaluations on real-world questions grounded directly in recorded sensor data and videos, without relying on FDS-based latent-state reconstruction, expanding our coverage of actual fire scenarios. (For Reviewer ckZy\textcolor{blue}{\text{ckZy}}.)

  2. We clarified the motivation for the P1–P5 taxonomy and added observation ablations demonstrating the importance of structured sensor readings and field visualizations for solving physical-field reasoning tasks. (For Reviewer nbLj\textcolor{blue}{\text{nbLj}}.)

  3. We systematically analyzed all failure cases, identified four recurring failure patterns, and strengthened explanation-metric validation through comparisons with expert judgments and robustness analyses using physical-relation-aware metrics. (For Reviewer a2wE\textcolor{blue}{\text{a2wE}}.)

We sincerely thank the reviewers for your constructive feedback, which has helped improve the clarity, rigor, and empirical support of our work. We also sincerely appreciate the substantial effort the AC has dedicated to handling our submission.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.