acceptodds
Desk rejected at ICLR 2027

WildfireIA: Can Environmental Data at Fire Discovery Time Predict Whether a Wildfire Will Escape Initial Attack?

Desk rejected at ICLR 2027, so left out of the paper lists and the map.

Abstract

Initial attack (IA) refers to the first suppression actions taken to contain or extinguish a wildfire following its detection. Existing IA failure prediction studies often use non-public response records or regional settings, so it remains unclear how well public data aligned to the reported discovery day can support IA failure prediction at national scale. We present WildfireIA, a novel U.S. national-scale benchmark for IA failure prediction under a discovery-day information contract using public environmental and contextual data. WildfireIA aligns 38,128 naturally caused FPA-FOD wildfire events with FIRMS/VIIRS thermal detections, gridMET weather and fire-danger variables, LANDFIRE vegetation, fuel, and topography, OpenStreetMap access features, and WorldPop population density. To prevent data leakage, the benchmark fixes the event unit, size-based label rule, chronological split, metrics, and forbidden-feature list, and excludes final fire size, containment timestamps, and D+1 and later satellite detections from model inputs. We evaluate 16 representative models across tabular, temporal, spatial, and spatiotemporal families under the same protocol. Results show that public discovery-day data provides useful but incomplete signal for IA failure prediction: XGBoost achieves the best AUPRC of 53.3%; FIRMS/VIIRS is the least redundant source; and fuel is the strongest static predictor when dynamic observations are unavailable. We release preprocessing outputs and model-ready caches to support reproducible research on early wildfire risk assessment: https://github.com/WildfireIA-anonymous/WildfireIA.

Primary area:
datasets and benchmarks
Keywords:
Wildfire prediction, initial attack, IA failure prediction

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_RXBjOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work establishes a novel national benchmark for predicting initial attack (IA) failures from publicly available environmental and contextual data under a discovery-day information contract. The benchmark aligns 38,128 naturally ignited FPA-FOD forest fire incidents with FIRMS/VIIRS thermal anomaly detections, gridMET weather and fire-danger variables, LANDFIRE vegetation, fuel, and terrain layers, OpenStreetMap accessibility features, and WorldPop population data. Under a single evaluation protocol, it benchmarks 16 representative models in terms of tables, time, space, and the family of spacetime.

Critical strengths
  1. It builds a novel national benchmark to predict IA failures based on publicly available environmental and contextual data under the discovery day information contract.

  2. This work may be useful to the community due to the contract. The wildfire machine learning literature is filled with work that leaks results implicitly, but this paper clearly defines event units, label rules, time sequence division, prohibited feature lists and alignment rules between discovery days, and explains the reasons for each prohibition, establishing a referable standard for this direction.

  3. The experiments are meaningful, with 16 baselines across four representation families and six classification metrics, including Brier and ECE.

  4. The work releases preprocessing output, model-ready cache, configuration, checkpoints, and an end-to-end reproducible path has also been provided.

Critical weaknesses
  1. The LANDFIRE year used for each fire year should be indicated. It should be ensured that the disturbed year is earlier than the discovery date; the same applies to the OSM snapshot. If leakage is confirmed, a new experiment is required.

  2. The paper takes event-level triage as the application scenario, but the "utility" has not been compared with any real-world baseline: neither simple heuristic rules such as fire risk index thresholds were compared, nor physical spread models like FARSITE/FlamMap were compared, nor were decision analysis metrics provided.

  3. The generalization ability is insufficient. Only a single test year (2020, an abnormal year for fire activity) was used, and no spatial separation experiments were set up.

Questions for authors
  1. The LANDFIRE year used for each fire year should be indicated. It should be ensured that the disturbed year is earlier than the discovery date; the same applies to the OSM snapshot. If leakage is confirmed, a new experiment is required.

  2. The paper takes event-level triage as the application scenario, but the "utility" has not been compared with any real-world baseline: neither simple heuristic rules such as fire risk index thresholds were compared, nor physical spread models like FARSITE/FlamMap were compared, nor were decision analysis metrics provided.

  3. The generalization ability is insufficient. Only a single test year (2020, an abnormal year for fire activity) was used, and no spatial separation experiments were set up.

3 comments
AuthorsOpenReview ↗

Respond to Reviewer RXBj

Thank you for the thoughtful review. We have updated the PDF and addressed your concerns as below:

[W1/Q1] Historical Validity of LANDFIRE and OpenStreetMap.

We use LF2016 for fuel, vegetation, and disturbance features, and LF2020 only for topography, including elevation, slope, and aspect. LF2016 includes disturbances through 2016, while some fuel layers reflect modeled recovery toward 2019–2020 [1]. These layers may therefore contain information unavailable at the time of earlier fires. However, elevation, slope, and aspect are relatively stable terrain attributes that LANDFIRE rarely updates [2]. We therefore use LF2020 topography for wildfire events from 2016 to 2020.

For OpenStreetMap (OSM), we address historical alignment by specifying snapshots from January 1 of each fire year. These snapshot dates precede the corresponding fire discovery dates. We now clarify these source choices and their timing limitations in Section 2.2.

To assess whether our results depend on these sources despite the timing limitations, we conducted an additional ablation. We first excluded all LANDFIRE features and then also excluded OSM, keeping all other inputs and evaluation settings unchanged. Five-seed test AUPRC (%, mean±std):

InputXGBoostTransformerSwin-UNetSwinLSTM
Full53.3±0.350.6±0.351.1±1.251.7±1.2
No LANDFIRE48.9±0.448.6±0.548.6±0.850.5±0.6
No LANDFIRE/OSM48.6±0.447.1±0.646.0±1.647.3±0.6

All four models retain predictive signal after both exclusions. XGBoost achieves 48.6% AUPRC compared with 53.3% under full input. These results show that prediction does not depend entirely on LANDFIRE and OSM, although historical timing concerns remain for the full-source setting. Further details are provided in Appendix C, Table 17.

[W2/Q2] Operational Baselines and Decision-Relevant Evaluation.

To compare our models with established fire-danger measures, we added Energy Release Component (ERC) and Burning Index (BI) baselines. We used the same 6,168 test fires, including 513 positive cases, and measured ranking and recall at fixed review budgets. Results (%):

MetricERCBIXGBoostTransformerSwin-UNetSwinLSTM
AUPRC14.214.353.3±0.350.6±0.351.1±1.251.7±1.2
AUROC66.762.487.1±0.184.0±0.485.3±0.685.7±0.9
Recall@5%9.912.338.0±0.636.5±0.336.5±1.837.3±1.3
Recall@10%17.321.253.5±0.650.6±0.350.8±1.051.7±1.5

At a 5% review budget, XGBoost retrieves 38.0% of positive cases, compared with 9.9% for ERC and 12.3% for BI. These results support better event-level ranking under our protocol. Actual dispatch benefits remain untested. FARSITE and FlamMap simulate fire growth [3,4]. A direct comparison would require simulation inputs, a prediction horizon, and a rule linking simulated fire behavior to our labels. Further details are in Appendix C, Table 13.

[W3/Q3] Generalization Across Years and Regions.

We agree that one test year cannot establish performance across seasons. To assess test-year sensitivity, we added a forward-time evaluation using 2016–2017 for training, 2018 for validation, and 2019 for testing. We compare it with the original 2020 test. Five-seed test AUPRC/AUROC (%, mean±std):

Test yearXGBoostTransformerSwin-UNetSwinLSTM
201945.9±0.5/87.3±0.244.2±0.4/85.5±0.341.5±2.1/82.3±1.142.7±1.8/85.4±0.6
202053.3±0.3/87.1±0.150.6±0.3/84.0±0.451.1±1.2/85.3±0.651.7±1.2/85.7±0.9

The additional evaluation shows consistent predictive performance across two test years. XGBoost achieves the highest AUPRC in both years, with similar AUROC (87.3% in 2019 and 87.1% in 2020). The lower 2019 AUPRC (45.9% versus 53.3%) also reflects differences in positive rates (6.1% versus 8.3%) and training periods. These results provide further evidence of the benchmark's predictive value across fire seasons.

Both evaluations use chronological splits across U.S. wildfire events. This design assesses performance across years, while a geographically disjoint split would further assess transfer to unseen regions. We report the full five-seed results in Appendix C, Table 12.

References [1] LANDFIRE. LF2016 Remap Project Closeout Report. LANDFIRE. [2] LANDFIRE. Topographic Products. LANDFIRE. [3] Finney. FARSITE: Fire Area Simulator–Model Development and Evaluation. USDA Forest Service Research Paper RMRS-RP-4, 1998. [4] USDA Forest Service. FlamMap. Fire, Fuel, and Smoke Science Program.

Thanks for the response. The response addressed some concerns. However, the added generalization evaluation seems to be trained on a new training and validation dataset configuration, which did not use the same setup. It is better to test with the same model on different years. I will consider other reviewers' comments for further consideration. Thank you.

Response: Generalization across test years.

We thank the reviewer for the helpful clarification and suggestion to evaluate the same models across different test years. We have conducted additional experiments and updated the revised manuscript accordingly.

Our original benchmark uses 2016–2018 for training, 2019 for validation, and 2020 for testing. This chronological evaluation follows common practice in wildfire prediction studies, where the most recent available years are held out for testing [1, 2]. At the time of our original experiments, the FPA-FOD edition we used only covered events through 2020, which limited our evaluation to that test year.

Fortunately, an updated FPA-FOD edition was released in 2026, extending coverage through 2024. We have since processed the additional 2021–2024 events. The following table reports AUPRC / AUROC (%) as mean ± standard deviation across five seeds.

Test yearEventsPositive rateXGBoostTransformerSwin-UNetSwinLSTM
20206,1688.3%53.3±0.3 / 87.1±0.150.6±0.3 / 84.0±0.451.1±1.2 / 85.3±0.651.7±1.2 / 85.7±0.9
20216,5286.2%43.8±0.4 / 86.0±0.243.7±0.3 / 84.8±0.438.0±1.5 / 82.8±0.739.7±0.8 / 82.7±0.8
20226,5154.5%39.4±0.4 / 85.8±0.137.1±0.3 / 82.3±0.731.9±1.6 / 81.9±0.738.9±1.0 / 86.2±0.9
20236,3675.6%41.8±0.3 / 86.4±0.241.2±0.3 / 83.2±0.434.7±1.5 / 80.9±1.038.1±0.6 / 83.7±0.4
20247,2367.4%44.5±0.4 / 86.7±0.343.5±0.4 / 85.2±0.339.0±0.5 / 81.9±1.142.2±1.1 / 83.6±0.6

The original 2020 test year achieves the highest AUPRC, with XGBoost reaching 53.3%. Although AUPRC is lower in 2021–2024, these years also have lower positive-class prevalence (4.5–7.4%, compared with 8.3% in 2020), which can affect AUPRC. Importantly, XGBoost maintains relatively stable AUROC (85.8–87.1%) across all five years, suggesting that its discrimination ability is not limited to the unusual 2020 fire season. We acknowledge that performance varies across years, particularly for the spatial models.

We have incorporated this additional temporal generalization experiment and its results into Appendix C, Table 12 of the revised manuscript.

References [1] Kondylatos et al. Mesogeos: A Multi-Purpose Dataset for Data-Driven Wildfire Modeling in the Mediterranean. NeurIPS, 2023. [2] De Rango et al. CalWildFire: A High-Resolution In Situ Dataset for Wildfire Analysis and Modeling in the Mediterranean Region. Scientific Data, 2026.

Reviewer_VmSxOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces WILDFIREIA, a U.S. national-scale benchmark dataset from public multi-source data associated with fire events for predicting whether an event will escape initial attack. The benchmark identifies 38,128 natural FPA-FOD wildfire events from the period 2016-2020 and aligns them with collocated thermal signals from satellite observations (FIRMS/VIIRS), weather data (gridMET), fuel, vegetation and topography data (Landfire), access data such as roads and fire stations (OpenStreetMap) and population density data(WorldPop). The classification labels are constructed using a rule based on the reported burnt area that is identified for each event in FPA-FOD data: events with ≤ 10ha are marked as successful, events with ≥ 50ha are marked as failed and all intermediate values are unlabeled but kept in the provided dataset.

The main contributions are an aligned multi-source dataset from publicly available data, along with the construction pipeline and four model-ready representations (tabular, temporal, spatial, spatiotemporal). The authors also investigate several aspects of the dataset, including label thresholds, data alignment, regional performance and potential information leakage. 16 models are evaluated on the binary classification task using different architecture-specific representations. A detailed investigation of the feature-importance is conducted and the relevance of these data for predictions at discovery time is also examined.

From the models, XGBoost achieves the highest reported AUPRC of 53.3% while authors report that FIRMS/VIIRS provide the most informative input for the prediction, fuel is a useful static predictor and longer weather histories provide limited improvement.

Critical strengths

The paper introduces a valuable multisource and reproducible benchmark of publicly available data to investigate how and if they can answer a very critical question: will this fire event escape initial attack? The alignment of the heterogeneous data, the extensive analysis of their importance and the thorough guidelines on the prevention of target information leakage and the released code that also includes the model-ready representations are important strengths. The authors follow a clearly defined experimental protocol and provide the necessary tools and documentation to facilitate further research using WILDFIREIA.

The evaluation and assessment of the importance of each component of the dataset is quite extensive including source-removal ablations, static-source comparisons, weather-history length examination and metrics specific to the particularities of the task such as the imbalance of the positive/negative cases. The authors also provide an extensive limitations section which is quite informative and analytical and justifies a lot of the decisions taken throughout the dataset construction. The definition of the auxiliary stress-test regression task of predicting containment-duration complements nicely the previous outcomes and serves well the purpose of exploring the usefulness/limitations of the publicly available data in fire prediction and incident severity estimation tasks.

Critical weaknesses

Leakage control: The authors of the paper emphasize on the techniques they adopt to control leakage of labels to the input data that would lead to falsely improved performance of the evaluated models. Although some important steps are identified, some concerns still exist related to observations made after the discovery time of the event: Weather data: Weather data include daily variables also from the discovery day but not limited up to the discovery time. Therefore some information from the future is taken into consideration. This could potentially be addressed using weather observations or forecasts available at the prediction time (as an analog) , although this would require a different input protocol. FIRMS/VIIRS channels: Discovery-day FIRMS/VIIRS features are allowed. In Appendix C , Table 8, the performance of the model is assessed when conditioned on same day VIIRS data vs VIIRS data constrained to be taken in pre-discovery time vs no VIIRS data. The outcome that is reported is that the metrics calculated when VIIRS data are not provided and when VIIRS data from pre-discovery time are provided are similar and the main improvement in the results comes from the same-day VIIRS data. The authors conclude that this is an indication that FIRMS/VIRS should be interpreted as a discovery-day source. However, these results can also suggest that much of the improvement comes from observations acquired after discovery, which would not be normally available for a prediction made at discovery time. Label construction: The rule that is established for the label creation remains uncertain. Although the authors perform a sensitivity analysis using alternative thresholds for the binary classification problem, this does not establish whether the assigned labels correspond to the actual success or failure of initial suppression efforts. It would be useful to assess this correspondence using some events for which the operational outcomes are known.

Questions for authors

On label quality: Is there evidence on the validity of the 10/50 ha threshold rule even for a smaller subset of events, that could validate the 10/50 ha labels against observed initial attack outcomes?

On ensuring the assessment of the importance of information hidden in FIRMS/VIRS:When constructing the FIRMS/VIIRS input, a detection indicator is also provided. This indicator shows whether there are available observations for the fire event that satisfy the predefined colocation criteria that make an observation appropriate and relevant for consideration. It is provided complementarily along with the actual or zero filled (in case of unavailability) values. Looking at Figure 5 from Appendix C and as is also pointed out by the authors, VIIRS detections are substantially more frequent for large fires. This could lead to the implemented detection indicator dominating the predictions. Could the authors provide an ablation using only the VIIRS detection indicator, without the thermal measurements? This would help clarify whether the reported VIIRS contribution comes mainly from satellite observability or from the actual thermal information the importance of which is highlighted throughout the paper.

On model evaluation: Since the different model-ready representations might contain different information how do you ensure that the comparison of different models is fair and does not come from differences in the input information? For example the spatiotemporal representation based on the configurations available in the released code might not contain metadata such as latitude, longitude which might reveal spatial dependencies for the events, while the tabular and temporal representations do.

2 comments
AuthorsOpenReview ↗

Respond to Reviewer VmSx

Thank you for the thoughtful review. We have updated the PDF and addressed your concerns as below:

[W1] Leakage Control: Weather, VIIRS Timing, and Labels.

(1) Weather. We agree that day-D gridMET aggregates may contain observations after discovery and overstate performance for forecasts made at the discovery hour. Observations acquired earlier may also become available only after a release delay. We use an offline discovery-day protocol with weather from D−4 through D. Sections 2.1 and 4 now explain this timing limit.

(2) FIRMS/VIIRS timing. VIIRS observes fires during discrete satellite overpasses, and detections require processing before release. NASA FIRMS reports that global near-real-time detections are generally available within three hours of acquisition [4]. Our existing timing analysis (Appendix C, Table 8) supports the reviewer's concern. XGBoost AUPRC is 53.3% with same-day VIIRS, 38.1% with pre-discovery acquisitions, and 38.3% without VIIRS. Only 293 of 1,738 matched events have pre-discovery acquisitions. Much of the gain therefore comes from later observations. This test still uses day-D weather and does not establish discovery-hour performance.

(3) Label construction. NWCG defines IA [5]. Plucinski [6], Korkola et al. [2], and Cardil et al. [8] examined final fire size in IA containment research. Following Rodrigues et al. [7] and Xu et al. [1], we use 10/50 ha thresholds, with label 0 for fires ≤10 ha, label 1 for fires ≥50 ha, and intermediate fires excluded. Labels come from FPA-FOD final sizes [3], which are excluded from inputs. Our existing threshold analysis (Appendix C, Table 6) yields XGBoost AUROC of 85.4–87.1% across four separated rules. Section 2.1 now identifies the target as a size-based proxy.

[Q1] Validation Against Observed IA Outcomes.

We agree that recorded IA outcomes would strengthen validation. FPA-FOD has no consistent IA deployment or success field [3], and we have no linked subset to measure agreement. Prior studies support our proxy, while Appendix C, Table 6 tests threshold sensitivity. Across five area rules, XGBoost AUROC ranges from 85.4% to 87.1%. This evaluates threshold sensitivity. Agreement with actual suppression outcomes remains unmeasured. These studies support our choice of a size-based proxy. Section 2.1 now clarifies the missing operational IA records and the proxy-based target definition.

[Q2] VIIRS Detection Indicator vs. Thermal Measurements.

To separate detection availability from other VIIRS features, we ran an indicator-only ablation with other inputs and splits fixed. Five-seed test AUPRC (%, mean±std):

VIIRSXGBoostTransformerSwin-UNetSwinLSTM
Full53.3±0.350.6±0.351.1±1.251.7±1.2
Indicator49.6±0.345.9±0.245.4±1.248.7±1.1
None38.3±0.231.8±0.430.5±1.629.5±0.8

For XGBoost, the indicator adds 11.3 AUPRC points, and the other VIIRS features add 3.7. Full VIIRS performs best for all four models. These features include detection counts and thermal measurements, so FRP is not isolated. See Appendix C, Table 10.

[Q3] Fair Comparison Across Model Representations.

All models share events, labels, splits, and metrics, while representations differ. Swin-UNet and SwinLSTM already exclude explicit FPA-FOD metadata. We therefore tested XGBoost and Transformer after removing geographic fields, then all event metadata. Five-seed test AUPRC (%, mean±std):

ModelFullNo geoNo metadata
XGBoost53.3±0.353.4±0.453.1±0.3
Transformer50.6±0.350.4±0.450.2±0.3

AUPRC changes by only 0.2–0.4 points. XGBoost remains above Swin-UNet (51.1) and SwinLSTM (51.7). Other representation differences remain, so we compare complete model–representation pipelines. See Section 4, Appendix C, Table 14, and Appendix E, Table 18.

References [1] Xu et al. Modeling Wildfire Initial Attack Success Rate Based on Machine Learning in Liangshan, China. Forests, 2023. [2] Korkola et al. A comparative analysis of wildfire initial attack containment objectives and modelling strategies in Ontario, Canada. International Journal of Wildland Fire, 2024. [3] Short. Spatial wildfire occurrence data for the United States, 1992–2020 (FPA-FOD), Sixth Edition. USDA Forest Service Research Data Archive, 2022. [4] NASA FIRMS. Fire Information for Resource Management System. NASA Earthdata. [5] National Wildfire Coordinating Group. Initial Attack (IA). NWCG Glossary of Wildland Fire. [6] Plucinski. Factors Affecting Containment Area and Time of Australian Forest Fires Featuring Aerial Suppression. Forest Science, 2012. [7] Rodrigues et al. Modeling initial attack success of wildfire suppression in Catalonia, Spain. Science of the Total Environment, 2019. [8] Cardil et al. Assessing the suppression difficulty of wildland fires for initial attack response. International Journal of Wildland Fire, 2025.

Thank you for the response and for addressing all points raised. The additional experiments help clarify some details. Concerning the contribution of the VIIRS availability indicator vs the actual thermal measurements and the examination of the fair comparisons of the models, the ablations prove that the initial statements of the paper are in the right direction. I also appreciate the refinements on the text making it more precise. Although some limitations of the size-based labels remain still relevant, I believe the revised paper is solid, relevant and would be interesting to the ICLR community. I therefore remain to my initial score Weak Accept.

Reviewer_EgUXOpenReview ↗
Rating
1
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The authors construct a new dataset and associated benchmark problem, termed WildfireIA, focused on the prediction of wildfire "initial attack" (IA) success. The authors describe the details of the benchmark construction and composition, and the scientific questions that the benchmark aims to address. The authors also attempt to provide initial responses to these questions by running a large number of existing machine learning models on the benchmark.

Critical strengths
  1. This benchmark addresses an interesting set of questions, and is distinct from other benchmarks, and represents a challenging spatio-temporal remote sensing task with somewhat unique machine learning challenges, that is also societally beneficial.

  2. The curation of the data is useful and important contribution.

  3. The initial collection of models run on the data provides a lot of evidence to address some of the scientific questions.

Critical weaknesses
  1. Potentially Mismatch between Benchmark and Stated Scientific Goals. Based upon the content in the paper, it is unclear to me whether this benchmark actually addresses the main scientific question: that initial attack (IA) (i.e., firefighting) was successful or not. It appears that the dataset may actually address a different question: which fires grow beyond a certain size, and which ones don't, irrespective of firefighting. If so, this is still potentially a useful dataset, this needs to be made more clear in the paper, and the paper needs to be reframed around this different question. See question 3 below.

  2. Poor clarity. The overall structure of the paper is good, but some of the content in the paper suffer from very low clarity, making it difficult to evaluate the work. For example, there is a lot of jargon used throughout the paper, and this obscures the meaning in some key places. I'm somewhat familiar with this topic and I still don't follow the content in many places, including some important points (see Questions below). The authors should revise the paper for clarity for a somewhat more general audience, as well as elucidating some key points I raise in my questions below.

Questions for authors

I'm willing to adjust my scores upwards if some of my questions below can be addressed with clear and convincing responses.

  1. Problems with Clarity. This is not an exhaustive list. a) The FPA-FDO is discussed before it is defined, making the paper much harder to read (e.g., Line 70). There are several instances of this throughout the paper.
    b) Another example is on line ~125: what are "source groups" and "source subsets"? These terms are defined later, making it difficult to follow the explanation. Also, what type of data is in these sets, and their dimensionality? e.g., real numbers, integers? Do they have spatial or temporal dimensions? As far as I can tell, this is never defined?
    c) Another example is Figure 2. No clear description is provided either in its caption, or in the main text, leaving the construction of the dataset largely unclear. The figure uses symbols that are not defined where the figure is referenced on line 245, or ever in some cases, as far as I can tell.
    d) What is a "Canonicalization Pipeline" on line 243? Do you mean "construction pipeline", which is the language used in the very next sentence. Typos and/or language inconsistencies in section headings are unacceptable.

  2. The ground truth labels come from the final burned area of the fire, but where does this information come from? Is it from FPA-FDO? If yes, how do they derive this?

  3. Do we actually have any information about whether an "initial attack" (i.e., firefighting) occurred on each fire in the dataset? If yes, how do we know? Is it possible that it is a mix of fires that received an IA, and others that did not, but we don't know which ones? If we're not sure for any particular fire, do we know what proportion of fires received an IA? How do we know? If the fires did not receive any IA, then it seems like the goal of the benchmark is really to predict whether a fire is likely to grow beyond some particular size: not whether IA was successful: correct?

  4. My understanding is that the input features are spatio-temporal. How did the authors apply tabular models to this data? e.g., did you rasterize the entire spatio-temporal dataset? What was the dimensionality of the full input data? As noted earlier, this is never mentioned anywhere.

1 comment
AuthorsOpenReview ↗

Respond to Reviewer EgUX

Thank you for the thoughtful review. We have updated the PDF and addressed your concerns as below:

[W1] Potential Mismatch Between the Benchmark and Stated Scientific Goals.

We agree that the target needs clarity. Final fire size is an established IA containment criterion used by fire agencies and prior studies. The NWCG defines initial attack [1]. Plucinski [2] examined 334 Australian wildfires using area- and time-based IA success criteria. Korkola et al. [3] compared eight IA definitions across 26,171 Ontario fires and found similar model performance. Cardil et al. [4] also assessed IA success using reported final fire sizes in California. Building on IA studies in Spain [5] and China [6], we adopt Xu et al.’s 10/50 ha rule [6]. Fires ≤10 ha receive label 0, fires ≥50 ha receive label 1, and intermediate cases are excluded.

We now clarify this scope in the Abstract, Section 2.1, and Conclusion. Section 2.1 defines the labels as size-based proxies because FPA-FOD [7] does not record operational IA outcomes. Section 2.3 explains that final size constructs the labels and is excluded from model inputs. Appendix C, Table 6 tests alternative thresholds. XGBoost AUROC stays at 85.4%–87.1% across four threshold pairs, with 85.9% for the no-gap rule.

[W2] Clarity of the Benchmark Description.

We clarify the construction order. Discovery date and location link each FPA-FOD event to public sources. The canonical pipeline produces event tables, daily records, and spatial patches. The dataloader selects sources and builds four model-ready representations. Revised Figure 2 shows these stages and the chronological split. Sections 2.1–2.3 explain the terms and workflow. Section 4 and Appendix E, Table 18 give the inputs and dimensions.

[Q1] Terminology, Data Sources, and Construction Workflow.

(a) Section 1 expands FPA-FOD as Fire Program Analysis Fire-Occurrence Database. It provides event records, metadata, and final fire sizes [7].

(b) Section 2.1 defines eight source groups: metadata (M), VIIRS (R), weather (W), vegetation (V), fuel (F), topography (T), access (A), and population (P). A subset selects groups, such as {M,W}. Sources include numeric and categorical fields, daily sequences, and raster/vector layers. Section 4 explains how models use them.

(c) Figure 2 now shows source alignment, canonical artifacts, dataloading, four representations, and evaluation. The source symbols are defined in Section 2.1.

(d) Canonical Pipeline refers to alignment and standardization. Benchmark construction covers the full workflow, including dataloading and evaluation. Section 2.3 and Figure 2 now use these terms consistently.

[Q2] Ground-Truth Label Construction.

Final area comes from FPA-FOD's FIRE_SIZE field, reported in acres from fire-agency records [7]. We convert acres to hectares using 0.404686 and apply the 10/50 ha rule. VIIRS does not supply the labels. Section 2.3 explains this source and confirms that final size is excluded from all model inputs.

[Q3] Did All Fires Receive Initial Attack?

We cannot confirm IA deployment for each fire or estimate its proportion because FPA-FOD has no standardized IA deployment field [7]. Some fires may not have received IA. Section 2.1 now states this limitation. The benchmark predicts the final-size criterion. Validation against operational IA outcomes would require linked response records.

[Q4] Tabular Representation and Input Dimensions.

Tabular models combine metadata, VIIRS features, point measurements, spatial summaries, and weather summaries without flattening patches. Temporal models retain daily sequences, and spatial models retain patches. Section 4 explains these inputs. Appendix E, Table 18 lists the full-source shapes: tabular N×6029, temporal N×5×15 plus N×5940, spatial N×121×29×29, and spatiotemporal N×5×47×29×29.

References

[1] National Wildfire Coordinating Group. Initial Attack (IA). NWCG Glossary of Wildland Fire, PMS 205. [2] Plucinski. Factors Affecting Containment Area and Time of Australian Forest Fires Featuring Aerial Suppression. Forest Science, 2012. [3] Korkola et al. A comparative analysis of wildfire initial attack containment objectives and modelling strategies in Ontario, Canada. International Journal of Wildland Fire, 2024. [4] Cardil et al. Assessing the suppression difficulty of wildland fires for initial attack response. International Journal of Wildland Fire, 2025. [5] Rodrigues et al. Modeling initial attack success of wildfire suppression in Catalonia, Spain. Science of the Total Environment, 2019. [6] Xu et al. Modeling Wildfire Initial Attack Success Rate Based on Machine Learning in Liangshan, China. Forests, 2023. [7] Short. Spatial wildfire occurrence data for the United States, 1992–2020 [FPA_FOD_20221014], Sixth Edition. USDA Forest Service Research Data Archive, 2022.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.