acceptodds
Under review as a conference paper at ICLR 2027

PyroAdapt: Adapting Wildfire Prediction under Spatial Heterogeneity and Temporal Shift

Abstract

Prediction of wildfire occurrence is a rare-event problem compounded by spatial heterogeneity and temporal distribution shift, as fire occurrences are vastly outnumbered by non-occurrences, and predictor–fire relationship varies across space and time. Models trained on historical fire data may perform poorly under new conditions and require adaptation to the target distribution before operational use. We propose PyroAdapt, a pretrain–retrieve–rank framework that adapts a pretrained model to target conditions by retrieving historical locations with similar conditions and fine-tuning on the retrievals through risk ranking. For spatial adaptation, we condition risk on terrain, ecoregion embeddings, and fire rates, accounting for spatial context in the retrieval, and learn risk ordering from same-day fire–nonfire cell pairs. We compare direct ranking, residual pairwise DPO (RDPO), and selective ranking through a unified score-gap formulation that characterizes their gradient allocation. Over California (discretized into 666 0.250.25 grid cells), these objectives raise daily average precision from 21.62% for continued focal fine-tuning to 24.35–24.57%, and Top5% recall from 18.70% to 22.11–22.79%. Under a fixed daily detection budget of 34 cells (5% area), selective ranking captures 344 additional positive cell–days. For fires in the top 5%/10%/20% of dry matter consumption, selective ranking raises recall by 39.70/28.18/20.50 percentage points, respectively. Furthermore, rolling evaluations over Yosemite show that the gains from ranking persist under temporal distribution shift. Together, these results show that PyroAdapt prioritizes the most fire-prone locations under a daily budget constraint and detects more extreme fire events

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
wildfire prediction, domain adaptation, pretraining and finetuning, preference learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_pnHZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PyroAdapt, a Pretrain–Retrieve–Rank framework for wildfire occurrence prediction under severe class imbalance, temporal distribution shift, and spatial heterogeneity. A risk model is first pretrained on historical data. For a target period, the unlabeled covariates of the whole target period are used to retrieve similar labeled historical examples, and the pretrained model is fine-tuned on positive–negative pairs from these examples through pairwise ranking losses, without any target labels. The authors compare three ranking objectives, direct ranking, RDPO, and selective ranking, within a unified score-gap formulation, and provide theoretical analysis of their gradient behavior and of a finite-sample bound on missed fire locations under a fixed daily budget. Experiments cover temporal adaptation in Yosemite and location prioritization across California.

Critical strengths
  1. The problem setting is practically relevant, and the same-day pairing fits it well. Prioritizing the most fire-prone locations under a fixed daily budget matches how monitoring capacity is actually used, and restricting pairs to the same day trains the model to rank locations within a date rather than dates against each other.
Critical weaknesses
  1. The gains are hard to attribute to retrieval and adaptation. The main comparison is continued focal fine-tuning with a pointwise loss against same-day pairwise ranking. Since the evaluation metrics also measure within-day ranking, the improvement may come from aligning the training objective with the metrics rather than from retrieval or the adaptation framework. Useful controls are missing, including the same ranking objective trained on the full historical data, ranking adaptation without retrieval, and a standard learning-to-rank baseline. It is also unclear whether XGBoost uses the same ecoregion and history features as the proposed model.
  2. The advantage of selective ranking over direct ranking and RDPO is not established. The differences are small relative to seed variability, and the gradient analysis suggests the three objectives behave similarly under the chosen temperature. Yet the abstract and contributions highlight the additional captures attributed to selective ranking, without confidence intervals or significance tests. The claims about selective ranking should therefore be qualified.
  3. The higher recall for extreme fires may reflect threshold differences rather than better discrimination. Each policy uses its own F1-optimal threshold, so alarm rates are not comparable. Recall increases while F1 remains largely unchanged, which is consistent with a shifted operating point. The abstract’s claim that the method detects more extreme fire events is therefore not fully supported. Recall at matched alarm rates would show whether the method finds more extreme fires at a comparable alert burden.
  4. The label is not equivalent to wildfire occurrence. It is a binary indicator derived from burned dry matter after substantial spatial smoothing, which raises the positive rate and makes neighboring locations spatially correlated. The number of captured positives may therefore not correspond to the number of distinct fire events. Evaluation using unsmoothed labels or event-level wildfire records is needed to determine whether the reported gains translate into better detection of actual fire events.
  5. The retrieval strategy relies on an untested coverage assumption. Nearest-neighbor retrieval is useful only if the archive contains samples similar to the target year’s conditions. The paper acknowledges this limitation but does not evaluate cases where target covariates fall outside historical coverage. The choice k = 5 is fixed without a sensitivity analysis, and a fixed k does not ensure the neighbors are representative. The retrieval ablation uses continued focal fine-tuning rather than the proposed ranking losses, so it does not isolate retrieval’s contribution to ranking adaptation. Retrieval also uses covariates from the entire target year, which raises a practical concern if forecasts are needed before those covariates are available.
  6. The theoretical results do not fully support the emphasis placed on them. Some results appear to follow from standard arguments, including the reduction of binary-label RDPO. The budget bound also applies to any nonnegative target, rather than specifically to selective ranking, and provides no guarantee for a future year. Calling it a “certificate” may therefore overstate what it establishes. The paper should clarify how its contribution differs from prior work on binary-label RDPO, particularly Jiang and Sun (2026).
  7. The experiments do not isolate temporal shift or spatial heterogeneity. Yosemite is a single study site, and the California evaluation covers only one test year on the same grid cells. There is no ablation of the spatial inputs and no evaluation on held-out regions. The experiments therefore do not establish whether the method generalizes to locations with different spatial characteristics.
  8. Several experimental choices need clarification. The ranking temperature differs between the two datasets, but the rationale is not stated. The small number of seeds captures only optimization variability. The paper should report how the temperatures were selected and whether the baselines received comparable hyperparameter tuning.
Questions for authors

Does the gain over continued focal fine-tuning persist when the same ranking objective is trained on the full historical data or on a random subset of the same size, rather than on retrieved samples? These comparisons would help determine whether retrieval provides additional benefits or whether the gains mainly come from aligning the training objective with the evaluation metrics.

6 comments
AuthorsOpenReview ↗

Response to Reviewer pnHZ (1/3)

We thank the reviewer for a detailed and technically precise review. The suggested controls have substantially sharpened the paper. In the revision, every California comparison uses matched compute (same checkpoint, retrieved set, exactly 6,800 updates) and three-seed ensembles for every method, with paired bootstrap intervals over the 351 fire dates. Design choices are checked on a chronological validation split (pretraining 2002–2019; evaluation 2020–2021) that never touches 2022.

W1 / Question: attributing the gains; full-archive and random-subset controls; standard LTR baseline; XGBoost features.

  • Same objective on the full archive and on a size-matched random subset (the reviewer's question). The ranking gain over continued focal persists on every source, so it does not depend on retrieved samples:
California 2022, daily AP (3-seed ensembles)RetrievedRandom, same sizeFull archive
Continued focal22.5922.0422.35
Budget-weighted selective ranking26.0626.3625.90
  • Does retrieval add anything? On the two validation years, retrieval improves budget-weighted selective ranking over the random subset (+0.37 [+0.11, +0.64]; +0.52 [+0.17, +0.89]) and over the full archive (+0.75 [+0.43, +1.07]; +0.97 [+0.52, +1.44]). In 2022 it improves pointwise adaptation (+0.55 [+0.38, +0.71]), while under ranking the random subset is slightly better (−0.31 [−0.59, −0.03]). The retrieval ablation now uses the ranking losses, as the reviewer requested in W5.
  • Standard learning-to-rank baseline. Uniform same-day ranking is grouped RankNet on the same network: all same-day positive–negative pairs with equal weights and the date as the query. It reaches 25.28% daily AP, and budget-weighted selective ranking exceeds it by +0.78 [+0.40, +1.17].
  • XGBoost features. The original "XGBoost" row used weather inputs only, and we thank the reviewer for catching this. The revision adds Spatial XGBoost with the identical 26 inputs, including ecoregion and causal history (24.53%). Budget-weighted selective ranking exceeds it by +1.53 [+0.86, +2.24] in 2022 and +1.25 [+0.38, +2.17] in the 2021 validation year.
  • Objective–metric alignment. We agree that much of the gain comes from aligning the training objective with the evaluation, and we regard this as the central point of the paper rather than a confound: a daily budget is decided by within-day order, so the model should be trained on within-day order. The revision makes this alignment more explicit. Pair weights reflect each pair's effect on the top of the day's ranking, which improves on uniform same-day pairs on identical data, and the Top-B miss bound extends to these weights (new Corollary 1).

W2: selective vs. direct ranking and RDPO. We agree. At exactly 6,800 updates on an identical stream of minibatches and pairs, the three targets are indistinguishable (24.40 / 24.39 / 24.39 daily AP; RDPO − direct −0.01 [−0.15, +0.13]). This is the small-margin regime our sensitivity bound (Prop. 2) predicts at β = 0.1, so the reviewer's reading of the gradient analysis is correct. Selective's original Top-5% advantage over RDPO came almost entirely from seed 22 (−0.00 / +1.97 / +0.07). The revision removes the selective-specific capture claim and presents the three targets as comparable members of one family. What distinguishes our method is the budget-aligned weighting, which improves on unweighted selective ranking by +1.16 [+0.72, +1.59] in California 2022. On Yosemite 2021 it improves AUROC (+0.18 [+0.09, +0.27]), AUPRC (+0.25 [+0.05, +0.46]), and R@30 (+1.27 [+0.60, +1.93]), positive in all five seeds.

AuthorsOpenReview ↗

Response to Reviewer pnHZ (2/3)

W3: high-DM recall at matched alarm rates. We re-evaluated recall within the retrospective top-DM strata at matched budgets. When continued focal raises the same total number of alarms, budget-weighted selective ranking still recalls +27.6 [+5.0, +50.2] pp more of the top-5% DM cell–days, +18.2 [−0.8, +37.2] of the top 10%, and +13.5 [+5.8, +21.2] of the top 20%. The unweighted objectives give +31.5 to +33.3 pp on the top 5%. At 34 cells per day the gains remain positive on average (+12.4 / +10.5 / +8.0 pp), and the intervals mostly include zero. The revision replaces the unmatched figure with this comparison and narrows the abstract to "higher recall of the highest-dry-matter cell–days at an equal alarm budget."

W4: smoothed labels vs. distinct fire events. We re-scored the saved 2022 logits against two outcomes no model was trained on:

  • unsmoothed cell-level labels: 2,193 raw-DM-positive cell–days;
  • 1,266 distinct fire events, formed by linking raw-positive cell–days in the same or an adjacent cell on the same or next date.
California 2022, 3 seedsRaw-label daily APRaw-label R@5%Events detected at 34 cells/day
Continued focal6.73 ± 0.5622.73 ± 1.89348 ± 34
Spatial XGBoost8.75 ± 0.1629.38 ± 1.24392 ± 2
Direct ranking8.48 ± 0.6828.95 ± 4.23400 ± 13
Selective ranking8.65 ± 0.4429.71 ± 4.94397 ± 15
Budget-weighted selective8.96 ± 0.8230.78 ± 1.87397 ± 16

Every ranking objective detects more distinct events than continued focal in every seed under the same daily budget, about 50 more (+14%). The revision adds this table, states in the main text that budget counts are neighborhood-smoothed cell–days, and removes the cell–day count from the abstract.

W5: coverage, k, ranking-loss retrieval ablation, full-year covariates.

  • The retrieval ablation now uses the ranking losses (W1 above).
  • Sensitivity to k (new). With everything else fixed, budget-weighted selective ranking gives, for k = 1 / 5 / 20 / full archive: 32.09 / 32.48 / 32.00 / 31.73 (validation 2020), 31.94 / 31.96 / 31.27 / 30.99 (validation 2021), and 26.85 / 26.06 / 25.66 / 25.90 (2022) daily AP. On both validation years k = 5 is best or tied with k = 1. Larger neighborhoods are significantly worse in all three years (k = 20 − k = 5: −0.48, −0.69, −0.40). In 2022, k = 1 is higher (+0.79 [+0.38, +1.20]); we keep the pre-specified k = 5, which the validation years support, rather than tuning k on the test year.
  • Coverage (new analysis). We split the 2022 dates into tertiles by how far their target covariates lie from the archive (mean distance of each cell to its nearest historical row in the retrieval space). On the third of dates farthest from the archive, budget-weighted selective ranking still gains +2.16 [+0.98, +3.42] daily AP over continued focal, vs. +5.93 [+4.42, +7.50] on the nearest third. Retrieval adds +0.69 [+0.09, +1.32] over the full archive on the middle tertile but nothing on the farthest one (−0.31 [−0.70, +0.10]). This is what the coverage assumption predicts: when target conditions leave the archive, retrieval stops helping, while the ranking gain persists. The revision reports this analysis and states the coverage assumption next to the retrieval step.
  • Full-year covariates. The full-archive variant uses no target-period covariates (25.90% vs. 22.35% for continued focal on the same archive), so the operational concern applies to the optional retrieval step only. The revision labels that step as batch-transductive.
AuthorsOpenReview ↗

Response to Reviewer pnHZ (3/3)

W6: scope of the theory. We agree with the reviewer's characterization.

  • The binary-label reduction (Prop. A.1) is a direct algebraic reduction. We keep it only to define the Yosemite binary-label baseline, and the revision labels it as background.
  • The daily-budget bound applies to any nonnegative target, and that is its purpose: it identifies which members of the target family control avoidable Top-B misses. Direct and selective ranking do; RDPO, whose targets can be negative, does not. It is a deterministic, in-sample inequality on training dates with no guarantee for a future year. The revision therefore renames it the "Top-B miss bound" and states this scope next to the proposition. The new Corollary 1 extends it to budget-aligned pair weights.
  • Relation to Jiang & Sun (2026): that work combines supervised fine-tuning with a DPO preference between the true and the complementary label of each input, for a weather-only Yosemite model. This paper studies input-pair ranking between locations on the same date, a different object whose loss depends on score gaps between candidates and is invariant to per-day score shifts. On top of it, we contribute the score-gap family with its gradient-allocation and sensitivity results (Props. 1–2), the Top-B miss bound and its weighted extension, and the budget-weighted objective. The revision adds this comparison to Related Work.

W7: isolating temporal shift and spatial heterogeneity.

  • Spatial-input ablation with the learner held fixed: adding terrain, ecoregion, and causal history raises daily AP from 14.24 to 24.40 for XGBoost and from 14.82 to 21.10 for pretrained TabNet. The ranking gains are measured on top of this context.
  • More target years. Beyond 2022, the chronological validation protocol evaluates California 2020 and 2021. There, budget-weighted selective ranking exceeds direct ranking in both years (+0.79 [+0.43, +1.16]; +0.70 [+0.12, +1.33]). Yosemite adds three rolling target years.
  • Held-out regions. We agree these are not evaluated. The revision states that the claims concern temporal adaptation on historically observed locations; Appendix E describes a held-out-region design.

W8: temperatures, seeds, tuning parity.

  • Temperatures. California's β = 0.1 was a fixed default, never tuned on 2022. On the validation split, direct ranking is flat over β ∈ {0.03, 0.1} (paired difference +0.08 [−2.21, +2.36]) and declines for β ≥ 0.3 (β = 1: −2.64 [−4.75, −0.53]). The default therefore lies on the validation plateau. In 2022, ranking beats pointwise adaptation at every β across a 33× range. Yosemite uses β = 1 for every ranking objective; it is the only value we ran and was not tuned on any Yosemite year.
  • Tuning parity. All TabNet objectives share the checkpoint, optimizer, learning rate, batch size, and exact update count. Spatial XGBoost uses fixed tree hyperparameters with no early stopping. Every seeded method is reported the same way, as a three-seed ensemble. We also varied the learning rate, weight decay, and number of updates for the budget-weighted objective on the validation split only. The selected setting differed from the shared default only in weight decay and gave the same 2022 result, so the main table keeps the default shared by all methods (Appendix D.5).
  • Seeds. We agree that three seeds capture optimization variability within one year. We therefore report paired intervals throughout, add the 2020–2021 validation years, and keep Yosemite's three rolling target years.

We are grateful for these suggestions; they made the evaluation considerably more rigorous. We would be happy to address any remaining questions.

Reviewer_nSLfOpenReview ↗
Rating
2(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This work presents a retrieval-ranking adaptation framework for wildfire prediction under spatiotemporal distribution shift. Instead of standard imbalanced binary classification, it formulates the task as daily prioritization of high-risk zones and constructs within-day positive-negative sample pairs. The method retrieves similar labeled historical samples and fine-tunes with pairwise ranking objectives, including direct ranking, RDPO and selective ranking. Experiments are conducted on the Yosemite and California statewide wildfire datasets.

Critical strengths

The model objective aligns well with practical operational requirements, and the within-day sample pairing is a sensible design for daily wildfire risk prioritization.

Critical weaknesses

The paper does not sufficiently distinguish the performance contributions of the retrieval and ranking modules, making the methodological novelty of their combination unclear. Furthermore, the batch-transductive evaluation uses covariates from the entire target year, which may not be available in real-time forecasting scenarios. This limits the practical applicability of the reported results.

Questions for authors
  1. Could the authors provide more rigorous ablation studies to quantify the individual performance contributions of the retrieval and ranking modules?
  2. How would the method perform under a strictly chronological evaluation protocol using only covariates available at prediction time?
3 comments
AuthorsOpenReview ↗

Response to Reviewer nSLf

We thank the reviewer for the review and for noting that the objective fits operational requirements and that within-day pairing is a sensible design. The reviewer's two questions are the right ones: which component contributes what, and whether the method works without future covariates. The revision answers both with controlled experiments in which every method uses matched compute and is reported as a three-seed ensemble. Intervals are paired bootstrap intervals over the 351 fire dates of 2022.

Q1: Separating the contributions of retrieval and ranking. We crossed the adaptation objective with the source of training data. Every cell starts from the same pretrained checkpoint and receives exactly 6,800 updates; the random subset has the same size as the retrieved set.

California 2022, daily AP (3-seed ensembles)Retrieved (spatial KNN)Random, same sizeFull archive
Continued focal (pointwise)22.5922.0422.35
Direct ranking (same-day pairs)24.8924.7924.89
Uniform same-day ranking25.2825.3525.40
Budget-weighted selective ranking26.0626.3625.90
  • Ranking is the largest contribution and holds on every data source: +3.47 [+2.71, +4.27] daily AP over continued focal on the retrieved set, and +3.5 to +4.3 points on every source.
  • Budget weighting adds to it on identical data: +0.78 [+0.40, +1.17] over uniform same-day ranking and +1.16 [+0.72, +1.59] over unweighted selective ranking.
  • Retrieval is evaluated on two chronological validation years that never touch 2022 (pretraining and archive 2002–2019). There it improves budget-weighted selective ranking over the size-matched random subset in both years (+0.37 [+0.11, +0.64] in 2020; +0.52 [+0.17, +0.89] in 2021). Over the full archive the gains are +0.75 [+0.43, +1.07] and +0.97 [+0.52, +1.44]. In 2022 retrieval improves pointwise adaptation (+0.55 [+0.38, +0.71] over random). Under ranking in 2022, the random subset is slightly better (−0.31 [−0.59, −0.03]); we report this in the paper. Retrieval also reduces the adaptation set to 37% of the archive.
  • Against a non-adapted model on the same inputs (XGBoost with terrain, ecoregion, and causal history), the full method gains +1.53 [+0.86, +2.24] in 2022 and +1.25 [+0.38, +2.17] in the 2021 validation year.

Q2: Strictly chronological evaluation with prediction-time covariates only.

  • Every split is chronological. Models are pretrained on years strictly before the target year, history features use only labels from earlier years, and no target labels are used for training or selection.
  • Training the same objective on the full historical archive uses no target-period covariates at all and is therefore a strictly inductive protocol. In 2022 it attains 25.90% daily AP and 25.29% Top-5% recall, vs. 22.35% and 18.82% for continued focal on the same archive, so the main gain does not require target-period covariates.
  • The batch-transductive retrieval variant uses the whole target year's covariates (never its labels). The revision labels it as retrospective and reports it alongside the inductive variant.

Novelty of the combination. Three elements together define the method. First, same-day pairs cancel any date-level score offset, so training can only improve by ranking locations within a day, which is the operational decision. Second, budget-aligned pair weights put training where a daily budget is decided, and our Top-B miss bound links the weighted pairwise loss to avoidable misses under a fixed budget. Third, target-conditioned retrieval selects the historical slice used for adaptation without target labels, and improves ranking adaptation on held-out years. The ablation above quantifies each element separately.

Presentation. We streamlined Sections 4 and 6 around these three elements. California main-text results now use one protocol (matched updates, ensembles, bootstrap intervals), and secondary diagnostics are in the appendix.

We hope the new ablations clarify the contribution of each component, and we would welcome any further questions.

Thank you for your rebuttal, as well as for conducting the additional experiments. Your responses have satisfactorily addressed my concerns, and I have updated my review score accordingly.

AuthorsOpenReview ↗

Thank you!

Thank you for your thoughtful review and for considering our additional experiments and clarifications. We are glad that our responses addressed your concerns. Please let us know if you have any further questions.

Reviewer_Cvw4OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces PyroAdapt, a pretrain–retrieve–rank framework for adapting wildfire-risk models to changing environmental conditions without using labels from the target period. It retrieves historical examples with conditions similar to the target period and fine-tunes a pretrained model using direct ranking, residual pairwise DPO, or selective ranking. Evaluation includes rolling temporal-shift experiments on a nine-cell Yosemite grid and daily risk prioritization across 666 California cells. In California, selective ranking increases daily average precision from 21.62% with continued focal fine-tuning to 24.57% and Top-5% recall from 18.70% to 22.79%. The paper also reports increased recall for observations with high dry-matter consumption, although those comparisons use unmatched alarm rates.

Critical strengths

A practically motivated adaptation framework with a useful ranking objective. Retrieval provides an intuitive way to select historical examples relevant to target conditions. Comparing locations within the same day aligns training with the operational task of prioritizing locations under a fixed daily monitoring budget. The improvements in daily average precision and Top-5% recall provide evidence that this approach is useful in the evaluated setting.

Several useful experimental controls and additional temporal evaluations. The evaluation includes a frozen pretrained model, continued and weighted focal fine-tuning, XGBoost, multiple random seeds, and a retrieval ablation. Rolling Yosemite evaluations provide evidence beyond a single temporal split. The authors also clearly acknowledge important limitations, including the distinction between smoothed positive cell–days and independent fire incidents.

Critical weaknesses

The evaluation supports a narrower adaptation claim than the broader framing suggests. California is evaluated for one target year, and all evaluated locations are present in the historical data. This demonstrates temporal adaptation on a fixed spatial grid rather than generalization to unseen regions. The nine-cell Yosemite experiments do not resolve this limitation. Furthermore, adaptation uses covariates from the complete target year. This batch-transductive setting does not establish performance when only information available at prediction time can be used.

Some outcome claims and comparisons between ranking objectives are insufficiently supported. Spatial smoothing means that the reported 344 additional captures are positive cell–days, not additional independent fires. The high-dry-matter recall comparisons use policy-specific thresholds and unmatched alarm rates, so they do not establish improved detection at an equal monitoring budget. This weakens the abstract’s claim about detecting more extreme fire events. In addition, selective ranking’s Top-5% advantage is concentrated in one seed, and the objectives use different realized training durations, making their relative contributions difficult to isolate.

Questions for authors
  • Clarification of target-period information: Which target-year covariates are used for retrieval and adaptation, and when would those covariates become available? Can the authors distinguish the current batch-transductive evaluation from an operational setting in which future target-period covariates are unavailable?

  • Comparison affecting the extreme-fire claim: Do the high-dry-matter recall gains persist when methods are compared at matched alarm rates or under the same daily monitoring budget? If this comparison is unavailable, would the authors narrow the corresponding claims in the abstract and conclusions?

  • Comparison affecting the ranking-objective contribution: How do direct ranking, residual DPO, and selective ranking compare under matched update counts? Please also clarify how much of selective ranking’s mean Top-5% improvement is attributable to the strongest-performing seed.

Suggestions for future work include evaluation across additional California target years and genuinely unseen regions, stronger domain-adaptation or test-time-adaptation baselines, and sensitivity analyses for retrieval-neighbor count, ranking temperature, and spatial features. These would strengthen the paper but are distinct from the clarification requests above.

3 comments
AuthorsOpenReview ↗

Response to Reviewer Cvw4 (1/2)

We thank the reviewer for a careful and constructive review, and for recognizing that same-day comparison aligns training with daily prioritization under a fixed monitoring budget. The revision follows the reviewer's questions closely. Every California comparison now uses matched compute and three-seed ensembles for every method, plus a chronological validation split that never touches 2022. Revised text is in blue.

What changed in the method. The matched-update experiments the reviewer asked for (Q3) show that the reference targets of direct ranking, RDPO, and selective ranking are interchangeable at β = 0.1. What matters is which same-day pairs receive weight. Our primary objective, budget-weighted selective ranking, keeps the selective target and weights each same-day pair by how much swapping it would change the day's NDCG, which concentrates training on pairs that decide the top of the daily list. The Top-B miss bound carries over with an explicit weight floor (new Corollary 1). On California 2022, it raises daily AP from 22.59% (continued focal) to 26.06% and Top-5% recall from 19.13% to 25.02%. It exceeds unweighted selective ranking by +1.16 [+0.72, +1.59] daily AP on the same data and updates.

W1 / Q1: scope and target-period information.

  • Which covariates, and when are they available? Each target row has previous-day GridMET weather (12), static terrain and location (7), ecoregion, causal fire-history rates from prior years only (3) plus history length, and seasonality (2). Static fields and history rates are known before the target year; weather arrives daily with a one-day lag. The main protocol retrieves with the covariates of the whole target year, so it is a retrospective, batch-transductive evaluation, not an issue-time forecast. The revision states this in the abstract, Section 4, and the Discussion.
  • Operational setting without future covariates. Training the same objective on the full historical archive uses no target-period information at all. It attains 25.90% daily AP in 2022 (ensemble), vs. 22.35% for continued focal on the same archive. The prioritization gain is therefore available in a strictly inductive protocol; target-conditioned retrieval is a label-free addition on top of it.
  • Fixed grid, unseen regions. We agree, and the revision states explicitly that the paper evaluates temporal adaptation on historically observed locations and makes no claim about unseen regions. Beyond 2022, we now also report two further California target years, 2020 and 2021, in the chronological validation protocol (pretraining 2002–2019). There, budget-weighted selective ranking exceeds direct ranking in both years (+0.79 [+0.43, +1.16] and +0.70 [+0.12, +1.33]).

W2: "344 additional captures" are smoothed cell–days. We agree, and the abstract no longer reports this count. To check that gains carry over to underlying fires, we re-scored the saved 2022 logits against two outcomes no model was trained on: unsmoothed cell-level labels (2,193 cell–days), and 1,266 distinct fire events formed by linking raw-positive cell–days that are adjacent in space and time. Under the same 34-cell daily budget, every ranking objective detects more distinct events than continued focal in every seed, about 50 more (+14%). Budget-weighted selective ranking detects 397 ± 16 events vs. 348 ± 34 (per-seed gains +9 / +34 / +102), and attains the highest raw-label daily AP (8.96% vs. 6.73%). The table is added to the appendix (Table: distinct events).

AuthorsOpenReview ↗

Response to Reviewer Cvw4 (2/2)

W2 / Q2: high-dry-matter recall at matched alarm budgets. The reviewer is right that policy-specific thresholds give different alarm rates (ranking flags 16–22% of grid–days, continued focal 14–16%). We re-evaluated recall within the retrospective top-DM strata at matched budgets:

Recall gain over continued focal (pp), mean [95% CI]Top-5% DMTop-10% DMTop-20% DM
Budget-weighted selective, same total alarm count+27.6 [+5.0, +50.2]+18.2 [−0.8, +37.2]+13.5 [+5.8, +21.2]
Unweighted objectives, same total alarm count+31.5 to +33.3+20.8 to +22.3+14.5 to +15.3
Budget-weighted selective, same 34 cells/day+12.4 [−7.3, +32.2]+10.5 [+0.4, +20.5]+8.0 [−4.5, +20.6]

At an equal total alarm burden, most of the high-activity gain remains. Under the tightest daily budget it is positive on average, and the intervals mostly include zero over three seeds. The revision replaces the unmatched figure with this table and narrows the claim in the abstract and conclusion to "higher recall of the highest-dry-matter cell–days at an equal alarm budget."

W2 / Q3: objectives at matched updates; seed attribution.

  • Matched updates. We reran direct ranking, RDPO, and selective ranking from the same checkpoint for exactly 6,800 updates on an identical stream of minibatches and same-day pairs, so only the target differs. They are indistinguishable: 24.40 ± 0.71, 24.39 ± 0.68, and 24.39 ± 0.68 daily AP. RDPO − direct is −0.01 [−0.15, +0.13], and every per-seed difference is below 0.2 pp. The small differences in the original table came from different realized update counts, as the reviewer suspected. This is the regime our sensitivity bound (Prop. 2) predicts at β = 0.1.
  • Seed attribution. Selective − RDPO in Top-5% recall is −0.00 / +1.97 / +0.07 across seeds 11 / 22 / 33, so nearly all of the original mean advantage came from seed 22. The revision no longer singles out unweighted selective ranking in the abstract or contributions.

Suggested sensitivity analyses.

  • Temperature. On the validation split, direct ranking is flat over β ∈ {0.03, 0.1} and declines for β ≥ 0.3. The default β = 0.1 lies on this plateau. Ranking beats pointwise adaptation at every β across a 33× range.
  • Spatial features. With the learner held fixed, terrain, ecoregion, and causal history add 10.2 daily-AP points for XGBoost (14.24 -> 24.40) and 6.3 for pretrained TabNet (14.82 → 21.10).
  • Retrieval and number of neighbors. We crossed every objective with spatial KNN, a size-matched random subset, and the full archive (details in our reply to Reviewer pnHZ). We also varied the number of neighbors with everything else fixed. On both validation years k = 5 is best or tied with k = 1, and k = 20 is significantly worse in all three years (e.g., 2021: −0.69 [−0.97, −0.43]). In 2022, k = 1 is higher than k = 5 (+0.79 [+0.38, +1.20]); we keep the pre-specified k = 5, which the validation years support.

We hope these additions address the reviewer's concerns, and we would be glad to clarify anything further.

Dear Reviewer,

As the discussion period is approaching its end, we would greatly appreciate it if you could share your thoughts on our rebuttal and whether it has addressed your concerns.

We would also be happy to provide further clarifications or answer any additional questions you may have.

Thank you again for your time and thoughtful feedback!

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all reviewers. All three reviewers identify the same core strength: training on same-day location pairs aligns learning with the operational decision of prioritizing a fixed number of cells per day. The revision makes this alignment explicit in the objective, and the new experiments answer the main concerns directly:

  1. Budget-weighted selective ranking, compared fairly (pnHZ W1, nSLf Q1): a daily budget is affected only by inversions near the top of each day's ranking. We therefore weight every same-day pair by its LambdaRank |ΔNDCG| weight and keep the selective target [Δ_ref]_+. The Top-B miss bound carries over with an explicit weight floor (new Corollary 1). Under matched compute (same checkpoint, retrieved set, and exactly 6,800 updates), with every seeded method reported as a 3-seed ensemble, it improves 2022 daily AP from 22.59% (continued focal) to 26.06% (+3.47 [+2.71, +4.27], paired bootstrap over fire days) and Top-5% recall from 19.13% to 25.02%. It exceeds unweighted selective ranking by +1.16 [+0.72, +1.59] and grouped RankNet by +0.78 [+0.40, +1.17] on identical data. On Yosemite 2021 it gives the best mean on every metric, improving on selective ranking in all five seeds. See Section H.
  2. A standard learning-to-rank baseline and same-feature trees (pnHZ W1):
    • Uniform same-day ranking is grouped RankNet (Burges et al., 2005) on the same TabNet: all same-day positive–negative pairs, equal weights, the day as the query. It reaches 25.28% daily AP, and budget-weighted selective ranking exceeds it by +0.78 [+0.40, +1.17].
    • The tree baseline now uses the identical 26 inputs, including ecoregion and causal history (Spatial XGBoost). The original table's "XGBoost" was weather-only. Budget-weighted selective ranking exceeds Spatial XGBoost by +1.53 [+0.86, +2.24] in 2022 and by +1.25 [+0.38, +2.17] in the 2021 validation year.
  3. Retrieval's contribution (nSLf Q1/W5, 2nd reviewer): we crossed every objective with spatial KNN, a size-matched random subset, and the full archive at 6,800 updates.
    • On held-out validation years, which never touch 2022, retrieval improves budget-weighted selective ranking over random by +0.37 [+0.11, +0.64] (2020) and +0.52 [+0.17, +0.89] (2021). Over the full archive, the gains are +0.75 [+0.43, +1.07] and +0.97 [+0.52, +1.44].
    • In 2022, retrieval improves focal adaptation (+0.55 [+0.38, +0.71]).
  4. No target-year covariates needed for the ranking gain (Cvw4 Q1, nSLf Q2, pnHZ W5): budget-weighted selective ranking on the full historical archive, which uses no target-period information, attains 25.90% daily AP (ensemble), vs. 22.35% for continued focal on the same archive.
  5. Matched alarm budgets for high-dry-matter fires (Cvw4 Q2, pnHZ W3): when continued focal raises exactly as many alarms, budget-weighted selective ranking still recalls +27.6 pp more of the top-5% DM cell–days ([+5.0, +50.2]); the unweighted objectives give +31.5 to +33.3 pp. This comparison replaces the unmatched-threshold figure.
  6. Raw labels and distinct fire events (pnHZ W4, Cvw4 W2): against unsmoothed labels and 1,266 distinct space–time fire events, budget-weighted selective ranking detects 397 ± 16 events at 34 cells per day vs. 348 ± 34 for continued focal (more in every seed), with the highest raw-label daily AP (8.96% vs. 6.73%).
  7. Matched update counts and temperature selection (Cvw4 Q3, pnHZ W2/W8): on an identical pair stream at exactly 6,800 updates, Direct, RDPO, and Selective are indistinguishable (|Δ daily AP| ≤ 0.07 pp in every seed), consistent with our sensitivity bound at β = 0.1. β, previously a fixed default, lies on the 2020–2021 validation plateau (β ∈ {0.03, 0.1}), and ranking beats pointwise adaptation at every β across a 33× range.
  8. Coverage and number of neighbors (pnHZ W5, Cvw4): on the third of 2022 dates farthest from the archive, the ranking gain over continued focal persists (+2.16 [+0.98, +3.42]); retrieval stops helping there, as the coverage assumption predicts. Across k ∈ {1, 5, 20}, k = 5 is best or tied on both validation years, and k = 20 is worse in every year.

Revised text is marked in blue in the updated PDF.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.