acceptodds
Under review as a conference paper at ICLR 2027

Structure-Grounded Test-Time Adaptation of Tabular Foundation Models for Power Outage Prediction

Abstract

Weather-induced power outages disrupt critical infrastructure, making accurate forecasting essential for proactive response. Existing methods are typically trained once on heterogeneous historical weather events, limiting their ability to adapt to rare high-impact events. Tabular foundation models such as TabPFN instead leverage in-context labeled examples at inference time, enabling event-specific prediction without retraining. However, their native inference does not explicitly capture event-specific spatial dependencies among neighboring locations, which may vary substantially across events, particularly during extreme events. To resolve this limitation, we propose Structure-Grounded Test-Time Adaptation (SG-TTA), which uses spatial structure as test-time feedback to adapt a lightweight module while keeping the tabular in-context learner frozen. For each incoming event, SG-TTA retrieves relevant locations in historical events and their neighborhoods to augment in-context support and provide cross-event spatial references for alignment, while maintaining within-event consistency via neighbor reconstruction. Experiments across four real-world service territories show that SG-TTA consistently improves both TabPFN and TabICL, with the TabPFN variant achieving the lowest MAE in every territory. Further analysis in Connecticut (CT) confirms performance gains across outage-severity levels, especially on high-impact events.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Power outage prediction, Tabular Foundation Models, Test-time adaptation

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_ASguOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

The paper tackles the challenge of the usage of tabular foundation models within outage prediction. They are by vanilla design not capable for outage prediction. To address outage prediction, the authors present two methods for using those tabular foundation models to predict power outages caused by rare, high-impact weather events. The authors introduce Structure-Grounded Test-Time Adaptation (SG-TTA) framework. The framework consist of two mayor components the Location-Anchored Subgraph Retrieval and Structure-Grounded Test-Time Adaptation. Location-Anchored Subgraph Retrieval models the labeled context C∗C^* by leveraging historical subgraphs relevant to the location and its neighborhood. Structure-Grounded Test-Time Adaptation retrieves the subgraphs and uses LoRA Adapters to apply the test-time adaptation.

Critical strengths
  1. The paper provides a solution approach to the problem, that extreme weather events and their impact on the power system are not sufficiently captured by current vanilla TFMs.
  2. Ablation Study shows strengths really well. I like it.
Critical weaknesses
  1. Limitations and weaknesses of the method, its transferability, and possible future work are generally not discussed in the paper. Is the method adaptable to other extreme events, such as floods in flood-prone regions? Is the method adaptable to geographical locations other than the US?

  2. We need for you ablation experiments and all experiments a little more description. Please use the appendix for this and give additional input for this.

  3. The paper's formatting does not meet ICLR's requirements of centered tables. Furthermore, Tables 2,3 and 4 are not chronologically numbered.

My general merit is positive, I like the topic of the paper, I like the idea of modifying both the in-context samples and the test-time adaptation within. Therefore, my merit is a weak accept as it shows an use-case of machine learning in the age of TSFM for a very special and interesting use-case.

Questions for authors
  1. Notation needs explanation. What is the meaning of , especially in regard to C (the context)
  2. LoCalPFN and G2T-FM what inputs do they receive ? Can you possibly describe them more in the Appendix to ensure a fair comparison.
  3. Is vanilla without in context examples, therefore without fit ?
  4. Not sure about the 5 fold crossfold. Could time be a factor ? Because the split does not respect timely ordering.
  5. I really like The Ablation Study. But which parts are hold flexible and which parts are ablated. For Table 2 the Location-Anchored Subgraph Retrieval is alternated, is the Structure-Grounded Test-Time Adaptation kept? For Table 3 the main topic is the Structure-Grounded Test-Time Adaptation is for this the Location-Anchored Subgraph Retrieval kept? Is it possible to show the results of Location-Anchored Subgraph Retrieval without using Structure-Grounded Test-Time Adaptation and the other way around? Further, sometimes some of those methods in the ablation needs to be clarified. Can you elaborate about covariate KNN and task aligned KNN in Appendix? Simply on sentence what goes in and out.
4 comments
AuthorsOpenReview ↗

Reply to Reviewer ASgu: W1 & W2 & W3

W1. Limitations, transferability and possible future work.

We thank the reviewer for this suggestion and will add a dedicated Limitations and Future Work section to the revision.

Limitations. Our evaluation covers four territories of a single utility in the northeastern US, and the most severe events are rare, so evidence for them is descriptive.

Transferability. SG-TTA is potentially transferable to other extreme events, such as floods and wildfires. These tasks can share the spatial dependencies and scarcity of severe-event examples that motivate our framework. When historical events provide labeled observations at recurring locations, the tabular backbone, historical-context retrieval, and event-specific adaptation procedure can be reused.

Transfer would require task-specific covariates and targets, retraining the retrieval encoder, and potentially modifying the graph. For example, incorporating drainage connectivity for flood prediction. Historical neighborhoods may provide poor references for unprecedented events, and the structural objectives would require validation for the new task. The framework is not inherently US-specific, but geographical transfer similarly requires suitable local records and validation under different environmental conditions.

Future work. We would like to extend the framework to additional extreme-event prediction tasks and regions outside the US, including exploring cross-region retrieval where local historical data are limited.

W2. More detailed descriptions of the ablation study and experiments.

We summarize the setup of each experiment below and will add a dedicated appendix with these details to the revision.

Ablation experiments of context (Table 2). The TabPFN backbone remains frozen, and only context construction changes. Full history uses all training-fold rows, while Random samples 50K rows. Covariate kNN retrieves the 10 nearest training rows per target location in standardized covariate space; Task-aligned kNN uses the learned retrieval embeddings instead. Both pool the selected rows across the event. Our method retrieves three historical subgraphs per target location and pools their rows after deduplication. Thus, this experiment evaluates context construction without structural TTA.

Ablation experiments of adaptation (Table 3). The historical context C⋆C^\star is held fixed while the adaptation mechanism or objective changes. Frozen TabPFN performs no parameter updates. Supervised fine-tuning updates the adapters on pooled historical events from the training fold. Supervised fine-tuning on C⋆C^\star splits the retrieved historical rows into support and pseudo-query sets and updates the adapters using the pseudo-query labels.

Neighbor-feature concatenation augments the input with mean neighbor covariates. Direct hidden refinement optimizes target representations rather than model parameters. The trained graph adapter inserts a graph attention layer as adapter.

The LrelL_{rel} only and LselfL_{self} only variants retain one component of the structural objective, while the shuffled-reference variant replaces each location's reference distribution with that of a randomly chosen location, disrupting the correspondence between target locations and historical references. All variants that update parameters use the same optimizer, learning rate, number of steps, and adapted layers as SG-TTA.

W3. Formatting.

We thank the reviewer for pointing out these formatting issues, and we will correct these problems in the revision.

AuthorsOpenReview ↗

Reply to Reviewer ASgu: Q1-Q4

Q1. Notation explanation.

We thank the reviewer for pointing this out. The symbol in the question did not render on our side; given the reference to the context CC, we understand it to be the superscript ⋆\star in C⋆C^\star, and we would be glad to clarify any other notation.

The superscript ⋆\star marks quantities associated with the incoming target event, such as its covariates X⋆X^\star and predictions Y^⋆\hat{Y}^\star. Accordingly, C⋆C^\star denotes the context constructed for that event by pooling and deduplicating the labeled rows of the retrieved historical subgraphs; it contains only training-fold events and is shared by all locations of the target event.

Q2. LoCalPFN and G2T-FM what inputs do they receive? Can you possibly describe them more in the Appendix to ensure a fair comparison.

We thank the reviewer for this question and will describe both baselines in detail in the appendix.

LoCalPFN. Our LoCalPFN implementation uses local retrieval followed by supervised fine-tuning. For each location in a test event, we rank rows from the training fold by cosine similarity to the query in the standardized covariate space. We retrieve the 10 most similar rows per location and pool them across locations to form a labeled historical context for the event. For fine-tuning, we randomly split this pool into support rows and pseudo-query rows. The model conditions on the support features and labels and predicts the outage counts for the pseudo-queries. We compute the regression loss against their observed historical outage labels.

G2T-FM. We sample historical context rows from the training fold using the same random sampling procedure as the vanilla backbone. Each location is a node in the spatial graph. For each node, we concatenate its own covariates, the mean covariates of its one-hop neighbors, and the node's coordinates in the graph Laplacian eigenvectors. We use these graph-augmented features for both historical and test rows. For a controlled adaptation comparison, we additionally apply the same supervised fine-tuning procedure as for LoCalPFN, using support and pseudo-query rows sampled from the historical context.

Q3. Is vanilla without in context examples, therefore without fit?

Vanilla does use in-context examples. Both vanilla TabPFN and TabICL condition on 50K randomly sampled, labeled training-fold rows to predict the incoming event. “Vanilla” means using the pretrained backbone without our retrieval or test-time parameter updates; it does not mean prediction without context. Any preprocessing is fitted using the training context, while the backbone parameters remain frozen.

Q4. Not sure about the 5 fold crossfold. Could time be a factor? Because the split does not respect timely ordering.

Thank you for raising this question. We sorted the events by year and divided them into five contiguous folds, each covering a continuous range of years. Each fold is held out once as the test set, and the remaining four folds are used for training. Because folds are contiguous in time, temporally adjacent events, such as consecutive storms with similar weather conditions, almost always fall in the same fold. They can fall on opposite sides of the train-test split only at the boundaries between folds, which limits leakage from neighboring events.

This is a blocked design rather than a strictly forward-in-time evaluation: except when the last fold is held out, the training data include events that occurred after the test period. This does not expose the model to the test events themselves. Each event is predicted only from its own pre-event WRF weather forecast and static covariates, the events are independent storms, and the model uses no temporal sequence information. Training on later events therefore provides no direct information about the outcome of a test event.

AuthorsOpenReview ↗

Reply to Reviewer ASgu: Q5

Q5. Ablation settings and clarification of retrieval baselines.

We thank the reviewer for the positive comment and for this question. In Table 2, the TabPFN backbone is frozen and only the construction of the context varies, so no Structure-Grounded Test-Time Adaptation is applied. In Table 3, the retrieved context from Location-Anchored Subgraph Retrieval is held fixed, and only the adaptation mechanism or objective varies.

We additionally provide the complete 2×22\times2 comparison across all four territories: Random/Frozen, Random/TTA, Retrieved/Frozen, and Retrieved/TTA.

DomainRandom / FrozenRandom / TTARetrieved / FrozenRetrieved / TTA (SG-TTA)
CT367.8 / 98.6336.6 / 84.5321.7 / 87.7309.8 / 66.7
NH114.9 / 75.9112.2 / 63.5106.3 / 65.8101.9 / 53.3
WMA39.8 / 97.138.2 / 76.139.0 / 76.435.3 / 61.2
EMA152.8 / 115.1121.4 / 80.5142.0 / 92.2131.4 / 66.2

Both components contribute to the improvements. With the backbone frozen, retrieved context improves MAE and MAPE over random context in all four territories. With retrieved context held fixed, enabling TTA further reduces MAE by 11.9, 4.4, 3.7, and 10.6 on CT, NH, WMA, and EMA, respectively, and MAPE by 21.0, 12.5, 15.2, and 26.0 percentage points. TTA also improves both metrics under random context across all four territories. These results show that the gains are not attributable to retrieval alone. SG-TTA achieves the lowest MAPE in all four territories and the lowest MAE in three; on EMA, Random/TTA achieves lower MAE, indicating that combining retrieval and adaptation does not uniformly improve every metric.

Definitions of the kNN variants. We will add the following descriptions to the appendix:

  • Covariate kNN: Given a target location’s standardized covariates, it selects the 10 training-fold rows with the highest cosine similarity and pools the selected rows across target locations to form the labeled context.
  • Task-aligned kNN: Given the learned retrieval embedding of a target location, it selects the 10 nearest training-fold rows in that embedding space and pools them across target locations to form the labeled context.
AuthorsOpenReview ↗

Reply to Reviewer ASgu

Dear Reviewer,

Thank you very much for your time and constructive feedback on our submission. We sincerely appreciate your constructive comments and suggestions.

We have carefully considered your concerns and provided detailed point-by-point responses in our rebuttal. We would be very grateful if you could take a moment to review our responses when you have a chance.

We hope the revisions and clarifications have addressed your main concerns. We would also greatly appreciate any additional feedback and would be happy to further discuss any remaining questions or suggestions.

Best regards,

The Authors

Reviewer_ERyEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2(+1)
Read review
Paper summary

This paper proposes SG-TTA, a method that leverages spatial structure as test-time feedback to adapt tabular foundation models to individual, incoming weather-related power outage events. The approach begins by using a task-aligned retrieval encoder to identify historical events and their local neighborhoods that share similar weather and environmental characteristics with the target location; these labeled historical samples then serve as the prediction context. Subsequently, the method constructs cross-event relational constraints based on representation similarities between historical neighborhoods, alongside self-supervised constraints derived from the reconstruction of representations for adjacent locations within the current event. During the testing phase, the foundation model’s parameters remain frozen, while only a small number of LoRA (Low-Rank Adaptation) parameters are updated; the adapter is re-initialized at the onset of each new event. Experiments were conducted across four service areas of Eversource, comparing the proposed method against baselines including traditional machine learning models, GNNs, tabular foundation models, and various retrieval- or graph-based adaptation techniques. The study reports metrics such as MAE, MAPE, CRMSE, and R², and includes analyses on context construction, structural injection methods, feedback objectives, severity-based grouping, row-level errors, runtime, and gradient alignment.

Critical strengths

1- Power outage prediction is characterized by significant heterogeneity across events; neither simple fine-tuning on pooled historical data nor basic retrieval-based approaches are sufficient. The authors’ focus on "event-level adaptation" is well-motivated.

2- By freezing the backbone, updating only a lightweight set of parameters (~0.39%), and resetting the adapter for each event, the model avoids drastic modifications when faced with unlabeled target events. Furthermore, the retrieval process goes beyond simple covariate-based kNN. It employs a task-aligned encoder combined with local subgraphs, which aligns well with the domain prior that power outage responses are closely linked to local environmental conditions.

3- The paper provides a comprehensive ablation study.

4- The experimental metrics are highly relevant to real-world applications, extending beyond standard MSE to include severity-based grouping, as well as event-level and row-level error analyses.

Critical weaknesses

1- It is not explicitly stated whether the data split follows the chronological order of the events.

2- The source of performance gains is not clearly disentangled. A significant portion of the overall improvement may stem from the retrieval component rather than the Test-Time Adaptation (TTA) mechanism itself; since SG-TTA incorporates both, the individual contribution of TTA remains ambiguous. Although Tables 2 and 3 separate "context" and "adaptation" within the CT (Cross-Task) setting, they fail to provide a comprehensive 2×2 breakdown across all four domains—specifically: random context vs. retrieved context, crossed with no TTA vs. TTA.

  1. The fairness and reproducibility of the baselines require greater transparency. Since methods like LoCalPFN and G2T-FM also incorporate retrieval or structural adaptation, it is unclear whether their hyperparameter tuning budgets are equivalent to those used for SG-TTA. Furthermore, the main table does not fully isolate the discrepancy between using 50K random contexts (for all vanilla backbones) and retrieved contexts (for SG-TTA).

  2. In Table 1, the row groupings and rowspan formatting are misaligned. For instance, the "TabPFN/+LoCalPFN" rows appear within the "New Hampshire" section, making it difficult for readers to determine which domain the figures correspond to. Additionally, the metrics "AE Q25/Q50" and "APE Q25/Q50" in Table 1 are undefined. There is also a typo: "GZT-FM" should be "G2T-FM." Finally, Figure 2 focuses exclusively on the CT setting, and the sensitivity analysis for λ\lambda and TT is limited to a single dataset. I also suggest including results that span multiple domains.

Questions for authors

refer to weaknesses

6 comments
AuthorsOpenReview ↗

Reply to Reviewer ERyE: W1 & W2

W1. It is not explicitly stated whether the data split follows the chronological order of the events.

Thank you for raising this question. We sorted the events by year and divided them into five contiguous folds, each covering a continuous range of years. Each fold is held out once as the test set, and the remaining four folds are used for training. Because folds are contiguous in time, temporally adjacent events, such as consecutive storms with similar weather conditions, almost always fall in the same fold. They can fall on opposite sides of the train-test split only at the boundaries between folds, which limits leakage from neighboring events.

This is a blocked design rather than a strictly forward-in-time evaluation: except when the last fold is held out, the training data include events that occurred after the test period. This does not expose the model to the test events themselves. Each event is predicted only from its own pre-event WRF weather forecast and static covariates, the events are independent storms, and the model uses no temporal sequence information. Training on later events therefore provides no direct information about the outcome of a test event.

W2. Disentangled performance gains.

We add the complete 2×22 \times 2 ablation across all four territories, crossing random versus retrieved context with frozen versus adapted model parameters. Each entry reports MAE / MAPE (%).

DomainRandom / FrozenRandom / TTARetrieved / FrozenRetrieved / TTA (SG-TTA)
CT367.8 / 98.6336.6 / 84.5321.7 / 87.7309.8 / 66.7
NH114.9 / 75.9112.2 / 63.5106.3 / 65.8101.9 / 53.3
WMA39.8 / 97.138.2 / 76.139.0 / 76.435.3 / 61.2
EMA152.8 / 115.1121.4 / 80.5142.0 / 92.2131.4 / 66.2

Both components contribute to the improvements. With the backbone frozen, retrieved context improves MAE and MAPE over random context in all four territories. With retrieved context held fixed, enabling TTA further reduces MAE by 11.9, 4.4, 3.7, and 10.6 on CT, NH, WMA, and EMA, respectively, and MAPE by 21.0, 12.5, 15.2, and 26.0 percentage points. TTA also improves both metrics under random context across all four territories. These results show that the gains are not attributable to retrieval alone. SG-TTA achieves the lowest MAPE in all four territories and the lowest MAE in three; on EMA, Random/TTA achieves lower MAE, indicating that combining retrieval and adaptation does not uniformly improve every metric.

AuthorsOpenReview ↗

Reply to Reviewer ERyE: W3 & W4

W3. Fairness and reproducibility of baselines.

LoCalPFN. Our LoCalPFN implementation uses local retrieval followed by supervised fine-tuning. For each location in a test event, we rank rows from the training fold by cosine similarity to the query in the standardized covariate space. We retrieve the 10 most similar rows per location and pool them across locations to form a labeled historical context for the event. For fine-tuning, we randomly split this pool into support rows and pseudo-query rows. The model conditions on the support features and labels and predicts the outage counts for the pseudo-queries. We compute the regression loss against their observed historical outage labels.

G2T-FM. We sample historical context rows from the training fold using the same random sampling procedure as the vanilla backbone. Each location is a node in the spatial graph. For each node, we concatenate its own covariates, the mean covariates of its one-hop neighbors, and the node's coordinates in the graph Laplacian eigenvectors. We use these graph-augmented features for both historical and test rows. For a controlled adaptation comparison, we additionally apply the same supervised fine-tuning procedure as for LoCalPFN, using support and pseudo-query rows sampled from the historical context.

Optimization and tuning budget. For each event, both baselines use the same adaptation settings as SG-TTA: rank-16 LoRA updates to the query, key, value, and output projections in the last four Transformer blocks, with the same optimizer AdamW, learning rate 10−210^{-2}, and number of update steps T=5T=5. Only the LoRA parameters are updated. The baselines use historical pseudo-query labels for supervision, whereas SG-TTA uses its structural adaptation objective. We evaluated one fixed configuration per baseline. SG-TTA's hyperparameters were selected on CT and reused on NH, WMA, and EMA.

Random 50K versus retrieved context. The 2×22\times2 comparison in W2 separates the effects of context construction and adaptation, using the same 50K random-context setting as the main table. Within each context condition, Frozen and TTA use identical historical context rows. Ablation study of context construction (Table 2) further compares different context-construction strategies.

W4. Presentation, metric definitions, and multi-domain analysis.

We thank the reviewer for these careful observations. We will correct the row groupings and multirow alignment in Table 1 and typo in the revision.

Metric definition. AE Q25/Q50 and APE Q25/Q50 denote the 25th and 50th percentiles of event-level absolute error and absolute percentage error. These definitions are provided in Appendix C, and we will also include them in the Table 1 caption in the revision.

Multi-domain analysis. We extended the sensitivity analysis from CT to NH, WMA, and EMA, varying one hyperparameter at a time around the default (T=5T = 5, λ=1\lambda = 1). All variants use the same retrieved context, and T=0T = 0 corresponds to the frozen model. Each entry reports MAE / MAPE (%), with lower values indicating better performance. The † symbol indicates that the corresponding metric is significantly worse than the default (paired event-bootstrap 95% confidence interval for the difference excludes zero).

TTCTNHWMAEMA
0 (frozen)321.7 / 87.7†106.3 / 65.8†39.0† / 76.4†142.0 / 92.2†
1314.0 / 75.8103.8 / 56.338.4 / 61.5134.7 / 69.5
3309.9 / 70.1102.2 / 55.238.0 / 61.0132.4 / 67.7
5 (default)309.8 / 66.8101.9 / 53.335.3 / 61.2131.4 / 66.2
10314.0 / 62.6102.8 / 54.838.0 / 65.0†133.2 / 71.2†
λ\lambdaCTNHWMAEMA
0317.1 / 76.2103.2 / 56.836.2 / 62.5132.8 / 68.5
0.5310.9 / 70.6103.8 / 55.536.9 / 61.5131.9 / 68.4
1 (default)309.8 / 66.8101.9 / 53.335.3 / 61.2131.4 / 66.2
2313.0 / 62.8105.9 / 54.336.0 / 63.1132.9 / 71.7†

Across all four territories, adaptation significantly lowers MAPE relative to the frozen model, and the default setting lies in a stable region where nearby values change neither MAE nor MAPE significantly. Only overly strong adaptation (T=10T = 10 or λ=2\lambda = 2) degrades MAPE on WMA and EMA. The configuration selected on CT therefore transfers to the other territories without per-territory tuning.

AuthorsOpenReview ↗

Reply to Reviewer ERyE

Dear Reviewer,

Thank you very much for your time and constructive feedback on our submission. We sincerely appreciate your constructive comments and suggestions.

We have carefully considered your concerns and provided detailed point-by-point responses in our rebuttal. We would be very grateful if you could take a moment to review our responses when you have a chance.

We hope the revisions and clarifications have addressed your main concerns. We would also greatly appreciate any additional feedback and would be happy to further discuss any remaining questions or suggestions.

Best regards,

The Authors

Reviewer_TnjeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces SG-TTA, a method for predicting power outages associated with specific events. A supervised retrieval encoder identifies relevant historical neighborhoods, which provide labeled context and relational references. Lightweight LoRA parameters are then adapted through historical relation alignment and within-event neighbor consistency, without using outage labels from the target event. The experiments span four utility territories and two tabular foundation models. The TabPFN variant achieves the lowest event-level MAE in each territory, with further results at the location level and controlled ablation studies of the adaptation process.

Critical strengths
  1. Consistent improvements on a consequential application. The TabPFN variant reduces event-level MAE across all four territories, and the CT location-level results show that improvements extend beyond aggregate outage totals.
  2. Evidence isolating adaptation from retrieval. With the retrieved context fixed, adaptation reduces CT MAE from 336.63 to 309.84, approximately 8.0%. Individual-objective ablations and the shuffled-reference control further support the usefulness of the structural feedback.
Critical weaknesses
  1. The evaluation does not yet establish the reliability of the reported gains. Although event-disjoint cross-validation is reported, the documentation of independent hyperparameter selection and fold-specific fitting of the retrieval pipeline is insufficient. Furthermore, accuracy uncertainty and sample sizes for each severity group are not provided, making it difficult to assess both modest improvements and claims regarding extreme events.
  2. Comparative conclusions depend on insufficiently explained baseline reproductions. SA-HGNN’s CT results differ substantially from its published results. The evaluation protocols differ, so this is not evidence of an unfair comparison, but the discrepancy needs explanation. Missing tuning and implementation details for other adaptation baselines further limit confidence in the comparison.
Questions for authors
  1. Clarification: How were hyperparameters selected, and were the encoder, quantizer, preprocessing, and all selection procedures fitted separately within each training fold?
  2. Evidence needed for the current claims: Can you provide paired uncertainty intervals, severity-group event counts, variability across seeds, and the fraction of events whose prediction error increases after adaptation?
  3. Clarification: What explains the difference between the reproduced and published SA-HGNN results after accounting for evaluation splits? Please specify feature selection, graph construction, and tuning procedures.
  4. Clarification: What are the exact graph definition, representation extraction layer, adaptation optimizer, and realized context sizes? Were context and adaptation comparisons conducted with matched evidence and tuning budgets?
5 comments
AuthorsOpenReview ↗

Reply to Reviewer Tnje: W1 (Q1&Q2)

We thank the reviewer for these constructive suggestions.

W1/Q1. Hyperparameter selection and fold-specific fitting of the retrieval pipeline (encoder, quantizer, preprocessing and selection procedures).

Hyperparameter selection. We selected SG-TTA's hyperparameters on CT by varying one parameter at a time and comparing event-level MAE and MAPE pooled over the five folds. We used the same CT folds for selection and reporting, without a separate validation split. The sweeps included the number of adaptation steps (0 to 10, with 0 disabling adaptation), the loss weight λ∈0,0.25,0.5,1,2,4\lambda \in \\{0, 0.25, 0.5, 1, 2, 4\\}, the number of retrieved subgraphs per location K∈1,…,5K \in \\{1,\ldots,5\\}, and the learning rate η∈0.01,0.1,1.0\eta \in \\{0.01, 0.1, 1.0\\}. We applied the selected configuration to NH, WMA, and EMA without further tuning.

Fold-specific fitting of the retrieval pipeline. We assigned all locations from each event to the same fold. For each fold, we trained a separate retrieval encoder, fitted the covariate standardizer, and built the retrieval bank using only training-fold events. To train the retrieval encoder, we mapped location-level outage counts to 60 bins using the same predefined boundaries in every fold. Counts 0–29 each had their own bin, followed by bins of width five and a final overflow bin. We applied this mapping only to training-fold labels. For each test event, we fitted TabPFN's preprocessing on its retrieved context C⋆C^\star, which contained only labeled rows from the training fold, and kept it fixed throughout adaptation.

W1/Q2. Paired uncertainty intervals, severity-group event counts, variability across seeds, and the fraction of events whose prediction error increases after adaptation.

We compared SG-TTA with vanilla TabPFN on the same held-out events and report SG-TTA-minus-vanilla differences. For each metric, the first column gives the difference for the run reported in Table 1, with a paired 95% confidence interval obtained by resampling events. The second column gives the mean ± std of the difference across five seeds. MAPE differences are in percentage points.

The paired intervals for ΔMAPE were below zero in all four territories. For ΔMAE, the intervals were below zero in CT, WMA and EMA and included zero in NH.

DomainNΔMAE [95% CI]ΔMAE (mean ± std)ΔMAPE (pp) [95% CI]ΔMAPE (pp, mean ± std)
CT294-58.0 [-156.8, -22.7]-55.3 ± 6.5-31.8 [-85.1, -26.3]-32.6 ± 4.5
NH227-13.1 [-26.0, +2.8]-12.5 ± 3.5-22.6 [-37.6, -13.1]-23.3 ± 2.6
WMA271-4.6 [-7.8, -0.8]-5.2 ± 2.5-35.9 [-48.9, -7.8]-33.7 ± 5.4
EMA231-21.5 [-37.9, -16.6]-18.4 ± 7.0-48.9 [-66.9, -29.6]-43.7 ± 6.9

We further provide severity-group event counts and the fraction of events whose prediction error increases after adaptation. SG-TTA had higher seed-averaged absolute error than vanilla on 32–36% of events across territories. The Extreme group contains only four events (three in CT and one in EMA), none of which had higher error under SG-TTA. Because the group is so small, this result is descriptive rather than statistically conclusive. Combining High and Extreme into a high-impact group (total outages ≥ 750) gives 59 events across the four territories. SG-TTA had higher error than vanilla TabPFN on 17 of them (29%), lower than its overall rate of 32–36%.

DomainLowModerateHighExtremeOverall
CT65/212 (31%)16/44 (36%)12/35 (34%)0/3 (0%)93/294 (32%)
NH70/206 (34%)5/13 (38%)2/8 (25%)--77/227 (34%)
WMA97/265 (37%)1/3 (33%)0/3 (0%)--98/271 (36%)
EMA75/208 (36%)6/13 (46%)3/9 (33%)0/1 (0%)84/231 (36%)

Note. Each entry is n/N (%), where N is the number of held-out events in the severity group, and n counts events for which SG-TTA had higher absolute error than vanilla TabPFN. Severity thresholds follow the main paper. A dash (--) indicates no events in the group.

AuthorsOpenReview ↗

Reply to Reviewer Tnje: W2 (Q3 &Q4)

W2/Q3. SA-HGNN reproduction details and the discrepancy from published results.

We followed the published SA-HGNN configuration. Feature selection: the same 50 features (20 static, 30 dynamic) selected by Pearson correlation with outage counts. Graph construction: the static graph connects each location to its 8 nearest locations by geographic distance and is shared across events; for each event, the dynamic prior connects each location to the 8 locations whose dynamic features are most correlated with its own, and guides the learned event-specific adjacency during training only. Tuning: we kept the published values λ=0.01\lambda = 0.01 and γ=0.5\gamma = 0.5 and all other published hyperparameters. These values were tuned under leave-one-event-out (LOEO) evaluation, and we did not re-tune them for 5-fold cross-validation, which may also contribute to the gap.

The reported metrics differ in how strongly they depend on a few events. MAE, CRMSE, and R2R^2 are dominated by the largest storms, whose absolute errors can reach thousands of outages and are further amplified by squaring. MAPE is instead dominated by events with very small totals, where the percentage error becomes large. Median AE and median APE are the most robust to a few events. On CT, the three largest storms (about 1% of events) account for 43% of SA-HGNN's total absolute error and 92% of its total squared error. Excluding them reduces MAE from 391.5 to 224.2. Thus, errors on a few events can substantially change aggregate metrics.

This helps explain why the gap from the published results varies across metrics. LOEO and 5-fold cross-validation train on different sets of events (293 vs.\ about 235 on CT), which changes the predictions for these few influential events, and the sensitive metrics amplify those changes. On CT dataset, the largest gap appears in CRMSE (851 published vs.\ 1756 reproduced), while median AE remains close (62.5 vs.\ 67.4). We therefore view the protocol difference as a plausible explanation for much of the discrepancy. Importantly, all methods in our paper, including SG-TTA, are evaluated under identical event-level splits and metrics, so the relative comparisons are not affected.

AuthorsOpenReview ↗

Reply to Reviewer Tnje: W2 (Q3&Q4)

W2. Other adaptation baselines.

LoCalPFN. Our LoCalPFN implementation uses local retrieval followed by supervised fine-tuning. For each location in a test event, we rank rows from the training fold by cosine similarity to the query in the standardized covariate space. We retrieve the 10 most similar rows per location and pool them across locations to form a labeled historical context for the event. For fine-tuning, we randomly split this pool into support rows and pseudo-query rows. The model conditions on the support features and labels and predicts the outage counts for the pseudo-queries. We compute the regression loss against their observed historical outage labels.

G2T-FM. We sample historical context rows from the training fold using the same random sampling procedure as the vanilla backbone. Each location is a node in the spatial graph. For each node, we concatenate its own covariates, the mean covariates of its one-hop neighbors, and the node's coordinates in the graph Laplacian eigenvectors. We use these graph-augmented features for both historical and test rows. For a controlled adaptation comparison, we additionally apply the same supervised fine-tuning procedure as for LoCalPFN, using support and pseudo-query rows sampled from the historical context.

Optimization and tuning budget. For each event, both baselines use the same adaptation settings as SG-TTA: rank-16 LoRA updates to the query, key, value, and output projections in the last four Transformer blocks, with the same optimizer, learning rate (10−210^{-2}), and number of update steps. Only the LoRA parameters are updated. The baselines use historical pseudo-query labels for supervision, whereas SG-TTA uses its structural adaptation objective. We evaluated one fixed configuration per baseline. SG-TTA's hyperparameters were selected on CT and reused on NH, WMA, and EMA.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.