- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper tackles the challenge of the usage of tabular foundation models within outage prediction. They are by vanilla design not capable for outage prediction. To address outage prediction, the authors present two methods for using those tabular foundation models to predict power outages caused by rare, high-impact weather events. The authors introduce Structure-Grounded Test-Time Adaptation (SG-TTA) framework. The framework consist of two mayor components the Location-Anchored Subgraph Retrieval and Structure-Grounded Test-Time Adaptation. Location-Anchored Subgraph Retrieval models the labeled context by leveraging historical subgraphs relevant to the location and its neighborhood. Structure-Grounded Test-Time Adaptation retrieves the subgraphs and uses LoRA Adapters to apply the test-time adaptation.
Critical strengths
- The paper provides a solution approach to the problem, that extreme weather events and their impact on the power system are not sufficiently captured by current vanilla TFMs.
- Ablation Study shows strengths really well. I like it.
Critical weaknesses
-
Limitations and weaknesses of the method, its transferability, and possible future work are generally not discussed in the paper. Is the method adaptable to other extreme events, such as floods in flood-prone regions? Is the method adaptable to geographical locations other than the US?
-
We need for you ablation experiments and all experiments a little more description. Please use the appendix for this and give additional input for this.
-
The paper's formatting does not meet ICLR's requirements of centered tables. Furthermore, Tables 2,3 and 4 are not chronologically numbered.
My general merit is positive, I like the topic of the paper, I like the idea of modifying both the in-context samples and the test-time adaptation within. Therefore, my merit is a weak accept as it shows an use-case of machine learning in the age of TSFM for a very special and interesting use-case.
Questions for authors
- Notation needs explanation. What is the meaning of , especially in regard to C (the context)
- LoCalPFN and G2T-FM what inputs do they receive ? Can you possibly describe them more in the Appendix to ensure a fair comparison.
- Is vanilla without in context examples, therefore without fit ?
- Not sure about the 5 fold crossfold. Could time be a factor ? Because the split does not respect timely ordering.
- I really like The Ablation Study. But which parts are hold flexible and which parts are ablated. For Table 2 the Location-Anchored Subgraph Retrieval is alternated, is the Structure-Grounded Test-Time Adaptation kept? For Table 3 the main topic is the Structure-Grounded Test-Time Adaptation is for this the Location-Anchored Subgraph Retrieval kept? Is it possible to show the results of Location-Anchored Subgraph Retrieval without using Structure-Grounded Test-Time Adaptation and the other way around? Further, sometimes some of those methods in the ablation needs to be clarified. Can you elaborate about covariate KNN and task aligned KNN in Appendix? Simply on sentence what goes in and out.
4 comments
Reply to Reviewer ASgu: Q1-Q4
Q1. Notation explanation.
We thank the reviewer for pointing this out. The symbol in the question did not render on our side; given the reference to the context , we understand it to be the superscript in , and we would be glad to clarify any other notation.
The superscript marks quantities associated with the incoming target event, such as its covariates and predictions . Accordingly, denotes the context constructed for that event by pooling and deduplicating the labeled rows of the retrieved historical subgraphs; it contains only training-fold events and is shared by all locations of the target event.
Q2. LoCalPFN and G2T-FM what inputs do they receive? Can you possibly describe them more in the Appendix to ensure a fair comparison.
We thank the reviewer for this question and will describe both baselines in detail in the appendix.
LoCalPFN. Our LoCalPFN implementation uses local retrieval followed by supervised fine-tuning. For each location in a test event, we rank rows from the training fold by cosine similarity to the query in the standardized covariate space. We retrieve the 10 most similar rows per location and pool them across locations to form a labeled historical context for the event. For fine-tuning, we randomly split this pool into support rows and pseudo-query rows. The model conditions on the support features and labels and predicts the outage counts for the pseudo-queries. We compute the regression loss against their observed historical outage labels.
G2T-FM. We sample historical context rows from the training fold using the same random sampling procedure as the vanilla backbone. Each location is a node in the spatial graph. For each node, we concatenate its own covariates, the mean covariates of its one-hop neighbors, and the node's coordinates in the graph Laplacian eigenvectors. We use these graph-augmented features for both historical and test rows. For a controlled adaptation comparison, we additionally apply the same supervised fine-tuning procedure as for LoCalPFN, using support and pseudo-query rows sampled from the historical context.
Q3. Is vanilla without in context examples, therefore without fit?
Vanilla does use in-context examples. Both vanilla TabPFN and TabICL condition on 50K randomly sampled, labeled training-fold rows to predict the incoming event. “Vanilla” means using the pretrained backbone without our retrieval or test-time parameter updates; it does not mean prediction without context. Any preprocessing is fitted using the training context, while the backbone parameters remain frozen.
Q4. Not sure about the 5 fold crossfold. Could time be a factor? Because the split does not respect timely ordering.
Thank you for raising this question. We sorted the events by year and divided them into five contiguous folds, each covering a continuous range of years. Each fold is held out once as the test set, and the remaining four folds are used for training. Because folds are contiguous in time, temporally adjacent events, such as consecutive storms with similar weather conditions, almost always fall in the same fold. They can fall on opposite sides of the train-test split only at the boundaries between folds, which limits leakage from neighboring events.
This is a blocked design rather than a strictly forward-in-time evaluation: except when the last fold is held out, the training data include events that occurred after the test period. This does not expose the model to the test events themselves. Each event is predicted only from its own pre-event WRF weather forecast and static covariates, the events are independent storms, and the model uses no temporal sequence information. Training on later events therefore provides no direct information about the outcome of a test event.
Reply to Reviewer ASgu: Q5
Q5. Ablation settings and clarification of retrieval baselines.
We thank the reviewer for the positive comment and for this question. In Table 2, the TabPFN backbone is frozen and only the construction of the context varies, so no Structure-Grounded Test-Time Adaptation is applied. In Table 3, the retrieved context from Location-Anchored Subgraph Retrieval is held fixed, and only the adaptation mechanism or objective varies.
We additionally provide the complete comparison across all four territories: Random/Frozen, Random/TTA, Retrieved/Frozen, and Retrieved/TTA.
| Domain | Random / Frozen | Random / TTA | Retrieved / Frozen | Retrieved / TTA (SG-TTA) |
|---|---|---|---|---|
| CT | 367.8 / 98.6 | 336.6 / 84.5 | 321.7 / 87.7 | 309.8 / 66.7 |
| NH | 114.9 / 75.9 | 112.2 / 63.5 | 106.3 / 65.8 | 101.9 / 53.3 |
| WMA | 39.8 / 97.1 | 38.2 / 76.1 | 39.0 / 76.4 | 35.3 / 61.2 |
| EMA | 152.8 / 115.1 | 121.4 / 80.5 | 142.0 / 92.2 | 131.4 / 66.2 |
Both components contribute to the improvements. With the backbone frozen, retrieved context improves MAE and MAPE over random context in all four territories. With retrieved context held fixed, enabling TTA further reduces MAE by 11.9, 4.4, 3.7, and 10.6 on CT, NH, WMA, and EMA, respectively, and MAPE by 21.0, 12.5, 15.2, and 26.0 percentage points. TTA also improves both metrics under random context across all four territories. These results show that the gains are not attributable to retrieval alone. SG-TTA achieves the lowest MAPE in all four territories and the lowest MAE in three; on EMA, Random/TTA achieves lower MAE, indicating that combining retrieval and adaptation does not uniformly improve every metric.
Definitions of the kNN variants. We will add the following descriptions to the appendix:
- Covariate kNN: Given a target location’s standardized covariates, it selects the 10 training-fold rows with the highest cosine similarity and pools the selected rows across target locations to form the labeled context.
- Task-aligned kNN: Given the learned retrieval embedding of a target location, it selects the 10 nearest training-fold rows in that embedding space and pools them across target locations to form the labeled context.
Reply to Reviewer ASgu
Dear Reviewer,
Thank you very much for your time and constructive feedback on our submission. We sincerely appreciate your constructive comments and suggestions.
We have carefully considered your concerns and provided detailed point-by-point responses in our rebuttal. We would be very grateful if you could take a moment to review our responses when you have a chance.
We hope the revisions and clarifications have addressed your main concerns. We would also greatly appreciate any additional feedback and would be happy to further discuss any remaining questions or suggestions.
Best regards,
The Authors
Reply to Reviewer ASgu: W1 & W2 & W3
W1. Limitations, transferability and possible future work.
We thank the reviewer for this suggestion and will add a dedicated Limitations and Future Work section to the revision.
Limitations. Our evaluation covers four territories of a single utility in the northeastern US, and the most severe events are rare, so evidence for them is descriptive.
Transferability. SG-TTA is potentially transferable to other extreme events, such as floods and wildfires. These tasks can share the spatial dependencies and scarcity of severe-event examples that motivate our framework. When historical events provide labeled observations at recurring locations, the tabular backbone, historical-context retrieval, and event-specific adaptation procedure can be reused.
Transfer would require task-specific covariates and targets, retraining the retrieval encoder, and potentially modifying the graph. For example, incorporating drainage connectivity for flood prediction. Historical neighborhoods may provide poor references for unprecedented events, and the structural objectives would require validation for the new task. The framework is not inherently US-specific, but geographical transfer similarly requires suitable local records and validation under different environmental conditions.
Future work. We would like to extend the framework to additional extreme-event prediction tasks and regions outside the US, including exploring cross-region retrieval where local historical data are limited.
W2. More detailed descriptions of the ablation study and experiments.
We summarize the setup of each experiment below and will add a dedicated appendix with these details to the revision.
Ablation experiments of context (Table 2). The TabPFN backbone remains frozen, and only context construction changes. Full history uses all training-fold rows, while Random samples 50K rows. Covariate kNN retrieves the 10 nearest training rows per target location in standardized covariate space; Task-aligned kNN uses the learned retrieval embeddings instead. Both pool the selected rows across the event. Our method retrieves three historical subgraphs per target location and pools their rows after deduplication. Thus, this experiment evaluates context construction without structural TTA.
Ablation experiments of adaptation (Table 3). The historical context C⋆ is held fixed while the adaptation mechanism or objective changes. Frozen TabPFN performs no parameter updates. Supervised fine-tuning updates the adapters on pooled historical events from the training fold. Supervised fine-tuning on C⋆ splits the retrieved historical rows into support and pseudo-query sets and updates the adapters using the pseudo-query labels.
Neighbor-feature concatenation augments the input with mean neighbor covariates. Direct hidden refinement optimizes target representations rather than model parameters. The trained graph adapter inserts a graph attention layer as adapter.
The Lrel only and Lself only variants retain one component of the structural objective, while the shuffled-reference variant replaces each location's reference distribution with that of a randomly chosen location, disrupting the correspondence between target locations and historical references. All variants that update parameters use the same optimizer, learning rate, number of steps, and adapted layers as SG-TTA.
W3. Formatting.
We thank the reviewer for pointing out these formatting issues, and we will correct these problems in the revision.