- Rating
- 4(+1)
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ERST, which constructs entity-specific interaction features from changes in an aggregate representation after replacing one entity’s representation with a learned reference. These features update entity representations for forecasting. Experiments cover 12 datasets across five domains, multiple forecasting architectures, component ablations, and efficiency evaluations.
Critical strengths
S1. Comprehensive empirical evaluation. The experiments cover 12 datasets across five spatiotemporal domains, including different system scales and forecasting horizons. The evaluation across four additional forecasting architectures is particularly valuable: ERST improves MAE in 23 of the 24 architecture–dataset combinations and matches the baseline in the remaining case. This provides substantial evidence of its practical utility beyond a single backbone or dataset.
S2. Extensive and informative ablations. The paper investigates the response construction, reference inputs, replacement versus removal, response modulation and gating, and response assignment. The additional five-seed comparisons strengthen the evidence for selected design choices. These experiments go considerably beyond reporting overall forecasting accuracy and help establish which components contribute to performance.
S3. A concrete and computationally practical formulation. The replacement response is mathematically specified, and the shared-mean update avoids explicitly reconstructing the full entity set for every replacement. The resulting mechanism provides a practical way to generate entity-specific features, with supporting evaluations on large systems.
S4. I read the offered code, which is reasonable.
Critical weaknesses
W1. The abstract and introduction emphasize that existing approaches do not measure changes under entity replacement, but do not clearly identify the forecasting limitation this omission creates. They should explain what prediction-relevant information is missing and distinguish ERST as a reusable interaction module from the complete forecasting model built around it.
W2. The evidence does not fully substantiate the interpretation of the response as interaction information. Figure 1 shows that similar raw means can conceal entity-level differences, but does not establish that learned aggregation loses these distinctions. Nor does it explain why replacing a hidden representation with the learned reference captures meaningful interactions. The ablations support the construction’s predictive utility, but leave its advantage over a more direct context-dependent feature insufficiently isolated.
Questions for authors
Q1. What concrete forecasting limitation motivates the replacement response? Could the authors clarify the intended mechanism linking entity-specific replacement sensitivity to improved prediction, and explicitly distinguish the reusable interaction module from the full forecasting architecture?
Q2. How are the scalar values in Figure 1(b–c) obtained and normalized from the vector representations and responses? Which case or cases are used, and how do these plots connect to the observed differences in Figure 1(a)?
Q3. What makes the learned reference a useful comparison point beyond its empirical performance?
Q4. ERST appears to be a plug-and-play interaction module applicable to different forecasting models. Could the authors clarify whether the ERST model in Table 1 uses an existing backbone or a custom architecture? Additionally, given the competitive performance of GWNet, MTGNN, and TWIST in Table 1, why were these models not included in Figure 4? Evaluating ERST with these backbones would help assess whether it provides complementary benefits to their existing interaction mechanisms.
Q5. Attention-based architectures are relatively underrepresented in the evaluation. Could the authors include representative models such as GMAN, PDFormer, and STAEformer, particularly given the competitive performance reported for some of these models in prior evaluations? These comparisons would help assess ERST against attention-based interaction modeling.
4 comments
Q4 — ERST module across forecasting architectures. Tables 1–2 evaluate the complete ERST forecasting model, which uses a custom temporal encoder, entity transformations, ERST interaction layers, and a prediction decoder. The ERST interaction module consists of response construction and update, without the encoder or decoder (Section 3.1). Figure 4 evaluates the module with LSTM, STID, iTransformer, and PatchTST. Appendix I.2 extends this evaluation to GWNet, MTGNN, and TWIST, which already contain interaction mechanisms.
Table 20 reports the MAE of each original architecture and its ERST module variant:
| Dataset | GWNet | MTGNN | TWIST |
|---|---|---|---|
| PEMS03 | 14.61 → 14.18 | 14.91 → 14.66 | 15.11 → 14.91 |
| PEMS04 | 19.35 → 18.21 | 19.17 → 18.54 | 18.51 → 18.05 |
| PEMS07 | 21.22 → 19.85 | 21.02 → 19.96 | 19.70 → 19.40 |
| PEMS08 | 15.06 → 14.18 | 15.06 → 14.53 | 14.21 → 13.96 |
| KnowAir | 14.89 → 13.85 | 14.85 → 13.99 | 19.92 → 13.85 |
| SDWPF | 96.88 → 92.64 | 94.19 → 92.83 | 140.47 → 92.64 |
Each entry shows Original → + ERST Module; lower MAE is better. Adding the module reduces MAE for all three architectures on all six datasets. Across the seven architectures in Table 20, MAE decreases in 41 of 42 comparisons, with one tie. These results evaluate the benefit of adding the ERST interaction module to forecasting architectures that already model interactions. See Section 4.4, Figure 4, and Appendix I.2, Table 20.
Q5 — Attention-based baselines. Table 1 and Appendix F include GMAN, PDFormer, and STAEformer as attention-based forecasting baselines. They are evaluated under the same dataset protocols as the complete ERST forecasting model. Table 1 reports the following MAE:
| Dataset | GMAN | PDFormer | STAEformer | ERST |
|---|---|---|---|---|
| PEMS03 | 14.62 | 15.01 | 15.30 | 14.28 |
| PEMS07 | 20.01 | 20.32 | 19.53 | 19.51 |
| KnowAir | 14.75 | 15.18 | 14.82 | 13.98 |
| SDWPF | 113.84 | 102.89 | 104.05 | 93.08 |
| NYCTaxi-In | 13.75 | 13.51 | 13.74 | 13.43 |
| T-Drive-In | 18.11 | 17.90 | 17.26 | 16.56 |
ERST obtains lower MAE than all three attention baselines on these six datasets. The margins vary by dataset and model; for example, ERST and STAEformer differ by 0.02 MAE on PEMS07. ERST does not have the lowest MAE against these baselines on every dataset; on PEMS08, STAEformer and PDFormer obtain lower MAE (13.62 and 13.67, respectively, versus 14.08 for ERST; Appendix F, Table 8). See Section 4.1, Table 1, and Appendix F for the evaluation protocols and additional results.
Increase my score
Thanks for authors hardworking and careful reply. All of my concerns are addressed. Learning entity's bias compared with normal value via learning a response is interesting and make sense. I will improve my score to 4. Good luck!
We sincerely thank the reviewer for taking the time to carefully reconsider our submission and update the evaluation. Your thoughtful comments and constructive suggestions have been very helpful in improving the clarity and presentation of our work. We greatly appreciate the opportunity to address your concerns and further strengthen the manuscript through the discussion and revision process. Thank you again for your valuable feedback, careful consideration, and continued engagement with our work.
Response to Reviewer UyqZ: Motivation, Learned Reference, and Controlled Evaluations
We thank the reviewer for the detailed comments. We clarified the motivation, Figure 1, and the learned reference, and added controlled comparisons.
W1/Q1 — Motivation and forecasting mechanism. An aggregated representation F(H) combines information from all entities, but its value alone does not explicitly measure how that representation changes when one entity is replaced. This is the change ERST explicitly measures; we do not claim that existing graph or attention methods lose entity information. For entity i, ERST constructs a reference and measures the difference between the outputs of the same aggregation function before and after replacing only that entity's representation. Other entities, normalization, and the function remain fixed (Eq. 7). The response reflects both the change in that entity and the current aggregate context (Eq. 10). It is not a prediction or a causal effect. ERST modulates the response using entity, time, and layer information, then uses a gate to control the resulting update to the existing entity representation. This provides a separate response for each entity, which is used to update that entity's representation before prediction. The decoder predicts from the updated representations (Section 3.3). Removing the response increases PEMS03 MAE from 14.28 to 16.23 and T-Drive MAE from 16.56 to 24.01 (Table 3). The reusable ERST interaction module consists of response construction and update, without an encoder or decoder. Tables 1–2 evaluate a custom forecasting model containing this module; Figure 4 and Appendix I.2 evaluate the module added to other architectures (Section 3.1).
W2/Q2 — Figure 1 and direct-context control. Figure 1(a) plots observed PEMS08 flow for Cases A and B, not the output of learned aggregation. The cases have nearly equal means (296.48 and 296.66), but differ across entities; the six marked entities have the largest observed differences. This motivates examining entity-level changes, but does not show that learned aggregation loses information. Panels (b–c) use Case B at the first interaction layer of a trained ERST model. In (b), each response L2 norm is divided by the mean response norm (dashed line = 1); entities are ordered by their observed A–B differences. In (c), each response is replaced with the mean response vector, and the update term is recomputed in the same model. The plotted value is the L2 change in the update divided by the mean original update norm. Case A is not used as the learned reference in panels (b–c). Neither panel reports prediction error (Appendix I.1, Eqs. 62–63).
To compare the replacement response with a direct contextual feature, the Context Feature variant uses c_i = ρ(φ(b_i)) instead of the replacement response. It uses the same learned reference inputs, modulation, update, decoder, and training settings (Appendix H.1). The following are MAE results under the same dataset protocols:
ERST has lower MAE than Context Feature on all ten datasets in Tables 14–16. The margin is small on PEMS03 and larger on some other datasets. These comparisons support the predictive value of the replacement response beyond the tested Context Feature variant; they do not establish a causal interpretation (Section 4.3 and Appendix H.1).
Q3 — Why use a learned reference? The reference for entity i is constructed from its history, information from other entities, and time and layer embeddings, without directly using the current representation being replaced (Eqs. 2–3). It therefore provides a context-dependent comparison point rather than a fixed value. If the reference equals the current representation, the replacement produces zero response. ERST keeps the other entities, normalization, and aggregation function unchanged, unlike removing an entity (Eq. 7; Appendix B.2).
Under the local approximation in Eq. 10, the response depends on both the difference between the current and reference representations and the sensitivity of the aggregation function at the current mean. The reference is learned through the forecasting objective, rather than representing a unique or causal alternative state.
The five-seed PEMS08 results support its construction: ERST achieves MAE 14.074, compared with 14.228 for ψ = 0, 14.410 without history, and 14.418 without information from other entities. All three differences are significant in paired two-sided tests (p < 0.001). See Section 3.2, Appendix B.2–B.3, and Appendix H.1, Table 17.