- Rating
- 1
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 1
- Soundness
- 1
Read review
Paper summary
"Accuracy measured on a fixed set of stations" and "the ability to forecast at stations never seen during training" are two different abilities, and past evaluations conflated the two. Fixed-station evaluation directly demonstrates only temporal generalization to existing stations. Claiming, on that basis, that a model transfers to unseen stations is not supported by sufficient evidence.
Critical strengths
- The conceptual separation is genuinely valuable. Separating the parameter boundary (whether held-out prediction relies on untrained identity-indexed state) from the information boundary (whether prediction and routing are restricted to information permitted by the split), and further evaluating accuracy separately from transfer validity, is a clean taxonomic contribution.
- The paper raises a valuable question: fixed-station evaluation cannot support claims that a model transfers to unseen stations. It also proposes a new evaluation standard: how to verify that a model has station-transfer capability.
Critical weaknesses
- The paper lacks substantial background information and detailed explanation. The Introduction uses a large number of self-invented terms while citing only one reference, which makes it extremely difficult to read.
- The paper's novel contribution is held-out station forecasting, yet the corresponding held-out experiments include no baseline comparison (see Table 5). The experiments demonstrate forecasting feasibility, but they do not show that the paper solves the unseen-station forecasting problem, or that it outperforms existing transfer methods.
- Ablation experiments. The dual-path ablation does not directly demonstrate that the two paths improve station-transfer capability. This ablation was conducted on fixed-station tasks. It therefore supports: "using local and cross-region communication together improves forecasting at existing stations." It does not support: "the dual paths are the reason for better prediction at unseen stations."
Questions for authors
See weakness