- Rating
- 4(+2)
- Confidence
- 3
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes InRelCast for time series forecasting with exogenous variables. The method combines an endogenous forecasting backbone with a residual forecaster whose parameters are modulated by inferred relation representations. Historical conditions and observed residuals are encoded as demonstrations, from which a context encoder and relation decoder infer representations for future prediction. Reconstruction and pseudo-query objectives support this learning process.
The paper evaluates the method on several forecasting benchmarks, including cross-dataset transfer and longer-horizon forecasting. The results are promising and show improvements over a broad set of baselines. My main reservation is whether the experiments sufficiently establish the proposed mechanism (learning predictive relations from historical demonstrations) as the source of these improvements.
Critical strengths
-
A well-motivated forecasting problem. Exploiting historical target–covariate relationships to condition future predictions is a reasonable approach when these relationships vary across series or over time.
-
A coherent architectural design. The combination of an endogenous backbone, residual modeling, and conditional parameter modulation is technically interesting. The reconstruction and pseudo-query objectives provide explicit training signals for the proposed relation representations.
-
Broad and encouraging empirical results. The evaluation covers multiple application domains, transfer settings, and prediction horizons. The ablations examine several meaningful design choices, including relation inference, auxiliary objectives, context encoding, and parameter modulation. The supplementary fixed-configuration experiment also provides useful evidence that the results are not entirely dependent on selecting a separate configuration for each horizon.
Critical weaknesses
W1. The experiments do not yet isolate learning from historical demonstrations from ordinary conditional prediction.
The future query already contains an endogenous representation and exogenous information. A sufficiently expressive query-conditioned predictor could therefore perform well without substantially using the historical condition–response associations stored in the demonstration memory.
The existing ablations establish that parts of the proposed architecture are useful, but do not directly distinguish these explanations. Removing relation inference or replacing inferred relations with a pooled representation changes more than access to informative demonstrations. Similarly, comparing parameter modulation against input conditioning addresses how conditioning is implemented, rather than whether the model learns from historical demonstrations.
This distinction matters because in-context relation learning is the central contribution. A capacity-comparable query-only predictor, together with an intervention that shuffles the historical responses while preserving the conditions, would provide more direct evidence.
The interpretation of the latent codes also deserves some restraint. Since historical codes receive the observed residual and are trained to reconstruct it, successful reconstruction does not by itself establish that they represent a transferable relationship. They may encode residual-specific information. This does not invalidate the predictor, but limits what can be concluded about its mechanism.
W2. The coverage of the pseudo-query objective is limited for some long-horizon configurations.
The pseudo-query task is constructed by splitting the observed history, so its maximum prediction offset is bounded by the number of historical patches. This can be substantially shorter than the actual forecasting horizon. For example, with a context length of 96 and patch length of 12, there are eight historical patches, allowing at most six pseudo-query offsets under the described construction, whereas a horizon of 720 requires 60 future patches.
The forecasting loss still supervises the full output horizon; this is therefore not an absence of long-horizon training. However, the auxiliary objective directly teaches relation prediction over a much shorter range than the range over which the inferred relations are subsequently used. Consequently, the aggregate long-horizon results do not establish how much this objective contributes at distant forecast offsets. A breakdown of its benefit by prediction offset would clarify the scope of the claimed mechanism.
W3. Architectural variation and configuration selection make some performance gains difficult to attribute.
The released configurations for the common multivariate benchmarks enable options such as endo_direct and endo_linear, and occasionally exog_linear. These introduce alternative forecasting paths, including direct horizon prediction and linear contributions that bypass the relation-inference pathway. The appendix briefly mentions optional linear mappings, but their contribution to the reported results is not separately established.
These additions may be useful modeling choices. Nevertheless, the distinction matters when improvements are attributed to the proposed relation-learning architecture. Results for a clearly specified default model, accompanied by a compact ablation of these additional paths on representative datasets, would make the evidence easier to interpret.
The tuning protocol is also described more explicitly for some experimental settings than for others. Because several baseline results are taken from prior work, clearer reporting of the proposed method’s search space and selection budget would help assess comparability. I do not regard this as evidence of unfair tuning, and the fixed-configuration results partially mitigate the concern.
W4. The wind experiments suggest an additional explanation for part of the advantage over several baselines.
Table 16 is a valuable diagnostic: explicitly adding wind speed substantially improves several competing methods. For example, the average MSE decreases from 0.961 to 0.504 for DAG and from 1.067 to 0.523 for KITE. InRelCast remains competitive and retains its advantage, which is a positive result.
However, this also suggests that an appreciable part of the original gap to these methods reflects their ability to extract a useful nonlinear feature from the wind components. The experiment therefore supports the effectiveness of the overall architecture more directly than the specific claim about learning relations from demonstrations. Repeating the context-dependence control from W1 with wind speed supplied would help separate these explanations.
Questions for authors
-
How does InRelCast compare with a capacity-comparable query-only modulation model? How much does performance change when historical responses are shuffled across demonstration conditions?
-
How does the benefit of the pseudo-query objective vary with forecast offset, particularly when the forecasting horizon substantially exceeds the offsets covered by this objective?
-
Which reported configurations use the direct or linear forecasting paths, and how much do these paths contribute relative to the default relation-based architecture? Please also clarify the configuration-selection protocol across benchmark groups.
5 comments
Response to Reviewer 7uqe (2/2)
The default model has no bypass path, and it produces every TSF-X result. None of the configurations on the 12 TSF-X datasets, GEFCom2014 wind, or zero-shot transfer enables endo_direct, endo_linear, or exog_linear. Tables 1--3, the ablations, and the shuffle controls above come from the relation-based model alone, as does the ETTh1-720 offset analysis, whose configuration enables none of these paths, and our claims about learning relations from demonstrations rest on these results. The paths are optional components for some of the common multivariate benchmarks, namely ETTh1, ETTh2, ETTm2, and Electricity, where the exogenous variables are other channels of the same system and the target history carries most of the signal. On these benchmarks a direct path from the target history complements the relation pathway, which models the exogenous channels. We now list in Appendix A.2 which configurations enable each path, so that the default model and these variants are clearly separated.
The results do not depend on fine per-horizon tuning. On GEFCom2014 wind, InRelCast and every baseline are tuned with one protocol (Appendix A.2, Table 8), and in zero-shot transfer each model keeps its configuration for the source dataset. Starting from a base configuration, each run changes one hyperparameter among learning rate {1e-4, 5e-4, 1e-3}, width {64, 128, 256}, patch length {8, 12, 24}, and dropout, a final run merges the changes that lower the validation loss, and the run with the lowest validation loss is kept. On the TSF-X and common multivariate datasets, the baseline results are taken from DAG, GCGNet, and KITE where available and are otherwise obtained with the released configurations of the baselines, except GCGNet and KITE on the common multivariate datasets, which we tune with the same protocol (Appendix A.2), and the configuration of InRelCast for every dataset and horizon is released with the code. These results do not depend on fine per-horizon tuning. Table 26 shows that one configuration per dataset across both horizons still gives the best average MSE and MAE on the 12 TSF-X datasets, 0.289 and 0.306 against 0.321 and 0.333 for the second-best GCGNet, and the sensitivity study in Appendix E.2 shows that the MSE stays stable under the rank on all four datasets tested and under every hyperparameter on NP and GEFCom2014 zone 4.
The dependence on the demonstrations remains with wind speed supplied. We repeated the W1 controls on GEFCom2014 wind with the wind speeds at 10 m and 100 m added as exogenous variables, the setting of Table 21. Shuffle-within raises the MSE of InRelCast by 9.0%, Shuffle-cross by 8.6%, and the capacity-matched Query-only model is 11.4% worse. Supplying the nonlinear feature therefore does not remove the dependence on the demonstrations, and we attribute the remaining gap to how each wind farm converts the forecast wind into power. The results are included in Appendix D.6.
Follow-up before the end of the discussion
As the discussion period closes in a few hours, we would be grateful to know whether our response addresses your concerns.
The controls you proposed show that breaking the pairing in the demonstrations raises the error by 8.6% to 32.7% on DE and the wind datasets, and the capacity-matched Query-only model is 11.4% to 40.1% worse. Moreover, no TSF-X configuration enables a bypass path, so every TSF-X result comes from the relation-based model.
We are happy to clarify any remaining point.
Response to Reviewer 7uqe (1/2)
We thank the reviewer for the careful reading and for the precise suggestions, which we followed in the new experiments below. All new results use the evaluation protocol of the paper, and we mark the revised parts of the manuscript in blue.
InRelCast relies on the pairing in its demonstrations. We ran the two controls the reviewer proposed. Shuffle-within permutes the responses across the demonstrations of a window while keeping their conditions, Shuffle-cross takes the responses from another window, and both are applied at test time to the trained model. Query-only removes the demonstrations and is trained with a parameter count within 0.3% of InRelCast, so the comparison holds capacity fixed. We repeated the controls with three seeds on four datasets.
Change in MSE relative to InRelCast of the same run, positive is worse, averaged over horizons 24 and 360, mean and standard deviation over seeds where three seeds are given.
On DE and the wind datasets, breaking the pairing between conditions and responses raises the error by 8.6%--32.7%, and the capacity-matched Query-only model is 11.4%--40.1% worse, with small spread across seeds. A query-only model of the same size therefore does not recover what the demonstrations provide. The effect persists in zero-shot transfer, where the target series is never seen in training and the demonstrations are its only series-specific information. On NP, FR, BE, PJM, and Energy, the changes stay within the seed-to-seed spread. Section 4.3 now reports these controls in Table 6, and Appendix D.6 gives the complete table on all nine settings with standard deviations.
We removed the strong reading of the latent codes. We agree. A historical code receives the observed residual and is trained to reconstruct it, so a low reconstruction error alone does not show that the code represents a transferable relation. We removed this reading from Section 4.3 and now present the relation visualization only as a description of how the inferred vectors vary with the exogenous conditions. The shuffle controls above are the evidence that the relations are used.
The pseudo-query loss helps at all offsets, including those beyond its range on ETTh1-720. The concern applies to our ETTh1-720 setting, where look-back 96 and patch length 8 let the pseudo-queries cover at most 10 offsets while the horizon has 90 patches. We split the test MSE by the true forecast offset and compared InRelCast with the variant trained without Licl.
Change in test MSE from adding Licl, negative means Licl helps.
On ETTh1-720, Licl reduces the error by 35.6% on the offsets that the pseudo-queries never reach. We attribute this to the offset embedding and the shared relation decoder, which let the inference learned on short offsets carry over to later queries. On the TSF-X datasets with long look-back windows, the pseudo-queries already cover the whole horizon. While writing this analysis we also found that Section 3.4 states m∈{1,…,K−2}, whereas the code draws m from {max(1,⌊ρ(K−1)⌋),…,K−2} with ρ=0.25 by default and ρ=0.1 in this ETTh1-720 setting, and we corrected the text. The table is added as Appendix D.7.