- Rating
- 2
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes FUSE, a forecast-refinement framework that uses predicted future states as guidance rather than direct decoding targets. Motivated by Predicted State Deviation, where predicted future states can be farther from the true future state than the encoded base forecast, FUSE learns how far to move the base state along or against the predicted displacement and further adjusts the reconstruction residual. The method can be attached to frozen forecasting backbones and shows consistent improvements across multiple datasets and architectures.
Critical strengths
- The core motivation is clear and well connected to prior future-representation forecasting methods.
- The idea of treating the predicted state as guidance rather than an endpoint is simple but meaningful, and the geometric analysis provides good support for it.
- FUSE works as a plug-in without retraining the forecasting backbone and gives positive average gains across the MLP, DLinear, and iTransformer backbones.
Critical weaknesses
- The “Predicted State Deviation” phenomenon depends partly on the particular autoencoder-defined latent space and predictor quality. It would be useful to establish how robust the observation is to different state-space constructions.
- Appendix D.1 states that lookback length is selected using the lowest test MSE. If this is indeed the protocol used for the main results, it introduces test-set leakage in model selection and should be replaced with validation-based selection.
- The current implementation uses a single global scalar and for all samples within a trained model. Since the optimal state correction is clearly sample-dependent in the geometric analysis, a more fine-grained or adaptive refinement mechanism seems worth exploring.
Questions for authors
- Could the authors clarify whether lookback length is actually selected on the test set? If so, can all results be recomputed using validation-based selection?
- How does FUSE compare with an equally lightweight refinement module operating directly in observation space?
2 comments
Reply to W3 and Q2
W3: Shared coefficients and an adaptive refinement experiment
FUSE is designed as a lightweight refinement module and adapter that improves an existing forecaster without retraining its weights. The predictor supplies input-dependent guidance, while shared coefficients learn how to use it without another coefficient network per window.
For frozen encoder , decoder , base forecast , and predicted future state ,
S_{0,i}&=E(Y_{0,i}),\quad \Delta S_i=\widehat S_i-S_{0,i},\quad R_{0,i}=Y_{0,i}-D(S_{0,i}),\\\\ Y_{\mathrm{ref},i}&=D(S_{0,i}+\alpha\Delta S_i)+(1+\beta)R_{0,i}. \end{aligned}$$ Both $\Delta S_i$ and $R_{0,i}$ vary across windows, so the corrections remain input-dependent. Each trained model learns its own shared $\alpha,\beta$ through the forecasting objective. The window-wise geometric optimum uses the true future for diagnosis (Appendix A.1, lines 864–892); the learned shared scale provides an aggregate compromise across windows. Following your suggestion, we tested input-adaptive $\alpha$, which controls the geometric state step, while keeping $\beta$ shared: $$\alpha_i=a_0+g_\phi(z_i).$$ The shared scalar $a_0$ is learnable; $z_i$ concatenates temporal means and standard deviations of encoded history $E(X_i)$, base state $S_{0,i}$, and displacement $\Delta S_i$. The controller is a 16-hidden-unit MLP with unbounded output, allowing interpolation, extrapolation, or reversal across windows. Both this experiment and the observation-space comparison below use the default MLP base on ETTm1, Weather, Solar, and Electricity, $L=336$, $H\in\\{96,192,336,720\\}$, and seeds 2021/2022. Each contains 32 paired settings, matching data splits, frozen Base, common-core initialization, optimizer settings, training budgets, early stopping, and validation-MSE checkpoint selection. For adaptation, we trained both versions anew (64 runs), sharing the frozen AE and loss weights. Shared FUSE starts at $\alpha=\beta=0$; adaptation starts at $a_0=\beta=0$ with a zero-initialized controller output layer. Both initially reproduce Base; other prediction, decoding, and residual paths are unchanged. | Dataset | Shared: MSE Gain (%) | Adaptive: MSE Gain (%) | |---|---:|---:| | ETTm1 | **$2.59\pm0.23$** | $1.27\pm0.21$ | | Weather | **$0.82\pm0.53$** | $-2.52\pm0.05$ | | Solar | **$2.96\pm0.39$** | $2.17\pm0.43$ | | Electricity | **$3.83\pm0.12$** | $2.52\pm0.24$ | | Mean | **2.55** | 0.86 | Gain is the per-setting percentage reduction in test MSE relative to Base. Entries are mean $\pm$ sample SD across seeds after averaging horizons within each seed; Mean weights datasets equally. All 32 adaptive models vary their coefficients across windows; 30 cross multiple geometric regimes. Adaptation helps on Solar at $H=720$ (5.81% Gain versus 1.37% for sharing), while sharing achieves higher average Gain across each dataset and each horizon. On the same RTX 4090 with FP16 and reused history encoding, complete single-window inference averages 3.981 ms for sharing and 4.778 ms for adaptation, a 20.06% mean paired increase. These results support shared scaling as an effective default for average MSE and inference cost. We have added this comparison and timing analysis to Appendix E.4 (lines 1953–1993; Table 22). **Q2: Comparison with equally lightweight observation-space refinement** We remove the autoencoder, setting $E=D=I$ and $R_0=0$: $$\widehat Y=Q_\phi(X),\qquad Y_{\mathrm{ref}}=Y_0+\alpha(\widehat Y-Y_0),$$ where $X$ is history and $Q_\phi$ predicts a future reference directly in observation coordinates. It uses the same temporal prediction core and a matched trainable parameter budget. Both methods optimize forecasting MSE plus auxiliary MSE and per-timestep cosine losses, with the same weights, on their unscaled references: $\widehat Y$ versus true future $Y$, or $\widehat S$ versus $E(Y)$. Observation-space refinement starts at $\alpha=0$, reproducing Base as FUSE does. | Dataset | Observation space: MSE Gain (%) | FUSE: MSE Gain (%) | |---|---:|---:| | ETTm1 | +1.89 | **+2.59** | | Weather | +0.63 | **+0.82** | | Solar | +2.02 | **+2.96** | | Electricity | +1.07 | **+3.83** | | Mean | +1.40 | **+2.55** | Gains are averaged over horizons and seeds. Observation-space refinement improves average MSE on every dataset (+1.40% overall), supporting our idea of learning to use an auxiliary forecast as a displacement reference. Full FUSE further improves average MSE on every dataset (+2.55% overall), supporting the additional value of its state-representation, decoding, and residual-adjustment pipeline. We have added this comparison to Appendix E.5 (lines 1997–2031; Table 23). **We hope our responses have addressed your concerns and would be grateful if you could reconsider your original rating. Thank you again for your thoughtful feedback and valuable time!**
Reply to W2/Q1 and W1
Thank you for highlighting the protocol wording and the robustness of our motivating observation.
W2 & Q1: Validation-based lookback selection
In our experiments, lookback lengths were selected by validation MSE for each method, dataset, and horizon. The phrase “lowest test MSE” in Appendix D.1 was a typographical error. We have corrected it (lines 1495–1497) to:
Test MSE and MAE are reported for the selected configuration. Checkpoints are also selected by validation MSE, as stated in Appendix C.2 (line 1435): “Within each forecasting training run, checkpoints are selected by validation MSE.” Appendix C.3 likewise specifies “validation forecasting MSE” for forecasting checkpoints (lines 1455–1457). This correction describes the protocol used for the submitted results; the protocol and results are unchanged. We apologize for the confusion.
W1: Robustness across state spaces and predictor quality
We evaluated six state-space constructions with three predictor configurations (576 models): the original AE, a deeper AE, a denoising AE, a low-dimensional AE, truncated PCA, and Identity. The original AE uses state dimensions 32/64/256/512 for ETTm1/Weather/Solar/Electricity, respectively; the low-dimensional AE and PCA use ⌈C/2⌉, where C is the number of observed variables. Identity uses observation coordinates directly.
For each space, we evaluate a standalone Direct predictor, Larger Direct (twice the hidden width and four blocks instead of two), and the predictor jointly trained in FUSE. Direct predictors receive both observation-space forecasting MSE and state MSE/cosine supervision. Base forecasters and representations remain frozen. All conditions use the default MLP base, L=336, H∈96,192,336,720, and seeds 2021/2022: 32 settings per condition, with validation-MSE checkpoint selection.
Let Y0 be the base forecast, Y the true future, and S the predicted state. With the encoder E of each construction, S0=E(Y0),S⋆=E(Y),ΔS=S−S0. Here, pw counts windows where S is farther from S⋆ than S0; pws counts windows that have both a worse endpoint and a state on the segment from S0 to S closer to S⋆ than S0. Both percentages use all test windows as the denominator and are averaged equally across settings.
PSD and useful intermediate positions appear across all six constructions. Standardizing active coordinates by their training-set SD (≥10−6) likewise preserves the observation: Direct's mean pw ranges from 39.86% to 57.28%, and pws from 31.92% to 41.26%.
Across all 192 tested settings, the wider and deeper predictor still exhibits PSD. Even in the 44 capacity pairs where Larger Direct reduces aggregate test state error, average pw remains 46.75% (versus 48.75% for Direct), and pws remains 33.62% (versus 34.83%). Under the original AE, Direct's mean state-error ratio relative to encoded Base is 0.951, while pw remains 40.41%. Thus, better aggregate state prediction still leaves windows with worse endpoints and useful intermediate states.
A within-checkpoint comparison connects this observation to forecasting: holding predicted states fixed and setting β=0 in both paths, the learned scale yields lower test MSE than endpoint use (α=1) across all six spaces and all 192 settings. These results support both the robustness of PSD across the tested constructions and the value of learning how to use a predicted reference.
We have added this study to Appendix A.4 (lines 1018–1097; Table 9).