acceptodds
Under review as a conference paper at ICLR 2027

Forecast, Guide, Refine: Future State Guidance for Time Series Forecasting

Abstract

Future target representations can complement observation space supervision in multivariate time series forecasting. However, we identify Predicted State Deviation: a future state predicted from encoded history can inaccurately represent the true future, making it unreliable for direct decoding into a final forecast. Yet such an imperfect prediction may still provide useful guidance. We propose FUSE, a future state guidance framework following a forecast, guide, and refine paradigm. FUSE first generates a base forecast in observation space and encodes it as a base state, while predicting a future state from encoded history. Rather than directly decoding the predicted state, FUSE learns how far to move the base state along or against the displacement toward it. The resulting refined state is decoded to observation space and combined with an adjusted reconstruction residual to preserve base forecast information. FUSE can also be directly applied to existing backbones without retraining or modifying the backbone. Experiments on eight benchmarks show competitive accuracy with a simple MLP and consistent average improvements across three backbones. Ablations validate state refinement and residual adjustment, while decomposition analyses reveal corrections in forecast mean, waveform scale, and residual shape.

Primary area:
learning on time series and dynamical systems
Keywords:
multivariate time series forecasting, future state guidance, forecast refinement, representation learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TWR8OpenReview ↗
Rating
2
Confidence
4
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes FUSE, a forecast-refinement framework that uses predicted future states as guidance rather than direct decoding targets. Motivated by Predicted State Deviation, where predicted future states can be farther from the true future state than the encoded base forecast, FUSE learns how far to move the base state along or against the predicted displacement and further adjusts the reconstruction residual. The method can be attached to frozen forecasting backbones and shows consistent improvements across multiple datasets and architectures.

Critical strengths
  1. The core motivation is clear and well connected to prior future-representation forecasting methods.
  2. The idea of treating the predicted state as guidance rather than an endpoint is simple but meaningful, and the geometric analysis provides good support for it.
  3. FUSE works as a plug-in without retraining the forecasting backbone and gives positive average gains across the MLP, DLinear, and iTransformer backbones.
Critical weaknesses
  1. The “Predicted State Deviation” phenomenon depends partly on the particular autoencoder-defined latent space and predictor quality. It would be useful to establish how robust the observation is to different state-space constructions.
  2. Appendix D.1 states that lookback length is selected using the lowest test MSE. If this is indeed the protocol used for the main results, it introduces test-set leakage in model selection and should be replaced with validation-based selection.
  3. The current implementation uses a single global scalar α\alpha and β\beta for all samples within a trained model. Since the optimal state correction is clearly sample-dependent in the geometric analysis, a more fine-grained or adaptive refinement mechanism seems worth exploring.
Questions for authors
  1. Could the authors clarify whether lookback length is actually selected on the test set? If so, can all results be recomputed using validation-based selection?
  2. How does FUSE compare with an equally lightweight refinement module operating directly in observation space?
2 comments

Reply to W2/Q1 and W1

Thank you for highlighting the protocol wording and the robustness of our motivating observation.

W2 & Q1: Validation-based lookback selection

In our experiments, lookback lengths were selected by validation MSE for each method, dataset, and horizon. The phrase “lowest test MSE” in Appendix D.1 was a typographical error. We have corrected it (lines 1495–1497) to:

For each method, dataset, and forecasting horizon, we select the lookback length from L∈336,512,720L\in\\{336,512,720\\} with the lowest validation MSE.

Test MSE and MAE are reported for the selected configuration. Checkpoints are also selected by validation MSE, as stated in Appendix C.2 (line 1435): “Within each forecasting training run, checkpoints are selected by validation MSE.” Appendix C.3 likewise specifies “validation forecasting MSE” for forecasting checkpoints (lines 1455–1457). This correction describes the protocol used for the submitted results; the protocol and results are unchanged. We apologize for the confusion.

W1: Robustness across state spaces and predictor quality

We evaluated six state-space constructions with three predictor configurations (576 models): the original AE, a deeper AE, a denoising AE, a low-dimensional AE, truncated PCA, and Identity. The original AE uses state dimensions 32/64/256/512 for ETTm1/Weather/Solar/Electricity, respectively; the low-dimensional AE and PCA use ⌈C/2⌉\lceil C/2\rceil, where CC is the number of observed variables. Identity uses observation coordinates directly.

For each space, we evaluate a standalone Direct predictor, Larger Direct (twice the hidden width and four blocks instead of two), and the predictor jointly trained in FUSE. Direct predictors receive both observation-space forecasting MSE and state MSE/cosine supervision. Base forecasters and representations remain frozen. All conditions use the default MLP base, L=336L=336, H∈96,192,336,720H\in\\{96,192,336,720\\}, and seeds 2021/2022: 32 settings per condition, with validation-MSE checkpoint selection.

Let Y0Y_0 be the base forecast, YY the true future, and S^\widehat S the predicted state. With the encoder EE of each construction, S0=E(Y0),S⋆=E(Y),ΔS=S^−S0.S_0=E(Y_0),\qquad S^\star=E(Y),\qquad \Delta S=\widehat S-S_0. Here, pwp_w counts windows where S^\widehat S is farther from S⋆S^\star than S0S_0; pwsp_{ws} counts windows that have both a worse endpoint and a state on the segment from S0S_0 to S^\widehat S closer to S⋆S^\star than S0S_0. Both percentages use all test windows as the denominator and are averaged equally across settings.

State spaceDirect: pwp_w / pwsp_{ws} (%)Larger Direct: pwp_w / pwsp_{ws} (%)FUSE: pwp_w / pwsp_{ws} (%)
Original AE40.41 / 33.0145.54 / 37.7240.90 / 33.03
Deeper AE42.38 / 34.5343.59 / 35.5142.50 / 34.67
Denoising AE46.12 / 36.1347.67 / 37.0549.10 / 38.73
Low-dim AE48.94 / 35.9250.95 / 37.7852.70 / 38.48
Truncated PCA62.76 / 36.2064.95 / 38.1265.14 / 38.14
Identity57.28 / 31.9259.08 / 34.2457.13 / 32.07

PSD and useful intermediate positions appear across all six constructions. Standardizing active coordinates by their training-set SD (≥10−6\ge 10^{-6}) likewise preserves the observation: Direct's mean pwp_w ranges from 39.86% to 57.28%, and pwsp_{ws} from 31.92% to 41.26%.

Across all 192 tested settings, the wider and deeper predictor still exhibits PSD. Even in the 44 capacity pairs where Larger Direct reduces aggregate test state error, average pwp_w remains 46.75% (versus 48.75% for Direct), and pwsp_{ws} remains 33.62% (versus 34.83%). Under the original AE, Direct's mean state-error ratio relative to encoded Base is 0.951, while pwp_w remains 40.41%. Thus, better aggregate state prediction still leaves windows with worse endpoints and useful intermediate states.

A within-checkpoint comparison connects this observation to forecasting: holding predicted states fixed and setting β=0\beta=0 in both paths, the learned scale yields lower test MSE than endpoint use (α=1\alpha=1) across all six spaces and all 192 settings. These results support both the robustness of PSD across the tested constructions and the value of learning how to use a predicted reference.

We have added this study to Appendix A.4 (lines 1018–1097; Table 9).

Reply to W3 and Q2

W3: Shared coefficients and an adaptive refinement experiment

FUSE is designed as a lightweight refinement module and adapter that improves an existing forecaster without retraining its weights. The predictor supplies input-dependent guidance, while shared coefficients learn how to use it without another coefficient network per window.

For frozen encoder EE, decoder DD, base forecast Y0,iY_{0,i}, and predicted future state S^i\widehat S_i,

S_{0,i}&=E(Y_{0,i}),\quad \Delta S_i=\widehat S_i-S_{0,i},\quad R_{0,i}=Y_{0,i}-D(S_{0,i}),\\\\ Y_{\mathrm{ref},i}&=D(S_{0,i}+\alpha\Delta S_i)+(1+\beta)R_{0,i}. \end{aligned}$$ Both $\Delta S_i$ and $R_{0,i}$ vary across windows, so the corrections remain input-dependent. Each trained model learns its own shared $\alpha,\beta$ through the forecasting objective. The window-wise geometric optimum uses the true future for diagnosis (Appendix A.1, lines 864–892); the learned shared scale provides an aggregate compromise across windows. Following your suggestion, we tested input-adaptive $\alpha$, which controls the geometric state step, while keeping $\beta$ shared: $$\alpha_i=a_0+g_\phi(z_i).$$ The shared scalar $a_0$ is learnable; $z_i$ concatenates temporal means and standard deviations of encoded history $E(X_i)$, base state $S_{0,i}$, and displacement $\Delta S_i$. The controller is a 16-hidden-unit MLP with unbounded output, allowing interpolation, extrapolation, or reversal across windows. Both this experiment and the observation-space comparison below use the default MLP base on ETTm1, Weather, Solar, and Electricity, $L=336$, $H\in\\{96,192,336,720\\}$, and seeds 2021/2022. Each contains 32 paired settings, matching data splits, frozen Base, common-core initialization, optimizer settings, training budgets, early stopping, and validation-MSE checkpoint selection. For adaptation, we trained both versions anew (64 runs), sharing the frozen AE and loss weights. Shared FUSE starts at $\alpha=\beta=0$; adaptation starts at $a_0=\beta=0$ with a zero-initialized controller output layer. Both initially reproduce Base; other prediction, decoding, and residual paths are unchanged. | Dataset | Shared: MSE Gain (%) | Adaptive: MSE Gain (%) | |---|---:|---:| | ETTm1 | **$2.59\pm0.23$** | $1.27\pm0.21$ | | Weather | **$0.82\pm0.53$** | $-2.52\pm0.05$ | | Solar | **$2.96\pm0.39$** | $2.17\pm0.43$ | | Electricity | **$3.83\pm0.12$** | $2.52\pm0.24$ | | Mean | **2.55** | 0.86 | Gain is the per-setting percentage reduction in test MSE relative to Base. Entries are mean $\pm$ sample SD across seeds after averaging horizons within each seed; Mean weights datasets equally. All 32 adaptive models vary their coefficients across windows; 30 cross multiple geometric regimes. Adaptation helps on Solar at $H=720$ (5.81% Gain versus 1.37% for sharing), while sharing achieves higher average Gain across each dataset and each horizon. On the same RTX 4090 with FP16 and reused history encoding, complete single-window inference averages 3.981 ms for sharing and 4.778 ms for adaptation, a 20.06% mean paired increase. These results support shared scaling as an effective default for average MSE and inference cost. We have added this comparison and timing analysis to Appendix E.4 (lines 1953–1993; Table 22). **Q2: Comparison with equally lightweight observation-space refinement** We remove the autoencoder, setting $E=D=I$ and $R_0=0$: $$\widehat Y=Q_\phi(X),\qquad Y_{\mathrm{ref}}=Y_0+\alpha(\widehat Y-Y_0),$$ where $X$ is history and $Q_\phi$ predicts a future reference directly in observation coordinates. It uses the same temporal prediction core and a matched trainable parameter budget. Both methods optimize forecasting MSE plus auxiliary MSE and per-timestep cosine losses, with the same weights, on their unscaled references: $\widehat Y$ versus true future $Y$, or $\widehat S$ versus $E(Y)$. Observation-space refinement starts at $\alpha=0$, reproducing Base as FUSE does. | Dataset | Observation space: MSE Gain (%) | FUSE: MSE Gain (%) | |---|---:|---:| | ETTm1 | +1.89 | **+2.59** | | Weather | +0.63 | **+0.82** | | Solar | +2.02 | **+2.96** | | Electricity | +1.07 | **+3.83** | | Mean | +1.40 | **+2.55** | Gains are averaged over horizons and seeds. Observation-space refinement improves average MSE on every dataset (+1.40% overall), supporting our idea of learning to use an auxiliary forecast as a displacement reference. Full FUSE further improves average MSE on every dataset (+2.55% overall), supporting the additional value of its state-representation, decoding, and residual-adjustment pipeline. We have added this comparison to Appendix E.5 (lines 1997–2031; Table 23). **We hope our responses have addressed your concerns and would be grateful if you could reconsider your original rating. Thank you again for your thoughtful feedback and valuable time!**
Reviewer_RtQYOpenReview ↗
Rating
3(+1)
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work proposes FUSE, a refinement step on top of a frozen base forecaster. A per-timestep autoencoder defines a latent state space, a separate predictor maps the encoded history to a future state, and the base forecast is moved in latent space toward the predicted state by a learned scalar α, decoded, and corrected by the reconstruction residual scaled by β. The motivation is that the predicted state is usually farther from the target than the encoded base forecast (Predicted State Deviation). On eight standard benchmarks, FUSE with a simple MLP base is reported to be competitive with recent forecasters, and it improves MLP, DLinear, and iTransformer bases on all eight benchmarks at about 1 ms extra inference cost. Ablation studies cover α, β, and the state losses.

Critical strengths
  1. The method is simple and cheap. It adds two scalars on top of frozen components, recovers the base forecast exactly at α = 0, and gives consistent gains over three base models.
  2. The learned α values are reported (Tables 14-15), so one can see what the refinement actually does.
Critical weaknesses
  1. The guide step, a predictor from context embeddings to the embedding of the future under a frozen target encoder, is a JEPA objective [1, 2], but this work does not use the term, cites I-JEPA [1] only once as a masked-prediction method, and does not cite LeCun [2], V-JEPA [3], or the time-series versions TS-JEPA [4] and LaT-PFN [5]. A JEPA-style latent forecaster with a decoding head is the natural baseline and is missing from Table 1. The Predicted State Deviation argument is also not convincing. It compares a predictor trained only with state losses against a tuned base forecaster, so the predicted state being farther from the target is expected.
  2. What the latent space contributes is not shown. Since the predicted state comes from a separately trained predictor, FUSE is a learned blend of two forecasters plus a residual term. The ablation studies vary α, β, and the losses but never the space itself; a blend in observation space, or FUSE with an identity encoder, would show whether the autoencoder matters. The fixed scalar α is also questionable. Table 15 shows α varying widely across seeds and horizons for the same dataset, and an input-dependent α is never tried.
  3. All experiments use small base models on the usual benchmarks, the margins over TimeAlign in Table 1 are small with no standard deviations, and whether the method helps a time-series foundation model as the base is not tested. Fig. 5 excludes autoencoder pretraining and the base forecaster also has to be trained, so the reported cost understates the full pipeline.

[1] Assran M, Duval Q, Misra I, et al. Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture. CVPR 2023.

[2] LeCun Y. A Path Towards Autonomous Machine Intelligence. OpenReview, Version 0.9.2, 2022.

[3] Bardes A, Garrido Q, Ponce J, et al. Revisiting Feature Prediction for Learning Visual Representations from Video. TMLR 2024.

[4] Ennadir S, Golkar S, Sarra L. Joint Embeddings Go Temporal. NeurIPS 2024 Workshop on Time Series in the Age of Large Models (arXiv:2509.25449).

[5] Verdenius S, Zerio A, Wang RLM. LaT-PFN: A Joint Embedding Predictive Architecture for In-context Time-series Forecasting. arXiv:2405.10093, 2024.

Questions for authors
  1. How does FUSE relate to JEPA [2], and how does it compare with a JEPA-style latent forecaster [4, 5] under the same protocol?
  2. Does the gain survive when the blend is done in observation space, or with an identity encoder?
  3. Does FUSE help when a time-series foundation model (Chronos, TimesFM, Moirai, Sundial) is the frozen base?
  4. Why a fixed scalar α rather than an input-dependent one?
4 comments

Reply to W1 & Q1: JEPA connection, latent forecasting baselines, and PSD

Thank you for highlighting the JEPA connection and the missing latent forecasting comparisons.

1. Relationship to JEPA

Guide shares JEPA's principle of predicting target embeddings from context. We have expanded Related Work (Section 2, lines 135–147) to discuss LeCun [2], V-JEPA [3], TS-JEPA [4], and LaT-PFN [5], alongside I-JEPA [1]. FUSE uses a reconstruction-pretrained, frozen state space and studies how a future-state estimate can refine an existing forecast. The estimate defines a displacement from the encoded base forecast, with a learned scale and reconstruction-residual adjustment. Our contribution concerns how predicted states are used for refinement.

2. Two targeted comparisons under a common evaluation protocol

We added two more experiments. Both experiments use ETTm1, Weather, Solar, and Electricity, L=336L=336, H∈96,192,336,720H\in\\{96,192,336,720\\}, and two seeds 2021/2022. FUSE uses the default MLP base.

A. Direct latent forecasting. We independently train Y^direct=D(Pθd(E(X))),\widehat Y_{\mathrm{direct}}=D(P_{\theta_d}(E(X))), where XX is history and E,DE,D are FUSE's frozen encoder and decoder. We match the predictor architecture, common-module initialization, optimizer settings, training budget, and validation-MSE checkpoint selection. The predictor is optimized for its own decoded forecast using forecasting MSE plus the same state MSE/cosine losses and weights as FUSE.

B. TS-JEPA. We use its official implementation and retain its pretraining and prediction-head pipeline. TS-JEPA [4] provides a dataset-specific training baseline; LaT-PFN [5] targets zero-shot in-context forecasting after synthetic-prior pretraining. TS-JEPA and FUSE use identical data splits, preprocessing, history length, target channels, horizons, test windows, and MSE/MAE evaluation. TS-JEPA hyperparameters, including learning rates, are selected on the validation set; forecasting checkpoints are selected by validation MSE.

3. Results addressing the baseline and PSD concerns

The table reports MSE / MAE averaged over horizons and seeds. We have added the Direct latent comparison to Appendix E.3 (lines 1917–1937; Table 20).

DatasetDirect latentTS-JEPAFUSE
ETTm10.3439 / 0.37640.3441 / 0.37560.3373 / 0.3686
Weather0.2285 / 0.26770.2258 / 0.26170.2199 / 0.2538
Solar0.2164 / 0.24800.2201 / 0.24080.2053 / 0.2322
Electricity0.1677 / 0.26970.1571 / 0.25250.1547 / 0.2452

For the requested latent forecaster with a decoding head, FUSE achieves lower average MSE and MAE on all four datasets, with average reductions of 4.844.84\\% and 5.805.80\\% over Direct. For the published JEPA-style baseline, FUSE also has lower average MSE and MAE on all four datasets, reducing them by 3.253.25\\% and 2.872.87\\% over TS-JEPA. These reductions are computed per dataset–horizon pair after seed averaging, then averaged equally.

For the concern about state-only supervision versus a tuned base, Experiment A is trained with both forecasting and state supervision. We diagnose its predicted states against target S⋆=E(Y)S^\star=E(Y) and base state S0=E(Y0)S_0=E(Y_0), where YY is the true future and Y0Y_0 is the base forecast. Here, pwp_w is the percentage of windows with a predicted endpoint farther from S⋆S^\star than S0S_0; pwsp_{ws} is the joint percentage with both a worse endpoint and an interpolated state closer to S⋆S^\star than S0S_0. Both use all test windows as the denominator. The PSD diagnostic is included in Appendix E.3 (lines 1938–1949; Table 21).

Datasetpwp_w (%)pwsp_{ws} (%)
ETTm143.2029.05
Weather49.6036.58
Solar24.8722.41
Electricity28.5128.51
Mean36.5529.14

Percentages are averaged equally over horizons and seeds. PSD and useful intermediate states thus persist in a predictor directly optimized for forecasting accuracy, consistent with their presence in jointly trained FUSE (Appendix A.3, Table 8, lines 1009–1015). Table 18 (lines 2118–2132) connects this geometry to forecasting: holding predicted states fixed and setting β=0\beta=0, learning the scale improves default-MLP average MSE Gain from −2.733-2.733\\% to +1.842+1.842\\% over direct endpoint use. Together, these results support separating endpoint accuracy from guidance utility.

Reply to W2, Q2 & Q4: State-space contribution and shared refinement

We tested observation-space refinement and input-adaptive scaling on the same evaluation matrix: default MLP base; ETTm1, Weather, Solar, and Electricity; L=336L=336; H∈96,192,336,720H\in\\{96,192,336,720\\}; two seeds 2021/2022. Each experiment contains 32 paired settings, with matched training budgets and validation-MSE checkpoint selection.

1. Observation-space blending / identity encoder (W2 & Q2)

Setting E=D=IE=D=I gives R0=0R_0=0 and

Y_{\mathrm{ref}}=Y_0+\alpha(\widehat Y-Y_0),$$ where $X$ is history, $Y_0$ is the frozen base forecast, and $Q_\phi$ predicts a future reference in observation coordinates. The scalar $\alpha$ is learned, shared across samples, and unrestricted in sign. Identity and FUSE use the same temporal prediction core, matched trainable parameter budgets, paired core initialization, frozen Base, optimizer settings, and early stopping. Both optimize forecasting MSE and auxiliary MSE/per-timestep cosine losses on their unscaled references, with identical loss weights in their respective coordinates. Identity starts at $\alpha=0$ and FUSE at $\alpha=\beta=0$, initially reproducing Base. We have added this comparison to Appendix E.5 (lines 1997–2031; Table 23). | Dataset | Identity: MSE Gain (%) | FUSE: MSE Gain (%) | |---|---:|---:| | ETTm1 | +1.89 | **+2.59** | | Weather | +0.63 | **+0.82** | | Solar | +2.02 | **+2.96** | | Electricity | +1.07 | **+3.83** | | Mean | +1.40 | **+2.55** | Gain is the per-setting test MSE reduction relative to Base, averaged equally over horizons and seeds. Identity improves average MSE on every dataset (+1.40% overall): learning to use an auxiliary forecast as a displacement reference is useful even in observation coordinates. Full FUSE further improves average MSE on every dataset (+2.55% overall), supporting the added value of its complete state-representation, decoding, and residual-adjustment pipeline. **2. Explanation of Table 19 variation (W2: seeds and horizons)** Each trained model has its own predictor and shared $\alpha$; Table 19 pools separate horizons and training runs (lines 2136–2145). For Electricity with the default MLP, the overall range is $[0.119,0.475]$, while different seeds give $0.119,0.152$ at $H=96,L=336$ and $0.475,0.474$ at $H=720,L=720$. The correction is $\alpha\Delta S_i$, where $\Delta S_i=P_\theta(E(X_i))-S_{0,i}$ and $S_{0,i}=E(Y_{0,i})$. Predictor and scale are jointly optimized, so $\alpha$ measures how that model uses its learned displacements. Its value alone does not determine correction magnitude or forecast quality. **3. Shared scaling and the additional adaptive experiment (Q4)** We assign input-dependent modeling to the predictor and model-level scale control to $\alpha$, learning an aggregate compromise through the forecasting objective. This also supports a lightweight adapter that improves a frozen backbone without retraining it or running another coefficient network per window. We tested an adaptive alternative: $$\alpha_i=a_0+g_\phi(z_i),$$ where $z_i$ concatenates temporal means and standard deviations of $E(X_i)$, $S_{0,i}$, and $\Delta S_i$. The controller is a 16-hidden-unit MLP with unbounded output; $a_0$ is shared and learnable. All other paths are unchanged, with $\beta$ shared. Both variants were trained anew: 64 runs with the same frozen Base/AE, common-module initialization, losses, optimizer settings, and early-stopping rules. Shared FUSE starts at $\alpha=\beta=0$; adaptation starts at $a_0=\beta=0$ with a zero-initialized controller output layer. Both initially reproduce Base. We have added this comparison to Appendix E.4 (lines 1953–1986; Table 22). | Dataset | Shared: MSE Gain (%) | Adaptive: MSE Gain (%) | |---|---:|---:| | ETTm1 | **$2.59\pm0.23$** | $1.27\pm0.21$ | | Weather | **$0.82\pm0.53$** | $-2.52\pm0.05$ | | Solar | **$2.96\pm0.39$** | $2.17\pm0.43$ | | Electricity | **$3.83\pm0.12$** | $2.52\pm0.24$ | | Mean | **2.55** | 0.86 | Entries are mean $\pm$ sample SD across seeds after averaging horizon-level Gains within each seed. The final row weights datasets equally. Sharing achieves higher average Gain on every dataset for both seeds. All 32 adaptive models vary $\alpha_i$ across windows; 30 cross multiple geometric regimes. Adaptation helps locally, including Solar at $H=720$ (5.81% versus 1.37% with sharing), while sharing gives the higher overall average. On an RTX 4090 with FP16 and reused history encoding, complete single-window inference averages 3.981 ms for sharing versus 4.778 ms for adaptation, a 20.06% mean paired increase (lines 1987–1993). These results support shared scaling as a default combining higher average MSE Gain with lower inference cost.

Reply to W3 & Q3: Foundation models, variability, and full training cost

1. Refining frozen foundation models (W3 & Q3)

We selected TimesFM 2.5 and Chronos-2 from the model families you suggested to cover different forecasting approaches. TimesFM predicts variables separately in our setup, while Chronos-2 jointly processes variables within a window.

We evaluate ETTh1/ETTh2/ETTm1/ETTm2 with L=336L=336, H∈96,192,336,720H\in\\{96,192,336,720\\}, and two seeds 2021/2022, using the same splits, preprocessing, and evaluation windows. Each foundation model and the existing autoencoder remain frozen. Using target-dataset training labels, we independently train FUSE's predictor and refinement coefficients for each backbone: 32 fresh FUSE runs per backbone, with validation-MSE checkpoint selection.

Errors are averaged over horizons and seeds. Gain (%) is calculated relative to Base per setting, then averaged equally.

We have added these results to Section 4.2 (lines 321–323, 355–359; Table 3), with full results in Appendix D.2 (lines 1508–1511; Table 15).

BaseDatasetBase MSE / MAE+FUSE MSE / MAEMSE Gain (%)MAE Gain (%)
TimesFM 2.5ETTh10.4270 / 0.41220.4074 / 0.4124+4.613−0.005
TimesFM 2.5ETTh20.3795 / 0.39470.3559 / 0.3854+6.422+2.418
TimesFM 2.5ETTm10.3871 / 0.37840.3381 / 0.3690+12.618+2.430
TimesFM 2.5ETTm20.2975 / 0.32520.2548 / 0.3108+13.977+4.303
Chronos-2ETTh10.4363 / 0.41830.4103 / 0.4175+6.012+0.246
Chronos-2ETTh20.3625 / 0.38500.3483 / 0.3817+4.048+0.856
Chronos-2ETTm10.3953 / 0.37690.3364 / 0.3657+14.684+2.878
Chronos-2ETTm20.2953 / 0.32030.2576 / 0.3086+12.366+3.548

FUSE lowers MSE in every tested configuration. Average MSE/MAE gains are 9.419.41\\%/2.29\\% for TimesFM and 9.289.28\\%/1.88\\% for Chronos-2, demonstrating effective refinement of both frozen foundation models.

2. Three-seed results and refinement without retraining Base (W3)

For FUSE and TimeAlign, we report mean ±\pm SD across three seeds, averaging four horizons within each seed before computing SD.

DatasetFUSE MSETimeAlign MSEFUSE MAETimeAlign MAE
ETTm10.334 ± 0.0090.340 ± 0.0120.366 ± 0.0070.367 ± 0.008
ETTm20.245 ± 0.0050.246 ± 0.0060.304 ± 0.0090.306 ± 0.007
ETTh10.398 ± 0.0100.406 ± 0.0100.418 ± 0.0090.420 ± 0.007
ETTh20.337 ± 0.0120.338 ± 0.0140.382 ± 0.0150.382 ± 0.013
Weather0.217 ± 0.0080.219 ± 0.0090.253 ± 0.0080.248 ± 0.007
Electricity0.152 ± 0.0110.155 ± 0.0120.244 ± 0.0120.245 ± 0.009
Traffic0.389 ± 0.0120.389 ± 0.0130.255 ± 0.0130.249 ± 0.011
Solar0.186 ± 0.0100.195 ± 0.0120.217 ± 0.0090.220 ± 0.012

FUSE with the default MLP achieves competitive accuracy and lower mean MSE on most datasets. It also refines an existing forecaster without retraining its weights. TimeAlign (Hu et al., 2026) jointly optimizes its prediction, reconstruction, and alignment objectives while updating the forecasting backbone. FUSE instead learns refinement from history and forecast outputs while keeping the backbone frozen, without accessing its internal representations or changing its architecture. The original backbone experiments (lines 310–320) and the new foundation-model results support this reuse, with millisecond-level added inference cost in the conventional-backbone settings of Figure 5(b) (lines 524–527).

3. Full three-stage training cost (W3: Figure 5)

In addition to Figure 5's iteration and inference measurements, we timed a complete training run from scratch on ETTm2, L=720,H=720L=720,H=720, using the default MLP and one RTX 4090.

StageTime (s)
AE pretraining434.57
Base training17.58
FUSE refinement49.99
Total502.13 (8.37 min)

Timing includes data loading/preprocessing, training through actual early stopping, validation, checkpoint saving, and best-weight restoration; environment initialization and final testing are excluded. Batch sizes are 32/256/512; AE uses FP32 and Base/FUSE mixed precision.

The pointwise AE can be reused across horizons and backbone architectures when variables, preprocessing, and state dimensions are compatible, amortizing its pretraining cost. In this measured configuration, AE plus FUSE training costs 484.55 s when Base already exists; refinement alone costs 49.99 s when both Base and a compatible AE are available.

We hope our responses have addressed your concerns and would be grateful if you could reconsider your original rating. Thank you again for your thoughtful feedback and valuable time!

Thanks for the thorough response. The direct latent forecaster, TS-JEPA and the identity-encoder ablation were what I asked for. I think two results should go into the main text of the final version. The identity-encoder variant already gives +1.40% of the +2.55% gain. Predicted State Deviation seems to be above 80% of windows in most settings only for predictors trained with state losses, and 25-50% with forecasting supervision (Table 21). FUSE also seems to be on par with TimeAlign once SDs are reported. Even so, the gain is consistent across the base models and the method is cheap, so I have raised my score to 3.

Reviewer_zmHLOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how inaccurate predicted future representations can improve an existing time series forecast. Its central motivation is the observation of predicted state deviation, where a future state predicted from history can be farther from the encoded ground-truth future than the state obtained by encoding an existing forecast, making direct decoding unreliable. The key insight is that an inaccurate predicted state may still provide useful information on refinement direction. Accordingly, FUSE is proposed to encode a base forecast and uses a learned scalar to move its state along or against the displacement toward the predicted future state, the refined state is then decoded and combined with an adjusted reconstruction residual to preserve base-forecast information. Experiments on eight benchmarks demonstrate competitive accuracy and consistent average improvements across three frozen forecasting backbones.

Critical strengths
  1. The distinction between prediction quality and guidance value is interesting and intuitive. The geometric diagnostics and comparisons using identical predicted states provide useful evidence that an imperfect endpoint can still support forecast improvement.
  2. The proposed method FUSE can be combined with existing forecasting backbones without modifying or jointly retraining them, increaseing its practical usefulness, and the experiments across three backbones support its compatibility with different forecasting architectures.
  3. The ablations examine the contributions of state refinement, residual adjustment, and state supervision. The decomposition into mean, scale, and shape corrections helps explain how the framework improves forecasts.
Critical weaknesses
  1. Predicted State Deviation is empirically established, but its underlying cause remains unclear. The analysis in this paper does not sufficiently distinguish limitations of the state predictor from effects of the reconstruction-trained representation or differences in training objectives, which limits the explanatory value of this observation.
  2. The method uses one refinement scalar shared across all inputs. Although the displacement remains input-dependent, the paper does not sufficiently explain why a globally fixed degree of trust is appropriate for this heterogeneous guidance.
  3. The current comparisons do not fully isolate the benefit of future-state guidance from that of adding another trainable correction model.
Questions for authors
  1. What do the authors consider the main explanation for Predicted State Deviation? Is it primarily a limitation of learning future dynamics in the chosen representation, a consequence of reconstruction-based representation learning, or a difference between the objectives used to train the state predictor and base forecaster? A clear clarification for this would help establish when the proposed motivation should apply.
  2. I’m not familiar with this field, and one point I don’t fully understand is why predict an absolute future state from history and subsequently subtract the encoded base forecast, rather than predict a correction conditioned on both history and the base state? What advantage does the explicit future-state target provide beyond regularizing an additional correction model?
  3. Followed up the point in #2 in weaknesses, how should the sample-level geometric motivation be reconciled with a single shared refinement coefficient? A trained model cannot choose interpolation for one window and reversal for another through this coefficient, so it would help to clarify which aspects of the motivating geometry the implemented method is intended to capture.
  4. Predicted State Deviation varies across forecasting windows, but why does FUSE use a single shared coefficient α to control state refinement, rather than adapting α to each input? How does this design accommodate windows that may require different refinement strategies, such as interpolation or reversal?
2 comments

Thank you for your thoughtful review and constructive suggestions.

Reply to W1 & Q1: Explanation of PSD and scope of the motivation

Our explanation centers on a distinction: the encoder is trained to preserve information for reconstruction, which does not automatically make future states easy for a given predictor to estimate. With history XX, true future YY, frozen encoder EE, and base forecaster FF, the two prediction paths are S^=Pθ(E(X)),S0=E(F(X)),S⋆=E(Y).\widehat S=P_\theta(E(X)),\qquad S_0=E(F(X)),\qquad S^\star=E(Y). The representation defines the prediction coordinates and error geometry; predictor capacity and training objectives shape approximation errors along these different paths. PSD occurs when ∣S^−S⋆∣F>∣S0−S⋆∣F.\\|\widehat S-S^\star\\|_F>\\|S_0-S^\star\\|_F.

Beyond the independent state-only diagnostic in Appendix A.2 (lines 895–911), Appendix A.3 (Table 8; lines 1009–1015) reports average PSD window proportions (pwp_w) of 47.86%, 24.18%, and 36.14% with MLP, DLinear, and iTransformer bases after joint FUSE training. Thus, even with both state and forecasting supervision, the predicted endpoint can be less accurate than the encoded base.

Our motivation applies when an additional future-state estimate is available to refine an existing forecast, but its relative endpoint accuracy varies across windows. FUSE learns how to use this reference relative to the base; it requires no PSD test at inference and can use both better and worse endpoints as refinement references.

Reply to W3 & Q2: Future-state prediction versus direct conditional correction

FUSE first estimates a future-state reference from history, then learns how to use its displacement from the base state. This separates estimating the future from choosing the refined position. The alternative you suggest directly learns a correction conditioned on both history and the base:

\text{FUSE:}\quad &\widehat S=P_\theta(E(X)),\quad \Delta S=\widehat S-S_0,\quad S_{\mathrm{ref}}=S_0+\alpha\Delta S;\\\\ \text{Conditional correction:}\quad &C=G_\phi(E(X),S_0),\quad S_{\mathrm{ref}}=S_0+\alpha C. \end{aligned}$$ Here, $P_\theta$ reads encoded history alone, $G_\phi$ also reads $S_0$, and $\alpha$ is learned and shared across samples within each model. In full FUSE, state supervision gives $\widehat S$ the explicit target $S^\star$; forecasting supervision learns how to use the reference. Both objectives are optimized jointly, while the refined position can differ from $\widehat S$. **Matched correction-model experiment.** We cross the two designs with forecasting-only and forecasting-plus-state supervision. The conditional variants share architecture and paired initialization, and both retain FUSE's decoder, learned scale, and reconstruction-residual adjustment. Their state losses supervise the unscaled reference $S_0+C$, using the same MSE/cosine losses, weights, and target $S^\star$ as FUSE. Following Appendix E.1 (lines 1835–1841), we use the default MLP base, ETTh1/ETTh2/ETTm1/ETTm2, $L=336$, $H\in\\{96,720\\}$, and two seeds: 16 paired settings. Methods share frozen Base/AE, optimizer settings, training budgets, early stopping, and validation-MSE checkpoint selection; trainable parameter counts differ by less than 0.02%. FUSE initializes $\alpha=0$ with its default predictor initialization. The corrector initializes $\alpha=1$ and a zero output layer, giving $C=0$. Both set $\beta=0$ and start from the base forecast $Y_0=F(X)$. Gain is the per-setting percentage reduction in test MSE relative to Base, averaged equally. FUSE entries reuse Table 17 (lines 2108–2115); conditional-correction results are new. We have added this comparison to Section 4.4 (Table 6; lines 481–492, 510–519). | Supervision | Direct conditional correction | FUSE | |---|---:|---:| | Forecasting only | −1.127% | +1.147% | | Forecasting + state | +0.176% | **+2.118%** | **What the explicit reference contributes beyond an auxiliary loss.** FUSE performs better even without state supervision (+1.147% versus −1.127%), so this difference cannot come from adding a state loss. With identical state supervision, FUSE retains higher Gain (+2.118% versus +0.176%). Within FUSE, adding state supervision also improves Gain without increasing parameters. Together, these comparisons support explicitly learning a future-state reference and its use in refinement, beyond adding correction capacity or applying the same auxiliary losses to a conditional corrector.

Reply to W2, Q3 & Q4: Shared refinement and the adaptive-scale experiment

W2: Why a shared scale is appropriate for heterogeneous guidance

FUSE aims to improve existing forecasts at low added cost, without modifying or retraining the backbone. We assign input-dependent modeling to the state predictor and model-level scale control to a shared learnable α\alpha, avoiding an extra coefficient network per window.

For base forecast Y0,iY_{0,i} and frozen encoder EE, let S0,i=E(Y0,i)S_{0,i}=E(Y_{0,i}) and ΔSi=S^i−S0,i\Delta S_i=\widehat S_i-S_{0,i}. The shared scale is an aggregate compromise across windows with different ideal steps. For fixed states, Appendix A.1 (Eqs. 30–31; lines 864–892) expresses the shared geometric optimum for mean normalized squared target distance as a weighted mean of window-wise optima. Actual FUSE learns α\alpha jointly with the predictor and residual coefficient β\beta through the final forecasting loss.

To test whether input adaptation improves this compromise, we implemented αi=a0+gϕ(zi),\alpha_i=a_0+g_\phi(z_i), where a0a_0 is shared and learnable; ziz_i concatenates temporal means and standard deviations of encoded history E(Xi)E(X_i), base state S0,iS_{0,i}, and displacement ΔSi\Delta S_i. The controller gϕg_\phi is an MLP with 16 hidden units and unbounded output, allowing interpolation, extrapolation, or reversal. Other prediction, decoding, and residual paths are unchanged; β\beta stays shared.

We used the default MLP base on ETTm1, Weather, Solar, and Electricity, with L=336L=336, H∈96,192,336,720H\in\\{96,192,336,720\\}, and seeds 2021/2022: 32 pairs and 64 new refinement training runs. Pairs share frozen Base/AE, common-module initialization, losses, optimizer settings, training budgets, early stopping, and validation-MSE checkpoint selection. Shared FUSE starts at α=0\alpha=0; adaptation starts at a0=0a_0=0 with a zero-initialized controller output layer. Both set β=0\beta=0, initially reproduce Base, and undergo joint training.

We have added this comparison to Appendix E.4 (lines 1953–1986; Table 22).

DatasetShared: MSE Gain (%)Adaptive: MSE Gain (%)
ETTm12.59±0.232.59\pm0.231.27±0.211.27\pm0.21
Weather0.82±0.530.82\pm0.53−2.52±0.05-2.52\pm0.05
Solar2.96±0.392.96\pm0.392.17±0.432.17\pm0.43
Electricity3.83±0.123.83\pm0.122.52±0.242.52\pm0.24
Mean2.550.86

Gain is the per-setting test MSE reduction relative to Base, averaged over horizons within each seed, then reported as mean ±\pm sample SD across seeds. The final row weights datasets equally. Sharing achieves higher average Gain (2.55% versus 0.86%). Each dataset favors sharing for both seeds after horizon averaging; averages grouped by horizon also favor sharing.

Q3: Which aspects of the sample-level geometry FUSE implements

The geometry distinguishes endpoint accuracy from the usefulness of its displacement. Appendix A.1's window-wise optimum aigeoma_i^{\mathrm{geom}} uses the true future state Si⋆=E(Yi)S_i^\star=E(Y_i) for retrospective diagnosis: even an endpoint farther from the target than Base may yield a closer state by scaling or reversing its displacement. FUSE uses this idea at inference through a learned sign and scale: Sref,i=S0,i+αΔSi.S_{\mathrm{ref},i}=S_{0,i}+\alpha\Delta S_i. Endpoint use fixes α=1\alpha=1; learning it separates the state estimate from the position adopted. As explained in W2, each trained model uses a common interpolation, extrapolation, or reversal rule, while the correction αΔSi\alpha\Delta S_i varies with the input.

Table 18 (lines 2118–2132) fixes the same predicted states and β=0\beta=0 in both paths. With the default MLP, average MSE Gain rises from −2.733% for endpoint use to +1.842% for the learned scale, supporting the value of learning how to use the same reference.

Q4: Input-adaptive strategies and their accuracy–cost trade-off

All 32 adaptive models varied αi\alpha_i across test windows; 30 crossed multiple geometric regimes within a model. Adaptation helped in some settings: on Solar, H=720H=720, average MSE Gain was 1.37% with sharing versus 5.81% with adaptation. The shared scale nevertheless achieved the better overall average in W2, supporting a model-level compromise across heterogeneous windows.

On the same RTX 4090, using FP16 and reusing history encoding, mean latency for the complete single-window forecasting path was 3.981 ms with sharing and 4.778 ms with adaptation, a mean paired increase of 20.06%. The timing analysis is included in Appendix E.4 (lines 1987–1993). These results support shared scaling as the default for its combination of average test MSE and inference cost.

We hope our responses have addressed your concerns and would be grateful if you could reconsider your original rating. Thank you again for your thoughtful feedback and valuable time!

Comments on the paper

2 comments

We sincerely thank all reviewers for their careful evaluations, insightful questions, and constructive feedback. These comments motivated valuable additional experiments and analyses that helped clarify our design choices and significantly strengthened the empirical support for our work. We have revised the manuscript accordingly, and our point-by-point responses detail these new experiments while indicating the corresponding sections, tables, and line numbers in the revised paper. We greatly appreciate the time and expertise you have devoted to improving our work.

AuthorsOpenReview ↗

Summary of Revisions and Rebuttal

We thank the Area Chair and all reviewers for their time and constructive feedback. We have uploaded a revised manuscript incorporating the main additional experiments and clarifications.

1. Main finding and contribution

A predicted future state can be less accurate than the state obtained by encoding an existing forecast, yet still point in a useful direction. We found that even when the predicted state lies farther from the target state, moving partway toward it can bring the base state closer to the target. FUSE learns how far to move along or against this displacement to refine the forecast while keeping the backbone frozen.

2. Questions raised and evidence added

  • Does the finding depend on a particular state space or predictor? We observed the same pattern across six state spaces and with a larger predictor. Holding the predicted states fixed, using the learned step gave lower test MSE than taking the full step in all 192 paired settings (Appendix A.4).
  • What does an explicit future-state reference add? With matched parameter budgets and identical state supervision, FUSE reduced test MSE by 2.12% relative to the base forecast, compared with 0.18% for direct conditional correction (Section 4.4, Table 6). On each of the four datasets, FUSE also achieved lower average MSE and MAE than both a separately trained direct latent forecaster (Appendix E.3) and TS-JEPA (response to RtQY).
  • How much does the refinement design matter? Across four datasets, full FUSE reduced test MSE by 2.55% on average. The corresponding reductions were 1.40% for refinement in observation space and 0.86% for a step size adapted to each input. The shared step also had lower inference latency than the adaptive version (Appendices E.4–E.5).
  • Does refinement extend to foundation models? Training only the FUSE module lowered horizon-averaged MSE on each of the four ETT datasets for both frozen TimesFM 2.5 and frozen Chronos-2 (Section 4.2, Table 3). We also reported variability across seeds and measured the full three-stage training cost in our response to RtQY.

3. Reviewer discussion

  • Reviewer RtQY. We appreciate RtQY’s positive follow-up and their decision to raise their rating from 2 to 3. They highlighted FUSE’s consistent gains across backbones and its low additional cost.
  • Reviewer zmHL. In recommending weak acceptance (3), zmHL highlighted the distinction between a prediction’s accuracy and its value as guidance, and asked whether a direct correction model or a separate step size for each input would work as well. Our matched comparisons favored the explicit future-state reference and showed that a shared step gave larger average gains at lower inference cost than an input-adaptive step (Section 4.4 and Appendix E.4).
  • Reviewer TWR8. TWR8 recognized the paper’s motivation and geometric analysis, rating both presentation and informativeness 3 with an overall rating of 2. To address their questions, we tested the finding across six state spaces, compared FUSE with input-adaptive and observation-space refinement, and corrected Appendix D.1 to specify that lookback selection used validation MSE (Appendices A.4, E.4, and E.5).

As of this writing, zmHL and TWR8 have not posted follow-ups. We hope this summary and the revised manuscript assist the Area Chair in the final evaluation of our submission.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.