acceptodds
Under review as a conference paper at ICLR 2027

Latent Inference-Time Guidance of Time Series Foundation Models

Abstract

Time Series Foundation Models (TSFMs) currently provide state-of-the-art results in forecasting tasks. They are available out-of-the-box and rely on in-context learning to make their predictions, which makes the quality of their performance highly sensitive to the user-selected lookback, covariates, horizon and training data distributions. In practise, the quality of the forecasts are variable but complementary, which highlights the need for a principled ensembling approach, rather than selecting the best context. This paper introduces Latent Inference-Time Guidance for TSFMs, which adaptively combines a pool of TSFM forecasts through a time-dependent latent space with independent components. The framework comes equipped with identifiability and reconstruction guarantees, whilst maintaining the off-the-shelf aspect of foundation models. We provide experiments on datasets at various frequencies and from multiple domains: these show that the approach is competitive with traditional ensembling approaches.

Primary area:
foundation or frontier models, including LLMs
Keywords:
Time Series Foundation Models, Time Series Forecasting, Generative Modelling, Mixture of Experts

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_4nz6OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents Latent Inference-Time Guidance for Time Series Foundation Models (LITiG-TSFM), a probabilistic framework for combining and correcting predictions from frozen, pre-trained Time Series Foundation Models (TSFMs) without retraining them. The core idea is to frame TSFM ensemble forecasts as observations generated by a structured state-space model governed by time-dependent latent variables with independent components, leveraging concepts from nonlinear Independent Component Analysis (ICA).

Critical strengths

The paper introduces a creative bridge between nonlinear ICA (they refred to Δ\Delta-SNICA ) and inference-time ensembling of time-series foundation models. Th e core comtribution is to frame forecast combination and correction via independent latent components (ztz_t) without modifying the underlying frozen foundation models which represent a novel direction for zero-shot forecasting.

Critical weaknesses

I noticed a confusion between theoretical bounds and formulation presented in section 3, they defined a backward-factorised variational posterior qφ(z1:T∣y1:T,x1:T)=qφ,T(zT∣y1:T,x1:T)∏t=1T−1qφ,t∣t+1(zt∣zt+1,y1:t,x1:t)q_\varphi(z_{1:T} \mid y_{1:T}, x_{1:T}) = q_{\varphi, T}(z_T \mid y_{1:T}, x_{1:T}) \prod_{t=1}^{T-1} q_{\varphi, t \mid t+1}(z_t \mid z_{t+1}, y_{1:t}, x_{1:t}) and evaluate backward variational smoothing kernels (we read in Assumption B3 and Proposition 3.2). However, Appendix A.1 derives the ELBO but when I look at Appendix A.2/Section 5 (in Discussion) they describe implementation using a FORWARD factorisation qφ(z1∣x1,y1)∏t=2Tqφ(zt∣x1:t−1,y1:t−1,zt−1)q_\varphi(z_1 \mid x_1, y_1) \prod_{t=2}^T q_\varphi(z_t \mid x_{1:t-1}, y_{1:t-1}, z_{t-1}). They admit: "Whilst we use forward factorisations of the time series when implementing LITiG-TSFM, the reconstruction guarantee is given in a backward factorisation". I think there is a misconception of teh framework or there is something unclear is not disclosed here, because the optimization objective maximized during training does not match the backward variational family used in Proposition 3.2, the theoretical reconstruction guarantee does not apply to the actual proposed algorithm.

Also I think there is typos: The sum over transition distributions ∑t=2Tlog⁡pθ(zt∣zt−1)\sum_{t=2}^T \log p_\theta(z_t \mid z_{t-1}) has T−1T-1 terms, but line 790 writes Tlog⁡(2πρ2)T \log(2\pi \rho^2) instead of (T−1)log⁡(2πρ2)(T-1)\log(2\pi \rho^2). In addition, the final explicit ELBO formula duplicates and cancels inconsistent terms with mismatched indexing bounds (t=1t=1 vs t=2t=2).

Questions for authors

On the mismatch of theory vs implementation, could you explain why the forward-factorized variational posterior was implemented in practice if the stability guarantee (Proposition 3.2) requires a backward-factorized variational family? Can you provide proof that Proposition 3.2 holds under the forward variational posterior used in your ELBO code? For the ELBO Derivation Correction defined In Appendix A.1 (lines 810-834), how are the sign errors in the quadratic log-Gaussian variational density reconciled, and what is the exact closed-form expression implemented in your loss computation?

1 comment

On the mismatch of theory vs implementation, could you explain why the forward-factorized variational posterior was implemented in practice if the stability guarantee (Proposition 3.2) requires a backward-factorized variational family? Can you provide proof that Proposition 3.2 holds under the forward variational posterior used in your ELBO code?

We thank the Reviewer for pointing out the discrepancy between the theoretical and implemented factorisations. We chose to write Proposition 3.2 with a backward-factorized variational family to extend some recent theoretical results to our setting. The forward factorisation was chosen for simplicity in our step-wise forecasting setting. Thanks to your comment, we propose to extend the theoretical analysis of our work and to provide the conditions under which the stability result can be extended to the forward-factorized variational family. This can be done using similar forgetting properties on the forward Markov chains. We will therefore adjust the scope of Proposition 3.2 to cover both forward and backward settings.

For the ELBO Derivation Correction defined In Appendix A.1 (lines 810-834), how are the sign errors in the quadratic log-Gaussian variational density reconciled, and what is the exact closed-form expression implemented in your loss computation?

We thank the Reviewer for checking the derivation of the ELBO. The typos in the manuscript are not reflected in the implemented version, and will be corrected in the implementation for future versions of the manuscript.

Reviewer_XRgLOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes LITiG-TSFM, a latent inference-time guidance framework for combining forecasts from frozen time-series foundation models (TSFMs). The method introduces a nonlinear state-space model with temporally evolving independent latent components to adaptively aggregate and correct forecasts without fine-tuning the underlying foundation models.

The authors provide theoretical results on the identifiability of the denoised guidance signal and reconstruction stability under specific assumptions. Empirically, the approach is evaluated on four datasets using forecasts generated from different configurations of TabICLv2. The results demonstrate competitive forecasting performance against existing works.

Critical strengths
  • The paper addresses a practical challenge: the performance of TSFMs varies across forecasting contexts, thus motivating adaptive combinations of their prediction.
  • The theoretical analysis connects the framework to existing identifiability and reconstruction results for structured nonlinear ICA and state-space models
Critical weaknesses
  • My primary concern is that the motivation for incorporating nonlinear ICA remains insufficiently justified relative to the paper's main objective of improving forecasting accuracy. Besides accuracy, it is unclear why identifiability is necessary or beneficial for forecast aggregation. Moreover, a conventional nonlinear state-space model could potentially provide similar adaptive forecast-combination capabilities without imposing independent-component constraints.
  • There is insufficient reference relative to existing model aggregation literature. The paper claims that existing aggregation methods are typically restricted to linear schemes, while there could be relevant works on nonlinear forecast combination, dynamic model averaging, and learned ensemble weighting. Relevant examples include FFORMA [1] and a comprehensive review of forecasting ensembles (includes nonlinear combination methods, time-varying aggregation weights, and cross-learning approaches) [2].

[1] Montero-Manso, P., Athanasopoulos, G., Hyndman, R. J., & Talagala, T. S. (2020). FFORMA: Feature-based forecast model averaging [2] Wang, X., Hyndman, R. J., Li, F., & Kang, Y. (2023). Forecast combinations: An over 50-year review. International Journal of Forecasting

Questions for authors
  • What is the specific advantage of nonlinear ICA for forecasting? Can the authors explain what an identifiable independent-component representation enables that a conventional nonlinear state-space model cannot provide?
  • Does enforcing independent latent dynamics consistently improve forecasting accuracy, robustness, or calibration? Can the authors show, for instance, the advantages on OOD forecasting?
  • How does the proposed method compare with existing nonlinear aggregation approaches? Would a simple neural gating network achieve comparable performance when given the same expert forecasts?
1 comment

What is the specific advantage of nonlinear ICA for forecasting? Can the authors explain what an identifiable independent-component representation enables that a conventional nonlinear state-space model cannot provide?

We thank the Reviewer for raising the interest of nonlinear ICA for forecasting. ICA can be used to identify patterns and correlations in the data, where we can assume independent, component-wise latent sources of behaviour. This can be extended to the nonlinear case, which allows to capture more complex behaviours in the data.

The main benefit of the proposed framework is not simply to introduce nonlinear latent dynamics but to impose a structured and potentially identifiable latent representation. This follows recent works on identifiable ICA models where, under appropriate assumptions, the structure of the model provides identifiability guarantees.

Within this framework, we propose an algorithm to adapt the forecasts of different foundation models over time with statistical foundation for the latent representation.

Does enforcing independent latent dynamics consistently improve forecasting accuracy, robustness, or calibration? Can the authors show, for instance, the advantages on OOD forecasting? How does the proposed method compare with existing nonlinear aggregation approaches? Would a simple neural gating network achieve comparable performance when given the same expert forecasts?

We agree with the Reviewer that comparing LITiG-TSFM to a vanilla nonlinear aggregation approach could be interesting. Note that since our model comes equipped with identifiability and reconstruction guarantees, and that a nonlinear aggregation approach such as a neural gating network would not present such characteristics. We hope that the proposed experiments involving experts with different level of performance provides an empirical illustration of the robustness of our approach.

We propose a qualitative experiment to explore the expressivity of the latent space learned, by using a t-SNE projection to analyse the consistency of what has been learned.

In addition, we are currently conducting additional simulation experiments to compare our approach with a linear Kalman filter for a linear state space model without independent latent components. These experiments will be included in the revised manuscript.

Reviewer_WH2bOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes LITiG-TSFM, a learned latent state-space model that dynamically combines forecasts from frozen time series foundation models. It uses variational inference and shows results against standard ensemble method across four datasets.

Critical strengths

The approach is interesting.

Critical weaknesses
  • There is only one model being used, the premise is using different models
  • All TSFMs: this is misleadingly named. It is not a collection of different TSFMs
  • Evaluation on 4 datasets is not enough
  • TabICLv2 is not SOTA TSFM, it's a tabular model
Questions for authors

See weaknesses.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.