acceptodds
Under review as a conference paper at ICLR 2027

A Multivariate Financial Time Series Foundation Model with Test-Time Learning Relational Attention

Abstract

Multivariate forecasting can improve financial forecast accuracy by drawing on related time series that provide predictive information beyond the target series’ own history. However, financial time series are noisy, cross-series predictive relationships are non-stationary, and datasets contain different combinations of heterogeneous variables. We introduce FTFM, a financial time-series foundation model that couples causal temporal attention with Test-Time Learning Relational Attention. Causal temporal attention encodes each series’ history. Relational attention then combines a slow path that weights these features by learned compatibility with a fast path guided by a directed predictive relation memory. Feature compatibility can remain high even as predictive relationships weaken. The relation memory adds direct feedback on how well temporal features across series predict subsequent observed features, using the resulting prediction errors to revise the relationships guiding cross-series attention during inference. Furthermore, similar normalized trajectories can represent different meanings and forecasting implications, we introduce semantic embeddings that preserve distinctions between series such as prices, returns, and volatility after normalization, while shared projections and sampling-interval conditioning support different series combinations and temporal resolutions. In empirical experiments across eight financial, economic, and synthetic domains, FTFM achieves the lowest forecasting errors among the evaluated models on the majority of benchmark tasks, both before and after LoRA adaptation.

Primary area:
learning on time series and dynamical systems
Keywords:
Financial time series forecast, Time series forecast, Time-series foundation models

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_HYeMOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper introduces FTFM, a financial time-series foundation model equipped with Test-Time Learning Relational Attention. The proposed method addresses the challenges of financial forecasting by dynamically adapting cross-series relationships during inference through a predictive relation memory updated from observed contexts. FTFM combines causal temporal attention, relational attention, semantic conditioning, and shared representations to handle heterogeneous financial time series across different domains and sampling frequencies.

Critical strengths

s1: The proposed model achieves strong forecasting performance across multiple financial, economic, and synthetic benchmarks, often outperforming existing baselines and achieving the best results on a majority of evaluated tasks.

s2: The paper provides evidence that LoRA adaptation is an effective strategy for adapting pretrained financial time-series foundation models to downstream forecasting tasks.

Critical weaknesses

w1: The paper is relatively difficult to read, and the main technical contributions are not presented clearly enough. Although the paper introduces several components, including relational attention, test-time learning, semantic conditioning, and relation memory, the novelty and necessity of each component are not sufficiently highlighted.

w2: The related work section is somewhat disorganized and does not clearly explain how FTFM differs from existing time-series foundation models, financial forecasting models, or adaptive attention mechanisms. A more structured comparison with previous approaches would help clarify the technical novelty and contribution of this work.

w3: Although the paper provides ablation studies, the analysis is not sufficient to fully understand the contribution of each proposed component in the foundation model setting.

w4: The paper demonstrates that the proposed relational attention improves forecasting performance, but provides limited insight into why dynamically updating relations during inference is beneficial.

Questions for authors

Please refer to the weaknesses.

Reviewer_xyU7OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces FTFM, a financial time-series foundation model that combines causal temporal attention with Test-Time Learning Relational Attention. Its central component is a directed relation memory updated through normalized prediction-error corrections on transitions between observed temporal representations. The resulting relation strengths modulate cross-series attention scores and value aggregation while pretrained parameters remain fixed at inference. Semantic embeddings and sampling-interval conditioning support heterogeneous financial variables and panel compositions. The 27.42M-parameter model is pretrained on approximately 6.4 billion scalar observations and performs best across eight financial, economic, and synthetic domains.

Critical strengths

1. Relevant motivation and coherent design

Changing predictive relationships and heterogeneous variable semantics motivate adapting cross-series interactions while preserving series meaning. FTFM addresses these issues through context-local predictive learning and semantic conditioning, with shared parameters accommodating different panel compositions.

2. Competitive performance across heterogeneous forecasting settings

FTFM achieves the lowest MASE in 19/26 reported settings before adaptation and 20/26 afterward, against the respective comparator sets. Its consistent advantages across Crypto, Equities, and Futures after competing models receive LoRA adaptation support the usefulness of the complete model.

Critical weaknesses

1. The importance of the specific methodological contribution remains insufficiently established

Error-driven fast weights and self-supervised test-time state updates are established ideas, making the series-pair memory and its integration into attention the main architectural increment. However, the motivating mismatch between feature compatibility and predictive usefulness is not directly tested, and the benefit of converting latent-transition coefficients into attention strengths remains unclear. Controlled dependency changes and a matched adaptive-memory comparator would help establish whether this design specifically improves adaptation to changing predictive relationships.

2. Experimental ambiguities and incomplete reporting weaken confidence in the conclusions

The correction ablation is not precisely defined, and its significance tests treat horizons sharing underlying series as independent observations. The comparison also omits five benchmark settings, while pretraining coverage and overlap-exclusion rules are insufficiently documented. These issues limit confidence in mechanism-specific attribution, statistical significance, and the breadth of held-out generalization.

Questions for authors
  1. Clarification — ablation definition: What exact recurrence is used in the “no prediction-error correction” ablation? Does it retain an adaptive relation memory or remove the entire correction term?

  2. Validation — robustness of the ablation evidence: Do the ablation gains persist on multivariate panels when statistical testing accounts for dependence across horizons? Separating these results matters because the relational contribution is disabled for singleton panels.

  3. Clarification — pretraining–evaluation separation: What pretraining date ranges and overlap-exclusion rules substantiate separation from evaluation data, including equivalent instruments across sources and macroeconomic vintages?

  4. Validation — contribution of the proposed mechanism: Does FTFM adapt more effectively than ordinary attention and a generic delta-rule memory when predictive dependencies change while marginal series behavior remains similar? A comparison under matched pretraining data and model budgets would help isolate the contribution of the relation-specific design.

Reviewer_cibqOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

he paper proposes FTFM, a 27.42M-parameter forecasting foundation model for financial panels, where relations between series drift over time and datasets mix variable types and sampling intervals. Its core idea is test-time learning relational attention. Inside each context window, a series-by-series memory is re-fitted from the model's own one-step prediction errors (a delta rule), and this memory then guides cross-series attention.

Critical strengths
  • Consistent lead on the three OHLCV domains with a small model. FTFM has the lowest MASE and sMAPE at every horizon on Crypto, Equities and Futures, before and after LoRA. Both metrics agree, and the model is smaller than four of the five pretrained competitors.

  • Experimental details are fully specified such that it is possible to repeat/re-implement the method.

Critical weaknesses
    1. It is not clear how much of the evidence involves relations between different instruments, the setting that motivates the method. Of the 26 rows, 17 are either single series (where the relational path is off) or one asset's OHLCV fields (where it only relates fields of that asset). Only Crypto and FTS mixed can hold several instruments, and they also contain single-instrument panels. Does FTFM still lead, and does the relation memory help, on panels with several instruments?
  • The evidence that the relation memory drives the gains is not strong enough. The ablation gaps on single-series panels, where the mechanism is off, are as large as on panels where it is active, and the largest gain (FTS mixed) appears in MASE but hardly in sMAPE. Would the gains remain if the memory were switched off inside the same trained model?

  • It is not clear from the main text what the method does at a high level. The equations are complete, but the main text explains the method mainly through new terms and implementation details, without a plain summary or an example. A reader who skips the appendix will find the high-level idea hard to see.

Questions for authors
  • The method is motivated by relations between related assets, but only Crypto and FTS mixed contain panels with several instruments, and both also contain single-instrument panels. How many panels in each of these domains hold more than one instrument, and can you report Tables 1 and 4 for those panels alone, comparing full FTFM with the ordinary-attention ablation?

  • Could you please discuss "Learning Fast and Slow for Online Time Series Forecasting" (ICLR 2023), a related fast/slow online-adaptation approach.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.