acceptodds
Under review as a conference paper at ICLR 2027

Pythia: Toward Foundation World Models for Multimodal Time Series

Abstract

Time-series foundation models offer a unified approach to forecasting across heterogeneous domains. Textual context and auxiliary observations provide complementary information about temporal dynamics, yet reusable multimodal predictive representations remain underexplored. We introduce Pythia, a foundation world model that learns context-conditioned latent dynamics across datasets through a joint-embedding predictive architecture. A stop-gradient numerical reference guides contextual corrections to predicted future states. A separate probabilistic decoder then adapts to the frozen predictive representation and observed history, decoupling world-model pretraining from observation-space forecasting. On MUSE, Pythia-Tiny's normalized mean absolute scaled error (MASE) and weighted sum quantile loss (WSQL) are 0.6879 and 0.4269, reducing errors by 6.26% and 5.00% relative to the strongest model evaluated in the published MUSE leaderboard. Through a series of controlled experiments, we investigate how to design a time-series world model through shared pretraining and how joint-embedding predictive learning can incorporate multimodal information. The results support separating predictive representation learning from probabilistic readout and show complementary contributions from entity descriptions, events, and covariates.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series, Time-Series Foundation Models, World Models, Multimodal Time-Series

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ye4SOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

a JEPA-style latent predictor on top of a frozen Chronos-2 encoder, adds text, events and covariates as a residual correction in latent space, and then adapts the Chronos-2 decoder on the frozen representation. Pretraining is on public data plus internal company data. Evaluated on MUSE, where it beats the published leaderboard, with a lot of ablations on a small model.

Critical strengths
  • many matched controls, and the negative group effects are clearly presented.
  • comparison between frozen and jointly trained readout in Figure 3b is clear and well-presented.
  • ablation studies successfully shows the reference loss.
  • freeze and anchor grid will be helpful for anyone looking to adapt Chronos-2.
  • Checkpoints were selected based on a pretraining holdout set, rather than on MUSE.
Critical weaknesses
  • 'zero-shot,' but you pretrain on datasets like ACL18, TimeCAP, and Time-MMD up to the test cutoff (Apx B). The baselines in tab 1 did not have access to these series.
  • Chronos-2 decoder adaptation control achieves less compared to the published results. This seems that about a lot of improvement comes from in-domain training.
  • are the internal company data and the MUSE Industrial windows from the same source?
  • this only used seed 42, and the effects range is small. The capacity grid is not monotonic, which looks like noise to me.
  • the 6.26% is the best of four model sizes on the test set
  • why do you call this a 'world model'? There are no actions or rollouts involved.
  • JEPA for time series exists already (you cite LaT-PFN etc.), the delta is the residual context part
  • GPU hours?
Questions for authors

see above

Reviewer_uPZhOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Pythia, a JEPA-based framework for multimodal time-series forecasting. The method learns predictive latent representations from numerical histories, using textual descriptions, events, and covariates to refine predicted states. Training separates latent prediction from probabilistic forecasting: the first stage learns context-conditioned representations with additional alignment and reference losses, while the second stage adapts a probabilistic decode with the predictive representatino frozen. Experiments on MUSE show improvements over existing forecasting baselines, with additional ablations investigating the contributions of multimodal inputs, training objectives, and adaptation strategies.

Critical strengths
  • Strong empirical performance. The method achieves leading aggregate performance on MUSE compared with the published baselines, demonstrating the practical potential of JEPA-based multimodal time-series forecasting.
  • Extensive empirical analysis. The paper includes numerous ablations investigating training strategies, multimodal inputs, model capacity, and adaptation policies. These experiments provide useful insights into the design of time-series forecasting models.
  • Practical integration of multimodal information. The framework provides a unified way to incorporate textual descriptions, events, and numerical covariates into time-series prediction. The modular design is appealing, as it allows contextual information to refine numerical predictions while retaining a pretrained forecasting backbone.
Critical weaknesses
  • Limited novelty. The approach largely builds on established JEPA principles and pretrained forecasting architectures. Additional components, such as contextual residual correction and reference-guided losses, appear incremental, with limited conceptual justification.

  • Unclear justification for design choices. The framework introduces several additional objectives and architectural choices, including reference guidance, magnitude alignment, and selective freezing. While some ablations are provided, the motivations behind these choices and the selection of loss weights remain insufficiently justified.

  • Unclear attribution of performance gains. Pythia combines JEPA-based pretraining with several additional objectives and architectural modifications. Although ablations are provided, it remains unclear which components are primarily responsible for the improvements over the pretrained forecasting backbone.

Questions for authors

Q1. What are the key contributions beyond applying JEPA to multimodal time-series forecasting? In particular, what motivates the proposed contextual correction and reference-guidance mechanisms over simpler alternatives, such as direct multimodal fusion or standard JEPA training without the additional reference objective?

Q2. How sensitive are the results to the additional loss terms and their weights? Given the multiple architectural and training choices, which components are most responsible for the improvements over the pretrained forecasting backbone?

Q3. How dependent is Pythia's performance on Chronos-2 initialization? Would the JEPA-based pretraining framework remain effective with a different numerical backbone or randomly initialized encoder?

Reviewer_NJq7OpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes Pythia, a latent predictive world model for multimodal time-series forecasting. The main idea is to separate predictive representation learning from probabilistic forecasting, rather than directly optimizing an end-to-end forecasting objective.

Critical strengths
  1. The authors propose a two-stage training framework and compare several training strategies under the same Small-model configuration.
  2. The paper provides useful ablation studies on different context sources, including entity descriptions, events, and covariates, as well as the proposed numerical-reference loss.
Critical weaknesses
  1. The methodological novelty is somewhat limited. Most components of Pythia build on existing techniques. The numerical encoder and probabilistic decoder are initialized from Chronos-2, semantic representations come from Qwen3-VL-Embedding-8B, and contextual correction uses standard cross-attention and residual connections. The separation of latent prediction and decoding is also closely related to existing JEPA-based methods.

  2. The comparison with existing models is not fully fair. The paper reports 6.26% MASE and 5.00% WSQL improvements over Chronos-2. However, Pythia uses Chronos-2 initialization, additional pretraining data, two stages of training, and a large pretrained semantic encoder. It is therefore unclear how much of the improvement comes from the proposed method rather than additional data and computation. I would suggest adding:

    • A Chronos-2 baseline further trained on the same data with a comparable compute budget.
    • A simple Chronos-2 + multimodal fusion baseline trained directly with a forecasting loss.
    • Comparisons with other multimodal forecasting methods under the same training data conditions.
  3. The improvements from multimodal context are not consistent across tasks. In Section 4.4, compared with the numerical-only two-stage baseline, Pythia-Small improves MASE in 17 out of 29 groups and WSQL in 18 out of 29 groups, while only 15 groups improve on both metrics. This suggests that multimodal context does not consistently help forecasting. It would be useful to further analyze when contextual information helps or hurts performance.

  4. The reproducibility of the results is limited by the use of private training data. The pretraining corpus contains approximately 18 million eligible windows per stage, including proprietary Industrial data that account for around 26–27% of the corpus. Since these data are not publicly available, it is difficult to reproduce the results or understand their contribution to the reported improvements. A public-data-only experiment would be helpful.

  5. The experiments use only one random seed, and the authors do not report statistical significance. Given that some improvements are relatively small, results across multiple random seeds or confidence intervals would make the conclusions more convincing.

Questions for authors

Please see weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.