- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes Chrnicle, the first model to learn the relationship between text and time series and train the model from scratch. The authors conduct extensive experiments by comparing it with dedicated foundation models over 19 NLU tasks on 24 datasets.
Critical strengths
-
In this paper, the authors proposed a unified framework called Chrnicle, whcile is the first model to learn the relationship between text and time series and train the model from scratch.
-
The authors conduct extensive experiments and achieves competitive performance across over various tasks.
-
The authors compare the proposed method with strong unimodal foundation models.
Critical weaknesses
-
Although section 5.3 shows the effectiveness of the cross-modal alignment stage by comparing two stages checkpoint, this experiment can only partially demonstrate whether the proposed method effectively learns relationship between text and time series. The paper lacks the ablation study to demonstrate the effectiveness of the proposed method over the single modality variant, such as text-only and time series only model using the same pretrained Chronicle checkpoint. This aims to enhance the motivation of this paper that "these models are also evaluated almost exclusively against other multimodal baselines, not against the strongest unimodal foundation models in either domain, leaving open whether joint training is needed at all".
-
In section 5.2.1, the authors mention that supervised DL baselines are trained per-dataset. My concern is that Chronicle may benefits from additional pretraining data. Directly comparing Chronicle with supervised DL baselines trained per-dataset is not fair. So, the experimental results cannot demonstrate the overall superiority of Chronicle for time-series classification. Rather, it serves the evidence as the effectiveness of the proposed method with frozen representations. The authors should frame it carefully.
-
In Fgiure 3 and table 7, the results show that Stage 2 does not consistently improve multimodal performance across different setting. The paper lacks a detailed analysis for the inconsistency.
-
In Figure 2, the number of the parameters for many baseline models are not reported, making it hard to compare the effectiveness of the proposed method and other baseline methods fairly.
Questions for authors
Q1: Why does TimeMMD LoRA adaption achieves worse performance in Figure 3 and Table 7? Can you provide some analysis for the inconcistency of the improvement across different settings?
6 comments
W2. Concerns regarding Chronicle benefitting from additional pretraining data.
Section 5.2.1 also compares Chronicle with pretrained time series foundation models that were trained on far more time series tokens than Chronicle. This like-for-like comparison is in Table 3: three pretrained TSFMs (Chronos-2, Moirai-2, TimesFM) that also benefit from large-scale pretraining, evaluated with the identical frozen probe. They spend all of their pretraining on time series, whereas Chronicle spends about 8% of its compute on it. With 5 seeds on the 24 UCR/UEA datasets:
| Frozen probe, UCR/UEA | Chronicle wins / losses (datasets) | Wilcoxon |
|---|---|---|
| vs. Chronos-2 | 24 / 0 | p < 0.001 |
| vs. TimesFM | 22 / 2 | p < 0.001 |
| vs. Moirai-2 | 16 / 8 | p = 0.08 (n.s.) |
Chronicle has the best mean accuracy (0.753) and the best average rank (3.67 among the 10 baselines and both Chronicle checkpoints). We will make this clearer within the paper, thank you for your feedback.
W4. Figure 2 does not report parameter counts for many baselines. We will update Figure 2 to list every model's parameter count from its public model card (e.g. TimesFM-2.5 200M, Toto-Base 151M, Moirai-Large 311M, TiRex 35M, YingLong 300M) for clarity. Chronicle has 323.8M parameters, of which 189.6M process time series. The 134M text embedding table is never used for series inputs.
W3. Stage 2 does not improve performance consistently across settings, and the paper lacks an analysis.
The results are consistent once they are grouped by what each protocol measures. Some evaluations read the frozen representation, through a small head or with no training. Others ask the model to generate text or forecasts zero-shot. Stage 2 replaces a small share of unimodal generation training with alignment data, so the expected outcome is a gain in the representation and a small cost in zero-shot generation. That is what we observe.
| What is measured | Evaluation | Stage 1 → Stage 2 |
|---|---|---|
| Frozen representation | Retrieval, no training (1st / top 5 of 10; see W1) | 11.0% / 50.5% → 19.5% / 71.7% |
| Frozen representation | UCR/UEA accuracy (5 seeds) | 0.736 → 0.753 (15 datasets better, 7 worse; p < 0.01) |
| Frozen representation | Time-MMD frozen head, NMAE ↓ | 0.524 → 0.514 |
| Frozen representation | TimeCAP frozen head, macro-F1 (r from validation) | 0.607 → 0.609 (tie) |
| Zero-shot generation | NLU accuracy | 0.411 → 0.406 (n.s.) |
| Zero-shot generation | GIFT-Eval MASE / CRPS ↓ | 0.978 / 0.690 → 1.053 / 0.754 |
Stage 2 improves or ties every evaluation of the frozen representation, which is how a foundation model is used downstream. The cost is confined to zero-shot generation.
The UCR/UEA numbers are 5-seed results for every model on all 24 datasets under the probe configuration stated in the paper (Appendix F.4). They supersede the submitted Table 7 entry (0.729 ± 0.057), and Tables 3 and 7 will be updated accordingly. Section 5.3 will be rewritten around this analysis.
Q1. Why is Time-MMD LoRA worse with Stage 2?
We broke the result down by domain. Stage 2 is better than Stage 1 on 4 of the 9 domains and essentially tied on two more (Economy +0.003 and Public Health +0.005 MAE). The average is affected by two small single-channel domains: Security (+0.225 MAE), which has 207 training rows, and Social Good (+0.128 MAE), where the three seeds range from 0.50 to 0.79 MAE (standard deviation 0.146).
LoRA is the less reliable protocol at this data scale for both checkpoints: it is worse than the frozen head on 5 of 9 domains for Stage 1 and on 7 of 9 for Stage 2. The LoRA average therefore reflects the instability of fine-tuning on very small datasets and not an effect of the alignment stage. Under the frozen head, which has no such instability, Stage 2 is the better checkpoint (0.514 vs. 0.524).
Summary. Joint input beats both single-modality variants of the same checkpoint (W1). Against other pretrained models under one frozen probe, Chronicle gives the strongest time series representations (W2). Stage 2 improves or ties every evaluation of the frozen representation, and the one apparent exception traces to two very small domains (W3, Q1). Parameter counts are added (W4). We hope this resolves your concerns, and we would be grateful if you would reconsider your rating.
Response to Rebuttal
Thank you for the detailed explanation and additional results. Most of my concerns are addressed. For W2, my concern is about the comparison between Chronicle, which benefits from large-scale external pretraining, and supervised DL baselines trained independently on each downstream dataset without comparable pretraining. However, the results in Section 5.2.1 compares Chronicle with pretrained time series foundation models that were trained on far more time series tokens than Chronicle. I agree that comparisons with other pretrained time-series foundation models provide useful evidence of representation quality. However, this does not resolve the fairness concern regarding supervised DL baselines.
One thing I want to remind you is that you can update the new version of the pdf file during the rebuttal phase.
Thank you for the review and for the concrete suggestions, each of which we have carried out. The ablation you propose in W1 is done and supports the motivation mentioned. We would like to clarify a few things about the framing concerns in W2 and the comparisons with other pretrained models. The inconsistency in W3 resolves into a consistent pattern once the evaluations are grouped by what they measure.
W1. No ablation against text-only and series-only variants of the same pretrained checkpoint.
We ran this ablation. The Stage 2 checkpoint is frozen and the same head is trained on TimeCAP with joint, text-only and series-only input, plus a control in which captions are shuffled across samples, in training and test alike. Shuffling keeps the text and breaks only its link to the series. Results are means over 5 seeds and 3 domains.
The joint model beats each single-modality variant in all 5 seeds. With shuffled captions it falls to chance at repetition factor r = 1 and stays below series only at r = 64 (AUC 0.588), so it relies on the text matching the series and not on the presence of extra text.
We also tested whether the frozen model relates text and series before any downstream training. We take 771 held-out (series, description) pairs from the alignment data. Each description is placed after its true series and after 9 other series drawn at random, and we rank the 10 series by the likelihood the frozen model assigns to the description. No head or parameter is trained.
Stage 1 is at chance. Stage 2 ranks the true series first about twice as often as chance, and the 95% bootstrap CIs exclude chance.
Finally, we checked that this ability comes from joint training on paired data and not from more training. Three runs continue from the same Stage 1 checkpoint with the same architecture, schedule and compute (2B tokens). They differ only in a slice of 15% of the batches: real text–series pairs, the same series and descriptions re-paired at random, or no pairs (more time series instead).
Only real pairs produce the ability (real against random pairs: +7.0 points, 95% CI [3.9, 10.2]). The same amount of extra training without pairs leaves the model at chance. This is direct evidence for the motivation you quote: the relationship between the modalities is learned by joint training and is measurable in the backbone itself.