- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes complementary representations as a design principle for time-series language models, instantiated via a dual-view architecture that feeds a VLM with both a line-plot view and a normalized delay-embedding view of the same series. Extensive experiments on multiple time series reasoning benchmarks show the gains over single-view variants and competitive or state-of-the-art performance compared to specialized TSLMs.
Critical strengths
- The complementary view of a single time series is a promising approach and still remains underexplored.
- The paper is easy to follow and understand.
- The experiments on multiple reasoning benchmarks justify the advantage of the proposed method.
Critical weaknesses
- Though a visualized time series is effective, it is unclear how does it handle multi-variate time series with a larger number of series needed to handle. Moreover, for those excessively long time series, the visualization loses details. More discussion or potential solutions to these problems would be beneficial.
- A computational efficiency analysis is missing, since DINOv3-L + Qwen-VL visual stacks may be costly in terms of both memory and inference time.
- Did you examine learned attention patterns or token attributions to see whether different tasks rely more on the plot or delay tokens?
- In multi-view learning, models often rely on a single representation to generate the prediction. A more detailed analysis on this aspect would provide a deeper understanding of the advantage of complementarity.
Questions for authors
See weaknesses above.
3 comments
part 2
W3
W3. Did you examine learned attention patterns or token attributions to see whether different tasks rely more on the plot or delay tokens?
Thank you for this suggestion; we ran this analysis and added it to the paper. We measured it on 600 TSRBench items (50 per task file). We take the attention at the answer position (the last prompt token) to plot and delay tokens, averaged over heads and over the layers in each group. Since a series has 230 plot tokens and 64 delay tokens, we also compare attention per token (each view's attention divided by its token count, averaged over items):
| Layers | Plot attention | Delay attention | Attention per delay token / per plot token |
|---|---|---|---|
| Early (1–12) | 0.026 | 0.024 | 2.86 |
| Middle (13–24) | 0.086 | 0.011 | 0.49 |
| Late (25–36) | 0.091 | 0.007 | 0.23 |
| All | 0.068 | 0.014 | 0.63 |
By TSRBench category (all layers):
| Category | Plot attention | Delay attention | Per delay token / per plot token |
|---|---|---|---|
| Perception | 0.063 | 0.014 | 0.76 |
| Reasoning | 0.065 | 0.011 | 0.57 |
| Prediction | 0.098 | 0.029 | 1.07 |
| Decision | 0.052 | 0.009 | 0.52 |
| All | 0.068 | 0.014 | 0.63 |
In the early layers, the delay tokens together receive almost as much attention as the plot tokens, although there are 3.6× fewer of them (64 vs. 230 per series), while in the late, answer-forming layers attention goes mainly to the plot. Across tasks, attention to delay tokens is highest for prediction and lowest for reasoning and decision.
W4
W4. In multi-view learning, models often rely on a single representation to generate the prediction. A more detailed analysis on this aspect would provide a deeper understanding of the advantage of complementarity.
We thank the reviewer for pointing this out and helping us strengthen our manuscript. We examined this question by distinguishing the benefit of complementary views during training from their contributions at inference.
To measure inference-time reliance, we remove each view from the trained model at test time: we omit that view's embeddings, so the prompt is exactly as if the view had never been rendered, and compare accuracy and the share of answers that change against the model with both views. Removing the line plot reduces TSRBench accuracy by approximately 10 points, whereas removing the delay view leaves aggregate accuracy nearly unchanged. The delay view nevertheless affects predictions: removing it changes 6.1% of TSRBench answers, 2.8% of TSExam answers and 14.0% of ChatTS-B categorical answers (3.3% of numerical answers change by more than 0.1), and slightly reduces TSExam accuracy and ChatTS-B alignment. These results show asymmetric inference-time reliance, consistent with prior observations of unequal modality utilization [1, 2, 3].
In addition, our training ablations establish the benefit of learning from complementary views: the dual-view model outperforms matched Plot-only and Plot+Plot controls by 3.7 and 4.4 points, respectively, using the same backbone, data, and training schedule (Table 3). Such gains can coexist with unequal modality utilization, as also observed by Peng et al. [2]. Related work on multimodal training and unimodal deployment further demonstrates that access to additional modalities during training can improve performance even when those inputs are unavailable at inference [4].
Together, our results indicate that the value of the delay view extends beyond its marginal contribution at inference. They support a representation-learning benefit from complementary-view training that remains useful even when the plot provides the dominant inference-time signal. We have added the view-removal results and this discussion to Appendix C.9.
References
[1] Characterizing and Overcoming the Greedy Nature of Learning in Multi-modal Deep Neural Networks. Wu et al., ICML 2022.
[2] Balanced Multimodal Learning via On-the-fly Gradient Modulation. Peng et al., CVPR 2022.
[3] See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Bias. Kwon et al., NAACL 2025.
[4] Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment. Hong et al., TMLR 2026.
Thank you to the authors for their efforts during the rebuttal. The additional experiments and clarifications address most of my concerns. I will maintain my current score.
part 1
W1
Thank you for raising this.
To handle multivariate time series, each variate gets its own line plot and its own delay embedding, resulting in the use of 298 visual tokens per variate, so computation grows linearly with the number of variates. To see whether the number of variates affects the model's performance, we split TSRBench by the number of variates per item:
Accuracy varies moderately up to 9 variates (48.0–55.9%) and drops to 31.9% at 10 or more. We cannot conclude from this that the model is weaker with many variates: all items with 10 or more variates come from a single 12-lead ECG task, so the drop may reflect that task rather than the number of variates. Such items are also rare in our training data (1.5%). In contrast, on the multivariate ChatTS-B benchmark our model handles multiple variates better than ChatTS-14B (0.881 vs. 0.787 numerical alignment), which further suggests that the drop is not a general limitation with many variates.
To handle long time series, both views are rendered at a fixed resolution, so the token count stays the same regardless of length. To see whether series length affects the model's performance, we regenerated TSExam at each length using the benchmark-generation code from the TSExam repository (755 items each):
Our model holds up to 1,024 points, about the original length, and degrades beyond it, mainly on anomaly detection (97.2% at 256 points vs. 49.1% at 8,192). We hypothesize that, as the series grows, the anomaly occupies an ever smaller part of each view, so finding it becomes a harder task, and that two factors contribute: our training data has few long series (at most about 1,200 points), and because both views have a fixed size, each part of a long series gets less visual detail.
We will add the following limitation to the paper, with possible solutions:
W2
Thank you; we measured it and added the table to the paper (Appendix C.10). Setup: one RTX 4090, batch size 1, bf16, greedy decoding, about 200 TSRBench items drawn across all tasks. Latency includes rendering and tokenization, and both models give identical answer lengths.
The DINOv3 encoder takes 18 ms per sample, and decoding speed is unchanged (48 ms per token). The overhead is in the prefill, so with a fixed 32-token output it falls to 2.6%.