acceptodds
Under review as a conference paper at ICLR 2027

Twice Upon a Time: Complementary Representations for Time-Series Reasoning

Abstract

Time Series Language Models (TSLMs) enable language-based understanding and reasoning over temporal data, yet a fundamental question remains: what information should their time-series representations preserve, and what variation should they factor out? Existing TSLMs typically expose the language model to a single representation, imposing a fixed inductive bias across downstream tasks. We instead introduce complementary representations as a design principle for time-series reasoning. Our model combines two visual views: line plots preserve absolute measurements and the observed coordinate frame, while normalized delay embeddings factor out positive amplitude rescaling and offsets while exposing temporal structure. Dedicated visual encoders integrate both views into a shared language model through visual tokens and multi-level feature fusion. Evaluation-time interventions verify that the two views exhibit distinct invariance profiles, while single-view ablations show that their combination is particularly beneficial beyond basic perception, with the large gains in prediction and reasoning. Built on Qwen3-VL-8B, our dual-view model improves overall TSRBench accuracy from 44.5% with line plots alone to 53.2%, substantially exceeding the strongest evaluated dedicated TSLM baseline at 36.7%. It further achieves 89.9% accuracy on TSExam and 86.8% numerical accuracy on multivariate ChatTS alignment. Together, these results demonstrate that complementary representations provide an effective design principle for general-purpose TSLMs, enabling models to exploit temporal structure without sacrificing access to the original measurement frame.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
time series reasoning, time series language models, multimodal large language models, vision-language models, representation learning

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_ovqGOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes complementary representations as a design principle for time-series language models, instantiated via a dual-view architecture that feeds a VLM with both a line-plot view and a normalized delay-embedding view of the same series. Extensive experiments on multiple time series reasoning benchmarks show the gains over single-view variants and competitive or state-of-the-art performance compared to specialized TSLMs.

Critical strengths
  1. The complementary view of a single time series is a promising approach and still remains underexplored.
  2. The paper is easy to follow and understand.
  3. The experiments on multiple reasoning benchmarks justify the advantage of the proposed method.
Critical weaknesses
  1. Though a visualized time series is effective, it is unclear how does it handle multi-variate time series with a larger number of series needed to handle. Moreover, for those excessively long time series, the visualization loses details. More discussion or potential solutions to these problems would be beneficial.
  2. A computational efficiency analysis is missing, since DINOv3-L + Qwen-VL visual stacks may be costly in terms of both memory and inference time.
  3. Did you examine learned attention patterns or token attributions to see whether different tasks rely more on the plot or delay tokens?
  4. In multi-view learning, models often rely on a single representation to generate the prediction. A more detailed analysis on this aspect would provide a deeper understanding of the advantage of complementarity.
Questions for authors

See weaknesses above.

3 comments

W1

W1. Though a visualized time series is effective, it is unclear how does it handle multi-variate time series with a larger number of series needed to handle. Moreover, for those excessively long time series, the visualization loses details. More discussion or potential solutions to these problems would be beneficial.

Thank you for raising this.

To handle multivariate time series, each variate gets its own line plot and its own delay embedding, resulting in the use of 298 visual tokens per variate, so computation grows linearly with the number of variates. To see whether the number of variates affects the model's performance, we split TSRBench by the number of variates per item:

Variates per itemItemsOurs
11,58051.4
2–41,50055.9
5–935048.0
≥1033531.9

Accuracy varies moderately up to 9 variates (48.0–55.9%) and drops to 31.9% at 10 or more. We cannot conclude from this that the model is weaker with many variates: all items with 10 or more variates come from a single 12-lead ECG task, so the drop may reflect that task rather than the number of variates. Such items are also rare in our training data (1.5%). In contrast, on the multivariate ChatTS-B benchmark our model handles multiple variates better than ChatTS-14B (0.881 vs. 0.787 numerical alignment), which further suggests that the drop is not a general limitation with many variates.

To handle long time series, both views are rendered at a fixed resolution, so the token count stays the same regardless of length. To see whether series length affects the model's performance, we regenerated TSExam at each length using the benchmark-generation code from the TSExam repository (755 items each):

Series length2565121,0242,0484,0968,192
TSExam accuracy95.595.089.980.576.374.3

Our model holds up to 1,024 points, about the original length, and degrades beyond it, mainly on anomaly detection (97.2% at 256 points vs. 49.1% at 8,192). We hypothesize that, as the series grows, the anomaly occupies an ever smaller part of each view, so finding it becomes a harder task, and that two factors contribute: our training data has few long series (at most about 1,200 points), and because both views have a fixed size, each part of a long series gets less visual detail.

We will add the following limitation to the paper, with possible solutions:

  • Long series: training on longer series, a representation more robust to length (e.g. tiling the line plot over time or a multi-scale delay map), or a higher rendering resolution, trading some of the fixed token cost for detail.

W2

W2. A computational efficiency analysis is missing, since DINOv3-L + Qwen-VL visual stacks may be costly in terms of both memory and inference time.

Thank you; we measured it and added the table to the paper (Appendix C.10). Setup: one RTX 4090, batch size 1, bf16, greedy decoding, about 200 TSRBench items drawn across all tasks. Latency includes rendering and tokenization, and both models give identical answer lengths.

Plot onlyOursChange
Total parameters8.77B9.20B+0.44B (+5.0%)
Trainable parameters (Stage II)87.3M87.3Mnone
Visual tokens per series232298+28%
Latency per sample759 ms820 ms+8%
Peak GPU memory20.7 GB21.6 GB+4.5%

The DINOv3 encoder takes 18 ms per sample, and decoding speed is unchanged (48 ms per token). The overhead is in the prefill, so with a fixed 32-token output it falls to 2.6%.

W3

W3. Did you examine learned attention patterns or token attributions to see whether different tasks rely more on the plot or delay tokens?

Thank you for this suggestion; we ran this analysis and added it to the paper. We measured it on 600 TSRBench items (50 per task file). We take the attention at the answer position (the last prompt token) to plot and delay tokens, averaged over heads and over the layers in each group. Since a series has 230 plot tokens and 64 delay tokens, we also compare attention per token (each view's attention divided by its token count, averaged over items):

LayersPlot attentionDelay attentionAttention per delay token / per plot token
Early (1–12)0.0260.0242.86
Middle (13–24)0.0860.0110.49
Late (25–36)0.0910.0070.23
All0.0680.0140.63

By TSRBench category (all layers):

CategoryPlot attentionDelay attentionPer delay token / per plot token
Perception0.0630.0140.76
Reasoning0.0650.0110.57
Prediction0.0980.0291.07
Decision0.0520.0090.52
All0.0680.0140.63

In the early layers, the delay tokens together receive almost as much attention as the plot tokens, although there are 3.6× fewer of them (64 vs. 230 per series), while in the late, answer-forming layers attention goes mainly to the plot. Across tasks, attention to delay tokens is highest for prediction and lowest for reasoning and decision.

W4

W4. In multi-view learning, models often rely on a single representation to generate the prediction. A more detailed analysis on this aspect would provide a deeper understanding of the advantage of complementarity.

We thank the reviewer for pointing this out and helping us strengthen our manuscript. We examined this question by distinguishing the benefit of complementary views during training from their contributions at inference.

To measure inference-time reliance, we remove each view from the trained model at test time: we omit that view's embeddings, so the prompt is exactly as if the view had never been rendered, and compare accuracy and the share of answers that change against the model with both views. Removing the line plot reduces TSRBench accuracy by approximately 10 points, whereas removing the delay view leaves aggregate accuracy nearly unchanged. The delay view nevertheless affects predictions: removing it changes 6.1% of TSRBench answers, 2.8% of TSExam answers and 14.0% of ChatTS-B categorical answers (3.3% of numerical answers change by more than 0.1), and slightly reduces TSExam accuracy and ChatTS-B alignment. These results show asymmetric inference-time reliance, consistent with prior observations of unequal modality utilization [1, 2, 3].

In addition, our training ablations establish the benefit of learning from complementary views: the dual-view model outperforms matched Plot-only and Plot+Plot controls by 3.7 and 4.4 points, respectively, using the same backbone, data, and training schedule (Table 3). Such gains can coexist with unequal modality utilization, as also observed by Peng et al. [2]. Related work on multimodal training and unimodal deployment further demonstrates that access to additional modalities during training can improve performance even when those inputs are unavailable at inference [4].

Together, our results indicate that the value of the delay view extends beyond its marginal contribution at inference. They support a representation-learning benefit from complementary-view training that remains useful even when the plot provides the dominant inference-time signal. We have added the view-removal results and this discussion to Appendix C.9.

References

[1] Characterizing and Overcoming the Greedy Nature of Learning in Multi-modal Deep Neural Networks. Wu et al., ICML 2022.

[2] Balanced Multimodal Learning via On-the-fly Gradient Modulation. Peng et al., CVPR 2022.

[3] See-Saw Modality Balance: See Gradient, and Sew Impaired Vision-Language Balance to Mitigate Dominant Modality Bias. Kwon et al., NAACL 2025.

[4] Efficient Audiovisual Speech Processing via MUTUD: Multimodal Training and Unimodal Deployment. Hong et al., TMLR 2026.

Thank you to the authors for their efforts during the rebuttal. The additional experiments and clarifications address most of my concerns. I will maintain my current score.

Reviewer_6fWNOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper investigates how two complementary image representations can improve language models’ understanding of and reasoning about time series. The authors convert each time series into a line plot and a normalized delay plot. The former preserves absolute values and coordinate information, whereas the latter highlights the structure of temporal variation. The two images are processed by separate visual encoders and then fed into the same language model. The paper reports improvements on three benchmarks.

Critical strengths
  1. The research idea is clear and practically relevant. Time-series tasks involve both reading specific values and identifying patterns of change, which place different demands on the representation. Combining line plots that preserve numerical values with delay plots that remove scale differences offers a useful perspective on selecting and combining time-series representations.

  2. The key comparisons are reasonably fair, and the gains on some tasks are substantial. The authors include Plot+Plot and Delay+Delay controls while keeping the dual-encoder architecture, number of visual tokens, and training procedure consistent. These controls help rule out the explanation that the improvements simply come from increased model capacity or additional inputs. In Table 3, the full model outperforms Plot-only by 13.4, 10.6, and 11.0 percentage points on source identification, temporal relation reasoning, and inductive reasoning, respectively. These results indicate that combining the two views is indeed helpful for some tasks.

Critical weaknesses
  1. The main performance gain has not yet been shown to exceed a simple majority-class baseline. An item-by-item count of the official TSRBench event prediction (EP) data shows that 303 of the 360 questions have “no rain” as the correct answer. Therefore, a baseline that ignores the time series and always predicts “no rain” achieves 84.2% accuracy, exactly matching the full model’s EP score in Table 3. This count is based on a fixed version of the official EP dataset. (https://huggingface.co/datasets/umd-zhou-lab/TSRBench/blob/83c50b3becf15746187f79af23a2e9748dafdc05/prediction/event_prediction.jsonl)
  2. The experiments support the effectiveness of combining the two views, but do not sufficiently validate the authors’ proposed explanation. The paper attributes the gains to the different invariance properties of the two representations. However, the existing same-view and STFT controls do not isolate the respective contributions of normalization, the delay-window arrangement, and other factors. It therefore remains unclear whether the improvements primarily arise from the “complementary invariances” emphasized by the authors. The evaluation in Table 4 also has limitations. A small change in aggregate accuracy does not imply that predictions remain unchanged for individual questions: some predictions may change from correct to incorrect while others change in the opposite direction, causing the effects to cancel out. Moreover, page 5 states that positive scaling and translation should produce identical normalized delay plots, yet the accuracy of Delay-only in Table 4 changes from 44.7% to 43.4% after scaling.
Questions for authors

The first three items primarily concern additional analyses of existing results and clarification of the evaluation setup, and may directly affect the assessment of the paper.

  1. Does event prediction actually use the time-series information? Please confirm the EP dataset version, the actual number of evaluated questions, and the class distribution. Please also report the majority-class baseline, confusion matrices for the full model and Plot-only, recall for each class, and balanced accuracy, defined as the average recall across the two classes.
  2. How much of the dual-view advantage remains after excluding EP? Please report the overall accuracy of the full model, Plot-only, and Plot+Plot after excluding EP, together with results averaged equally across tasks. Please also clarify whether all models are scored on the same number of questions and how unparseable answers are scored, so that readers can assess whether the gains consistently extend across multiple tasks.
  3. Why do identical delay plots produce different results? Please specify the model checkpoint, decoding method, and random seed used for Table 4. Please check whether the delay plots are identical before and after scaling and translation, and report the proportion of questions with identical predictions under deterministic decoding. This would help distinguish the effects of representation changes from those of inference randomness.
  4. Additional ablations are recommended to test the proposed mechanism. While keeping the encoders, number of inputs, and training procedure consistent, please compare “original line plot + normalized line plot,” “original line plot + unnormalized delay plot,” and the current method. These comparisons would help distinguish the contribution of removing numerical scale from that of the delay arrangement.
6 comments

W1 / Q1 / Q2

W1. The main performance gain has not yet been shown to exceed a simple majority-class baseline. ...

Q1. Does event prediction actually use the time-series information? ...

Q2. How much of the dual-view advantage remains after excluding EP? ...

Thank you; you are correct. We used the same version of the EP dataset that you linked (TSRBench, commit 83c50b3): 360 questions, 303 "No Rain" and 57 "Rain". Always answering "No Rain" scores 84.2%, exactly our EP score.

Every model is at chance by balanced accuracy (the mean recall over the two classes): 50.0% for Ours, 50.5% for Plot only and 49.3–51.8% for all our variants, against 50.0% for the majority baseline. So EP gives no evidence of reasoning for any variant. Our model predicts "No Rain" on every EP item, matching the majority baseline; with the answer options reversed it still does, so it selects the content, not the letter. We therefore withdraw EP as evidence of time-series-dependent reasoning.

We now report TSRBench with and without EP (as in our response to Reviewer tXpi, W2):

ModelTSRBench w/o EPTSRBench with EP
GPT-5 (T+V)*53.455.6
GPT-5 (T)*53.255.5
GPT-5-mini-high (T+V)*52.754.1
o4-mini-high (T+V)*50.652.5
GPT-5 (V)*50.552.4
Gemini-2.5-Flash (T+V)*44.646.5
Claude-4.5-Haiku (T+V)*36.037.1
ChatTS-14B31.733.5
TS-Reasoner-7B34.636.4
TimeOmni-1-7B35.536.7
Qwen3-VL-8B41.038.9
Qwen3-VL-32B44.845.0
Plot+Plot46.845.0
Plot only47.545.2
Ours51.254.0

*Proprietary models (T: text, V: visual input).

Without EP, our model still outperforms every open model, dedicated TSLM and matched control (3.7 and 4.4 points above Plot only and Plot+Plot), as well as GPT-5 (V) and o4-mini-high.

All three models are scored on the same 3,765 questions without EP (4,125 with EP). Each answer is parsed as an option letter, and unparseable answers count as incorrect; every answer of these models parses. Averaged equally over the 14 tasks without EP (perception split into its four sub-tasks), our model reaches 54.9%, against 51.7% for Plot only and 50.7% for Plot+Plot, and it improves over Plot only on 9 of 11 task groups (see Reviewer tXpi, W2).

W2 / Q3 / Q4

W2. The experiments support the effectiveness of combining the two views, but do not sufficiently validate the authors' proposed explanation. ...

Q3. Why do identical delay plots produce different results? ...

Q4. Additional ablations are recommended to test the proposed mechanism. ...

Thank you; both parts led to important findings.

Normalization vs. delay embedding. We trained second views that separate the two factors, with the same data, schedule and token budget as our model:

ModelTSRBench w/o EP
Plot only47.5
Plot+Plot46.8
Plot + normalized plot (DINOv3 tower)46.1
Plot + normalized plot (native encoder)45.6
Plot + normalized STFT47.8
Plot + delay map, global linear range42.6
Plot + delay map, global signed-log range44.1
Ours (Plot + normalized delay map)51.2

Normalized views without the delay embedding stay at or near Plot only (45.6–47.8 vs. 47.5). Without per-series normalization, a delay map needs one global value-to-intensity range, which the series with the largest values dictate, so most series are compressed into a few intensities (70.5% use less than 1% of the linear range); both variants fall below Plot only. Only the combination reproduces the gain, and we will revise the mechanism text accordingly.

Table 4. You are right that aggregate accuracy hides per-item changes. Mathematically, the delay map should be identical under these transformations: each series is normalized by its own minimum and maximum, x̃ = (x − min x) / (max x − min x), which is unchanged by x → a·x + b for a > 0. The original Delay-only shift (44.7% → 43.4%) came mostly from sampled decoding (see the note to all reviewers). All three models are evaluated at their final checkpoint (Stage I: 2,613 steps; Stage II: 5,714 steps) with greedy decoding, one item per forward pass and deterministic kernels, so no seed is involved (the original sampled run set none); re-running the evaluation reproduces every prediction exactly. The affine part of the revised Table 4 (1,660 items without EP; accuracy, with the share of answers that differ from clean in parentheses):

ModelClean×0.1×10+5σ−5σz-score
Plot only56.852.2 (20.0%)54.5 (13.4%)52.7 (16.3%)52.3 (17.1%)52.5 (26.4%)
Delay only47.447.4 (3.2%)47.3 (2.6%)47.3 (2.1%)47.5 (2.1%)47.9 (2.9%)
Ours60.156.1 (14.5%)58.2 (12.5%)55.2 (15.0%)57.2 (13.4%)52.5 (19.5%)
  • The remaining Delay-only changes are numerical. The delay maps differ from clean only by rounding errors in their construction (at most 1.6·10⁻⁵ on the [0, 1] scale). On the 24–37% of items whose input stays bit-identical, no answer changes, and building the map in float64 gives answers identical to the clean version.
  • Ours is more stable than Plot only: it changes fewer answers under every transformation (12.5–19.5% vs. 13.4–26.4%).

The delay map remains invariant to these transformations; we will clarify that small numerical errors in converting the series can still change a few answers in practice.

Thank you for the clarification. I will not lower my score. However, the baselines in Table 3 are outdated. If you can add more recent baselines, I will raise my score. Suggested baselines include GPT-6 Astra, GPT-6.1 Sol, GPT-6.1 Terra, Claude Opus 5.5, Claude Sonnet 5.5, Gemini 4 Carbon, Gemini 4 Argon, DeepSeek-V4.1-Flash, Kimi K3, and GLM-5.3.

Reviewer_tXpiOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary
  • The authors explore whether giving a language model two different views of the same time series can help it better understand and reason about temporal data.

  • They combine traditional line plots, which preserve the original values, with normalized delay embeddings, which focus on temporal patterns regardless of scale. Both representations are fed into a Qwen3-VL-8B model through separate visual encoders.

  • Using both representations works better than using either one alone, with the biggest improvements coming from prediction and reasoning tasks rather than basic pattern recognition.

  • The paper shows that how we represent time series matters, and combining representations that capture different aspects of the data can lead to better reasoning performance.

Critical strengths

S1. The paper provides a clear motivation for combining line plots, which preserve absolute numerical information, with normalized delay embeddings, which capture temporal structure independently of scale. The analysis of how these representations respond to different transformations further supports the proposed design.

S2. The authors provide useful ablation studies comparing single-view and dual-view models, including controls that use the same representation in both branches. These experiments help demonstrate the benefit of combining complementary representations, with a notable improvement in overall TSRBench performance.

Critical weaknesses

W1. Several relevant works appear to be missing, particularly MLLM4TS [1], DMMV [2], and Time-VLM [3], which have previously explored combining complementary time-series representations. VL-Time [4] and VLM-TSC [5] are also relevant to visualization-based time-series understanding. While the proposed approach differs in its architecture and target tasks, I would appreciate a clearer discussion of how it builds upon these works and what distinguishes its contribution. Comparisons with the most relevant approaches, where feasible, would also help establish the novelty.

W2. Although the overall improvement on TSRBench is encouraging, much of the gain appears concentrated in Event Prediction (21.7% to 84.2%). Several other tasks show smaller improvements or declines. Additionally, TSExam accuracy remains unchanged at 89.9%, while ChatTS reasoning improves only marginally. It would be helpful to understand why the benefits vary so considerably across tasks.

W3. The ablations are useful, including the comparison with STFT. However, it remains unclear whether the improvements arise specifically from the temporal structure captured by delay embeddings or from introducing a second normalized representation. A comparison using an unnormalized line plot alongside a normalized line plot could help clarify this distinction.

References

[1] MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis. Liu et al.

[2] Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting. Shen et al.

[3] Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting. Zhong et al.

[4] A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualization. Liu et al.

[5] On the Feasibility of Vision-Language Models for Time-Series Classification. Prithyani et al.

Questions for authors

Q1. Could the authors clarify how their approach differs from the related works mentioned in W1, particularly MLLM4TS, DMMV, and Time-VLM? What would they consider the main methodological novelty of this work?

Q2. Given that much of the improvement on TSRBench comes from Event Prediction, could the authors explain why the benefits are less consistent across other tasks and benchmarks?

Q3. Would combining an unnormalized line plot with a normalized line plot produce similar improvements? I think this comparison would help establish whether the gains are specific to delay embeddings rather than simply adding another normalized representation.

3 comments

W1 / Q1

W1. Several relevant works appear to be missing, particularly MLLM4TS [1], DMMV [2], and Time-VLM [3], ...

Q1. Could the authors clarify how their approach differs from the related works mentioned in W1, ...

Thank you; we will add all five works to Related Work and Table 1. Prior work combines a rendering with the raw series (MLLM4TS [1], VLM-TSC [5], DMMV [2]), fuses a time-series encoder through a frozen VLM (Time-VLM [3]), or selects one visualization per task (VL-Time [4]). As you note, these works target forecasting or classification, or prompt frozen VLMs, rather than learning a time-series representation for free-form question answering. Our contribution is complementary representations as a design principle for time-series reasoning in TSLMs: we pair a line plot that keeps the scale with a delay embedding normalized per series that captures the dynamics independent of scale, so that each view supplies what the other lacks. Because both views are images, the model needs no patching or tokenizer for the raw series: each view uses a pretrained vision encoder, and the token count is fixed regardless of series length. To our knowledge, no prior work studies complementary representations for time-series reasoning in TSLMs, and the combination itself is new: delay embeddings have not been used as a TSLM input before. In matched controls, this pairing outperforms duplicated views, a normalized line plot, a normalized STFT and unnormalized delay maps (see W3; Reviewer 6fWN, W2).

W2 / Q2

W2. Although the overall improvement on TSRBench is encouraging, much of the gain appears concentrated in Event Prediction (21.7% to 84.2%). ...

Q2. Given that much of the improvement on TSRBench comes from Event Prediction, ...

Thank you; you are correct that EP is imbalanced: 303 of its 360 answers are "No Rain", so always answering "No Rain" scores 84.2%. We withdraw EP as evidence of reasoning (see Reviewer 6fWN, W1). Without EP, our model remains strong:

ModelTSRBench w/o EPTSRBench with EP
GPT-5 (T+V)*53.455.6
GPT-5 (T)*53.255.5
GPT-5-mini-high (T+V)*52.754.1
o4-mini-high (T+V)*50.652.5
GPT-5 (V)*50.552.4
Gemini-2.5-Flash (T+V)*44.646.5
Claude-4.5-Haiku (T+V)*36.037.1
ChatTS-14B31.733.5
TS-Reasoner-7B34.636.4
TimeOmni-1-7B35.536.7
Qwen3-VL-8B41.038.9
Qwen3-VL-32B44.845.0
Plot+Plot46.845.0
Plot only47.545.2
Ours51.254.0

*Proprietary models (T: text, V: visual input).

Our model is 3.7 and 4.4 points above the matched Plot-only and Plot+Plot controls, about 16 above the strongest dedicated TSLM, and ahead of GPT-5 (V) and o4-mini-high.

Across tasks. Gain over Plot only (points):

PercERCDARTRNRDRIRTSFQuaDMQuanDM
−0.4+11.7+0.3+7.3+9.4+1.8+2.4+3.0+5.7−5.7+11.7

Gains are largest where, in our interpretation, temporal structure matters (ER, AR, TR, TSF, QuanDM). Perception shows no improvement (87.0 vs. 87.4), and the largest decline is qualitative decision, a 12-lead ECG task (see Reviewer ovqG, W1).

TSExam is mainly a perception benchmark: our model reaches 0.899, on par with Plot only (0.897), and is ahead of every baseline (GPT-5 T+V: 0.808).

ChatTS alignment:

ModelA: cat.A: num.B: cat.B: num.Mean
ChatTS-14B0.8890.7880.8620.7870.832
Plot only0.8470.7750.8950.8640.845
Ours0.8840.8080.8970.8810.867

Our model improves over Plot only on all four metrics and is best on average. ChatTS reasoning is on par with Plot only (0.605 vs. 0.609, mean of the four reasoning evaluations), so we claim an alignment gain on ChatTS, not a reasoning gain.

Why the benefits vary. Once EP is removed, the gain is broad: our model improves over Plot only on 9 of 11 TSRBench task groups (mean +4.3 points). Its size follows the task: largest where temporal dynamics matter, and absent where the line plot alone already suffices (perception, TSExam).

W3 / Q3

W3. The ablations are useful, including the comparison with STFT. However, it remains unclear whether the improvements arise specifically from the temporal structure captured by delay embeddings ...

Q3. Would combining an unnormalized line plot with a normalized line plot produce similar improvements? ...

Thank you; we trained this control. The second tower receives a z-normalized line plot instead of the delay map (no axes and a fixed y-range, so it keeps the shape but not the scale), with the same data, schedule and 64-token budget as our model. We also feed the normalized plot to the native vision encoder, to rule out the choice of tower, and compare a normalized STFT image as the second view:

ModelTSRBench w/o EPTSRBench with EP
Plot only47.545.2
Plot+Plot46.845.0
Plot + normalized plot (DINOv3 tower)46.143.5
Plot + normalized plot (native encoder)45.643.2
Plot + normalized STFT47.846.7
Ours (Plot + normalized delay map)51.254.0

A second normalized view does not reproduce the gain: the normalized line plot falls below Plot only on either tower, and the normalized STFT stays close to Plot only. Of these matched alternatives, only the normalized delay map clearly improves over Plot only.

References

[1] MLLM4TS: Leveraging Vision and Multimodal Language Models for General Time-Series Analysis. Liu et al.

[2] Multi-Modal View Enhanced Large Vision Models for Long-Term Time Series Forecasting. Shen et al.

[3] Time-VLM: Exploring Multimodal Vision-Language Models for Augmented Time Series Forecasting. Zhong et al., ICML 2025.

[4] A Picture is Worth A Thousand Numbers: Enabling LLMs Reason about Time Series via Visualization. Liu et al.

[5] On the Feasibility of Vision-Language Models for Time-Series Classification. Prithyani et al.

Reviewer_tXpiOpenReview ↗

primary concern unresolved; maintaining score

I appreciate the authors taking the time to respond to my concerns. While the authors have included the suggested literature in their related works section, It is still unclear to me why their choice of representations is preferable to existing raw time series + plots pairings as used in previous works. I will therefore maintain my original score.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Note to all reviewers

Note to all reviewers

We thank the reviewers for their careful reading. While preparing this response, we found that our original evaluations used sampled decoding (temperature 0.7) instead of greedy decoding, because a library default replaced our setting, which changed some of the results. We therefore re-ran every evaluation of our model and of all our ablations with greedy decoding and deterministic kernels. All numbers for our models in this response come from these runs, and the revised paper will update all tables and figures accordingly; external baselines are reported as published. Most numbers move by less than a point (e.g. TSRBench: 53.2% → 54.0%); where a conclusion changes, we say so in the relevant answer.

AuthorsOpenReview ↗

Summary of revisions to the manuscript

We thank all three reviewers for their careful reading and constructive suggestions. Their feedback helped us strengthen the experimental evidence, clarify our claims, and improve the paper's presentation. Changes are marked in red in the revised PDF:

  1. Greedy re-evaluation (all reviewers). We re-ran all evaluations of our model and ablations with greedy decoding; most numbers move by under a point (TSRBench: 53.2% → 54.0%). The protocol is described in App. B.3.

  2. Event prediction (EP) (tXpi, 6fWN). We added an analysis of EP (App. C.1) and report TSRBench results with and without it; without EP, our method still outperforms all matched controls and dedicated TSLMs (Section 5).

  3. Novelty and related work (tXpi). We added vision-language methods for time series to Related Work and clarified our contribution: complementary representations as a design principle for TSLM reasoning (Section 2).

  4. Normalization controls (tXpi, 6fWN). Normalized line plots, a normalized STFT and global-range delay maps all stay at or below Plot only, separating the effect of the delay embedding from normalization (App. C.6).

  5. Invariance analysis (6fWN). We added the share of answers that change relative to clean inputs to Table 4 (Section 5.3).

  6. ChatTS reasoning (tXpi). We updated the ChatTS reasoning results and revised the corresponding claims (App. C.2, C.4).

  7. Added limitations, analysis and potential solutions (ovqG). We added an analysis of accuracy by number of variates and by series length (App. C.11), and a Limitations paragraph that discusses these limitations and potential solutions (Section 6).

  8. Use of the two views (ovqG). We added an attention analysis (App. C.8) and a view-removal analysis at inference (App. C.9).

  9. Computational cost (ovqG). We added the parameter, visual-token, latency and memory cost of the second view (App. C.10).

We are grateful for the reviewers' time and care. Their suggestions have made the paper stronger and clearer.

Thank you, The authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.