acceptodds
Withdrawn from ICLR 2027

Context in Time Series: Benchmarking and Aligning Time-Series Language Models

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Existing benchmarks for time-series language models can often be solved using either text or time-series alone, making it difficult to assess genuine cross-modal reasoning. In this paper, we introduce the Context in Time-Series Benchmark (CiT-Bench), which comprises forecasting, imputation, and anomaly detection tasks designed such that textual context is necessary for accurate prediction. The text states a future event or constraint, describes an event inside a missing block, or singles out the true anomaly among distractors. To facilitate model development on CiT-Bench, we additionally construct a language–time-series alignment dataset for training. Using this dataset, we show that improved understanding of both textual and time-series inputs can be achieved by training only the embeddings of a small vocabulary of time-series words, without fully fine-tuning the language model. We provide a unified framework for evaluating and improving cross-modal reasoning between language and time-series. Our benchmark is available at bluehttps://huggingface.co/anonymized.

Primary area:
datasets and benchmarks
Keywords:
timeseries, large language models, benchmark, context

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_mDvzOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper intorduces Context in Time-series benchmark, bridging the gap of lack of context in time sereis benchmark in anomaly detection and imputation tasks. In addition, the authors construct a langugage-time-series alignment dataset for training to improve the understanding of both textual and time-series inputs.

Critical strengths
  1. This paper extend Context in Time-series benchmark to include anomaly detection and imputation tasks.

  2. The authors construct a langugage-time-series alignment dataset for training to improve the understanding of both textual and time-series inputs.

  3. The authors evaluate multiple general-purpose LLM and time-series language models on the proposed benchmark.

Critical weaknesses
  1. The dataset cannot be accessed in the provided huggingface link. Without the documented dataset, the experimental results are not reproducible.

  2. The benchmark are synthetic, and I am concerned that the LLM models achieving good performance on this synthetic dataset is still robust on the real-world dataset. The authors could evaluate on a small real-world dataset or run some robustness test, such as under noisy or event condition, which will strength the generalizability.

  3. In the paper, the authors cliam that CiT-Bench evaluates models' ability to integrate textual context with numerical time-series observations. In the experiment, the authors should evaluate the same LLMs under three input settings: (1) time-series only, (2) textual context only, and (3) both time series and textual context (which is done in the experiment). This aims to verify whether the model can achieve the similar performance only relying on a single modality. If so, the motivation of pairing text with time series becomes weaker.

  4. In table 2, the number of queries (400) is larger than number of scenes (100 or 200) for three types of tasks. In the paper, the authors do not provide any details about the data split. I am somehow concerned about the potential data leakage if the training data and test data share the same scene and time series, especially given that the number of queries exceeds the number of scenes.

Questions for authors

Q1. Could the authors clarify whether all queries associated with the same scene are kept together whenever training, validation, or test partitions are constructed?

Q2: Could the authors provide the text-only and time-series-only evaluation to verify the necessity of integrating two modalities?

Reviewer_UfiNOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

The paper introduces CiT-Bench to evaluate whether language models can combine textual context with time-series across forecasting, imputation, and anomaly detection tasks. It also constructs CiT-Corpus, containing 3.4 million training examples, and proposes NeoTS, which learns numerical input embeddings while keeping the language-model backbone frozen. Evaluations compare ten models and show that existing time-series language models often lag behind general-purpose LLMs under the proposed benchmark. NeoTS improves its backbone on all three CiT-Bench tasks.

Critical strengths

The paper extends context-dependent evaluation beyond forecasting to imputation and anomaly detection, giving a broader picture of how models use textual context. It also contributes a large training corpus with automatically computed answers. NeoTS is straightforward and requires training only the new numerical embeddings; its improvements on both CiT-Bench and Context is Key demonstrate the potential of this approach. The overall presentation is also good.

Critical weaknesses

My main concern is that Qwen3.8-27B + NeoTS shows only modest improvements over the original Qwen3.8-27B on the proposed CiT-Bench. Also, the performance is not as good as Gemma4-31B and Gemini 3.1 Flash-Lite. I am unsure whether this concern can be addressed within the rebuttal period. If the authors can provide stronger results, I would be willing to recommend weak acceptance.

Questions for authors

Could the authors provide more results, including 1) the performance on general time-series benchmarks comparing with existing baselines and 2) the performance on task-specific benchmarks comparing with existing baselines to better demonstrate the advantages of the proposed method?

Reviewer_DVZBOpenReview ↗
Rating
2
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper introduces CiT-Bench, a benchmark for context-aware time-series anomaly detection, forecasting, and imputation. Its defining feature is that accurate predictions require an understanding of the accompanying textual context. The paper also introduces CiT-Corpus, a multiscale dataset pairing time series with textual descriptions, and proposes NeoTS, a new fine-tuning method.

Critical strengths

CiT-Corpus provides valuable training data for context-aware time-series reasoning. Understanding time series alongside their associated textual context is an important capability, and the availability of suitable training data is essential for developing it. The dataset’s scale, comprising 3.4 million examples, is a notable strength.

Critical weaknesses

The paper’s main limitation is its exclusive reliance on synthetic data. Including real-world data, potentially drawn from published studies or publicly available sources, would strengthen the evaluation. Synthetic data alone may not capture the complexity of real-world settings, and the paper does not sufficiently demonstrate how the benchmark relates to practical applications, such as ECG analysis or financial time-series modeling.

Another weakness is the limited description and evaluation of NeoTS. Figure 5 suggests relatively modest differences between Qwen3.8-27B with NeoTS and the base Qwen3.8-27B model: 37.25 versus 39.5 for context-in anomaly detection, 0.113 versus 0.119 for context-in imputation, and 0.093 versus 0.098 for context-in forecasting. These results warrant a more detailed discussion of the method’s practical benefits.

Furthermore, NeoTS is evaluated using only one backbone, leaving its generalizability across model families unclear. The paper also lacks ablation studies that would establish which components of NeoTS contribute most to its performance.

Questions for authors

Given the limited description of NeoTS and its evaluation using only the Qwen3.8-27B backbone, it is unclear whether the method generalizes to other LLM architectures. Could the authors evaluate NeoTS across multiple backbone model families?

Could the authors provide component-level ablation studies to clarify which elements of NeoTS account for the observed performance gains?

What motivated the selection of GPT-4o mini, Claude Haiku 4.5, and Gemini 3.1 Flash-Lite as the closed-source baselines? Given that these models prioritize cost efficiency, including more capable models would provide a stronger comparison.

Would evaluating current state-of-the-art models change the conclusions drawn about NeoTS in Figure 5? Can authors describe in more detail, how proposed benchmark relates to the real-world scenarios?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.