- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
-
Instead of adapting LLMs to time series through task-specific fine-tuning, the authors explore mid-training, an intermediate step between pre-training and post-training, to give LLMs a more general understanding of time series.
-
They take pretrained Qwen 3 models and train them on Book-of-Thoth, a dataset containing time series paired with natural language descriptions. The training focuses on both describing time series in text and generating time series from text.
-
Mid-training substantially improves time-series understanding. For example, trend recognition accuracy improves for the 8B model, although performance gains vary across tasks.
-
When fine-tuned on some of downstream training data, the mid-trained models generally perform better than directly fine-tuning the original LLMs, suggesting that mid-training helps models learn new time-series tasks with less data.
Critical strengths
S1. This work explores mid-training as a middle ground between general-purpose pre-training and task-specific fine-tuning, allowing LLMs to develop a broader understanding of time series before adapting to specific downstream tasks.
S2. The authors introduce Book-of-Thoth, a large-scale dataset of time-series–text pairs that supports learning in both directions (time series to text and text to time series). The automated data generation process also makes the approach relatively scalable.
S3. The experimental results show substantial improvements over the original Qwen3 models, particularly in recognizing temporal patterns. The evaluation across multiple benchmarks also helps demonstrate the usefulness of mid-training beyond a single task.
S4. The downstream fine-tuning experiments are particularly interesting, as Thoth generally outperforms the original Qwen3 model when both are fine-tuned using only five percent of the available training data. This provides further evidence that mid-training can help models adapt to downstream tasks with limited labeled data.
Critical weaknesses
W1. While the paper shows clear improvements over the original Qwen3 models, it would benefit from a more direct comparison with other training approaches under similar computational budgets. Without this, it is difficult to tell how much of the improvement comes specifically from mid-training rather than simply training the models on additional time-series data.
W2. Although the results are promising, the improvements are not consistent across all tasks, particularly classification. Additionally, the proposed KnoTS benchmark is relatively small and relies on LLM-based evaluation without reporting agreement with human evaluators. Further experiments on more diverse real-world datasets, along with human validation of the evaluation scores, would help strengthen the paper's broader claims about general-purpose time-series understanding.
Questions for authors
Q1. Could the authors provide a comparison between mid-training and direct task-specific fine-tuning under similar computational budgets? This would help clarify how much of the improvement comes specifically from mid-training rather than additional exposure to time-series data.
Q2. Since Book-of-Thoth relies heavily on synthetic time-series data, could the authors comment on how well the learned capabilities generalize to temporal patterns that differ from those seen during mid-training? Although the paper includes real-world evaluations, further out-of-distribution experiments would help support the broader generalization claims.
Q3. Given that KnoTS relies on LLM-based evaluation, have the authors considered comparing the evaluation scores against human judgments? This would help establish how reliable the reported improvements in reasoning and decision-making are.
Q4. Could the authors provide some insight into why the improvements are less consistent across certain downstream tasks, particularly classification? It would be interesting to understand whether this reflects limitations in the training data or the mid-training approach itself.
Q5. For the KnoTS benchmark, have the authors considered evaluating how much the model's answers depend on the actual time-series input? For example, removing or changing the time series while keeping the question and domain context fixed could help establish whether the model is genuinely reasoning over temporal information.