- Rating
- 2
- Confidence
- 3
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
LOTUS proposes a time-series language model based on “forecast grounding.” During Phase 1, the model maps time-series patches into LLM soft tokens and uses next-patch prediction from LLM hidden states to encourage the representations to preserve temporal information. It also combines raw temporal patches with FFT-based spectral features. The pretrained model is then adapted to downstream time-series classification and QA tasks using language supervision, and is evaluated across several backbones and benchmarks.
Critical strengths
- The paper is clearly written and easy to follow. The motivation, architecture, and two-stage training pipeline are presented clearly.
- The authors provide several diagnostic analyses and ablations, although some of them do not fully isolate the claimed contribution.
Critical weaknesses
-
Limited novelty.
The main idea is close to aLLM4TS, which already uses next-patch prediction for LLM-based time-series representation learning. LOTUS mainly adds time-series–language alignment and a standard FFT-based temporal–spectral fusion, so the method feels largely compositional. -
The forecast-grounding ablation is confounded.
The full model scores 55.19, while the language-only Phase 1 variant scores 51.79. However, this comparison changes both the forecasting objective and the use of Monash/synthetic forecasting data, so it does not clearly show whether the gain comes from next-patch prediction itself or simply from additional numerical pretraining data. -
Potential future-information leakage.
For multivariate inputs, channels are provided as separate blocks. If complete channel sequences are serialized sequentially, causal attention over token order may still expose later timestamps from earlier channels. The forecasting prompt also includes mean/std computed from the full series, which may contain future information. -
Possible answer leakage in ECG-QA.
In one appendix example, the clinical context already says “burst noise present,” while the question asks “Does this ECG show burst noise?”. This suggests that some samples may be answerable from textual metadata without reading the ECG signal. -
Baseline coverage is relatively weak.
The main matched comparisons are mostly against text-tokenized and OpenTSLM variants. Several more recent time-series language or multimodal methods are discussed but not directly compared, making it difficult to assess competitiveness against stronger recent baselines. -
Some claims are overstated and robustness is limited.
On TSRBench, LOTUS-270M scores 38.5 versus 38.2 for TS-Reasoner-7B, so the margin is very small despite the “25–50× larger” claim. Key ablations are also mainly done on Gemma3-270M, and ECG-QA uses only a fixed 40k subset out of about 159k training samples.
Questions for authors
-
Beyond combining aLLM4TS-style next-patch prediction with FFT features, what is the main methodological novelty of LOTUS?
-
Why are stronger and more recent time-series language or multimodal baselines not included in the main comparison?
-
How do the authors avoid future-information leakage in multivariate next-patch prediction, especially with channel-wise serialization and full-window statistics?
-
Some ECG-QA prompts appear to contain the answer in the textual metadata. How well would a text-only LLM perform without using the ECG signal?
10 comments
Response to Reviewer H4Wp (2/4)
Additional same-budget study. We also compared four Phase-1 settings under an identical budget. Every pretrained arm receives 6,000 Phase-1 updates, and all arms are then fine-tuned on each task with the same recipe and evaluated on the Table 1 test sets (Gemma3-270M, univariate forecasting corpora, one seed). The three-task mean is distinct from the six-split average in Table 3.
| Phase-1 objective | HAR | Sleep-EDF | ECG-QA | Mean |
|---|---|---|---|---|
| None | 67.84 | 64.41 | 13.56 | 48.60 |
| Language only | 69.76 | 62.04 | 20.38 | 50.73 |
| Forecasting only | 64.87 | 67.74 | 27.94 | 53.52 |
| Language + forecasting | 70.70 | 66.56 | 19.78 | 52.35 |
| Comparative Evaluation | Performance Gain (pp) | 95% Confidence Interval (over test items) |
|---|---|---|
| Language + forecasting vs. language only | +1.62 | [0.48, 2.76] |
| Forecasting only vs. none | +4.92 | [3.85, 5.99] |
| Forecasting only vs. language only | +2.79 | [1.76, 3.82] |
Adding forecasting to language pretraining improves the three-task mean at the same update count, so this gain cannot be attributed to additional optimization steps. Forecasting-only pretraining achieves the highest mean and improves over both no pretraining and language-only pretraining. Numerical predictive supervision therefore transfers to downstream language tasks even without Phase-1 language supervision. All three intervals exclude zero.
Taken together, the two controls show that the benefit of forecast grounding is explained neither by a larger optimization budget nor by the additional numerical corpora.
W3/ Q3: Potential future-information leakage
We would like to provide clarifications regarding the channel ordering and normalization below.
-
Channel ordering. Separate blocks per channel are used in downstream fine-tuning, where the forecasting objective is absent. During Phase-1 pretraining, a multivariate series occupies one block with patches interleaved by time: all channels of patch 1, followed by all channels of patch 2, and so on. With patch-wise encoding and causal attention in the LLM and next-patch decoder, the state predicting patch n+1 cannot attend directly to patch n+1 or later patch tokens from any channel. We verified this ordering in the dataset code and through perturbation checks across all three backbones. We will revise Figure 12 correspondingly to the above description.
-
Mean and standard deviation. Our normalization follows prior autoregressive time-series pretraining: LLM4TS [1] applies instance normalization to the whole input sample before its autoregressive next-patch alignment, so its statistics also cover the target patches. LOTUS likewise computes per-channel statistics over the observed input window, rather than the complete underlying series. We acknowledge that these statistics aggregate information from later target patches, although causal attention prevents direct access to their tokens. Our next-patch objective is auxiliary supervision for representation learning, not an evaluation of causal forecasting. The forecasting decoder is removed before adaptation, and all downstream classification and question-answering tasks operate on fully observed signals (Section 3.3). We will clarify this distinction in the revision.
Response to Reviewer H4Wp (3/4)
W4 / Q4: ECG-QA metadata and performance without the signal.
We note that the cited example does not demonstrate answer leakage, as the correct answer could contradict what is given in the context statement. For the record of the appendix example: its correct answer is “no,” although the context says “burst noise present.” Copying that statement would therefore give the wrong answer. The phrase is boilerplate from the public OpenTSLM prompt builder shared by the compared methods. Our audit found it to be identical in all 159,313 training prompts and all 5,000 evaluated test prompts. It therefore cannot distinguish positive from negative recordings. We double check the label counts, and the results we have confirm this:
| Yes/no test questions | Count | Context rule | Always "no" |
|---|---|---|---|
| Burst noise | 851 | 32.8% | 67.2% |
| All six context-related conditions | 3,042 | 32.3% | 67.7% |
Of the 851 burst-noise questions, only 279 have the answer “yes.” Across all 3,042 questions concerning the conditions mentioned in the context, copying its affirmative statements performs substantially worse than always answering “no.” The highlighted boilerplate therefore does not provide a recording-specific answer cue.
To further study the effect of the context, as well as the ECG itself, we additionally run a separate input-ablation and report the results below:
| Test-time input | ECG | Clinical context | Question | Accuracy (%) | Change from full input (points) |
|---|---|---|---|---|---|
| Full input | 33.16 | ||||
| Without clinical context | 30.06 | −3.1 | |||
| Without ECG | 19.56 | −13.6 | |||
| Question only | 19.76 | −13.4 |
Signal dependence. Removing the ECG reduces accuracy by 13.6%, while removing the clinical context reduces it by 3.%. Without the ECG, clinical context plus question achieves 19.56%, compared with 19.76% for the question alone. These controls show that the trained checkpoint uses the ECG signal and that the highlighted boilerplate does not explain its performance.
W5 & Q2: "Baseline coverage is weak"
We would like to defend our choice of baseline here. First, in terms of framework comparison, we compared our with the direct competitor OpenTSLM variants [2] and to our knowledge, one of the most recent baselines ([2] was accepted in ICML 2026). Second, in terms of backbones, we compared against multimodality models like Gemma-3-4B-pt (image) [3] and large time-series reasoning models like TS-Reasoner [4] (accepted in TMLR 2026), TimeOmni-1 [5] (accepted in ICLR 2026), Chat-TS [6] (accepted in VLDB 2025). We reckon that, the baselines included for comparison is comprehensive and up-to-date. However, we respect the reviewer's opinion, and we run additional experiments to include GEM [7] (accepted in Neurips 2025) for comparison. GEM is a multimodal ECG language model, and we compare our 0.6B model against 2 versions of it: zero-shot and fine-tuned. The results are shown below.
| Model | HAR | Sleep-EDF | ECG-QA | PAMAP2 | MCQ | T/F | Avg. |
|---|---|---|---|---|---|---|---|
| GEM-7B, zero-shot | 3.53 | 17.53 | 34.64 | 3.85 | 34.33 | 31.91 | 20.96 |
| GEM-7B, fine-tuned | 43.86 | 61.94 | 38.40 | 49.83 | 48.42 | 70.85 | 52.22 |
| LOTUS, Qwen3-0.6B | 63.66 | 77.20 | 31.58 | 53.42 | 56.72 | 76.90 | 59.91 |
As the results show, LOTUS-0.6B is ahead on every split over GEM-7B, except ECG-QA. We found out that one of the main reason GEM-7B leads on ECG-QA because it is an ECG specialist: its instruction-tuning data (ECGInstruct) already contains about 139K ECG-QA samples, so it has been trained on this task specifically before our fine-tuning. Its zero-shot row shows this directly. Without any training on our data, GEM-7B reaches 34.64% on ECG-QA, while staying near or below chance on HAR, Sleep-EDF and PAMAP2. Again, we want to note that the size gap of our model (LOTUS-0.6B) and GEM-7B are nearly 12 times, which further show the effectiveness and efficiency of our method.
Response to Reviewer H4Wp (1/4)
We thank the reviewer for the critical comments. We address each of the concerns below
W1 / Q1: "Limited novelty"
We acknowledge causal next-patch prediction through an LLM is shared with aLLM4TS, as discussed in Section 2.2, and we do not claim that objective as new.** We kindly note that LOTUS differs from existing time-series understanding models in two key aspects: (1) forecast grounding, which explicitly encourages language-aligned representations to preserve future-relevant temporal information, and (2) a temporal–spectral patch representation that jointly incorporates raw time-domain values and rFFT features before entering the LLM. Specifically, next-patch prediction supervises the post-LLM hidden states of soft time-series tokens, alongside language supervision. The forecasting decoder is subsequently removed, allowing us to evaluate whether the learned temporal representations transfer to downstream language tasks. For the reviewer’s clarity, we list the differences below.**
Language interface. aLLM4TS takes numerical patches as input and decodes numerical patches. It is evaluated on numerical tasks such as forecasting, anomaly detection and classification, and it produces no language output. LOTUS interleaves time-series tokens with text in one LLM and is evaluated on question answering with written rationales.
New finding. aLLM4TS shows that next-patch pretraining yields transferable numerical representations. LOTUS shows that the same supervision transfers to language understanding. In our same-budget study, forecasting-only pretraining with no language supervision raises the three-task mean by 4.9 points over no pretraining and by 2.8 points over language-only pretraining (tables in our response to Reviewer 9Lhr).
Joint context. LOTUS contextualizes channels, temporal patches and text together within one LLM, whereas aLLM4TS processes channels independently.
Evidence beyond FFT. Every arm in our new controls keeps the temporal-spectral encoder. With data, batches and decoder fixed, the next-patch target is ahead of a current-patch target in all three seeds, by 2.1 points on average, so the gain is not explained by the FFT branch or by the additional corpora.
W2: "The forecast-grounding ablation is confounded."
We acknowledge that the original comparison changes the objective and the data together, because the Monash and synthetic corpora contain no text and enter training only through the forecasting loss.
Data-matched control. We ran a three-arm study to separate the forecasting objective from the forecasting data. The first two arms share the same Phase-1 corpora, batches and decoder, and differ only in the decoder target: the decoder either predicts the next patch (LOTUS) or reconstructs the current patch (control). The third arm uses language-only pretraining. Each arm is pretrained with three seeds, and every pretrained model is then fine-tuned on Sleep-EDF alone with a fixed budget and no model selection (Gemma3-270M, full test set).