- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces OpenTSLM TeeMoE, a shared language backbone with independently trained LoRA experts for forecast aggregation, native contextual forecasting, and temporal analysis. A controller learns request-level mixtures of the frozen adapters and selects a numerical or language output path. The aggregation path refines a learned ensemble of 13 numerical forecasters, while the text path generates forecast trajectories or answers analytical questions. The model is evaluated on GIFT-Eval, Context is Key, and TimeSeriesExam, with controls for individual experts, adapter composition, joint training, and numerical ensembling, plus training-cost accounting.
Critical strengths
The paper evaluates a single composed checkpoint across three substantially different capabilities and addresses the main question of whether broad coverage dilutes specialist performance.
The authors performed a good amount of ablation studies such as separating the numerical ensemble from the editor to understand the gain of the proposed model.
Critical weaknesses
The joint training comparison does not seem fair. The three separate adapters together have more parameter count and hence probably more capacity than the single adapter. the size of adapters in the two settings are not matched so its hard to understand the benefit of the three separate adapters.
The evidence for a benefit from soft composition beyond selecting a specialist is weak outside Context is Key benchmarks. For the other two benchmarks, selecting the best expert seem well sufficient and the training cost would also possibly be cheaper if we only train that one adapter?
Questions for authors
in figure 1, it's a little confusing that the contextual forecasting uses the language model head since the output is numerical. why is that? the clarity can be improved.
quite a few table entries are OOT, this makes it hard to compare. there is no TSLM selected for comparison for the GIFT-EVAL task. and for table 2 GIFT-EVAL comparison, two experts have OOT. why is using all three experts ok but a single expert becomes OOT. and the only available expert (aggregate expert) performs very similar to your method in that case.
5 comments
Response to Review (Part 2 of 3)
Q1. Numerical forecasts from the language-model head
We thank the reviewer for pointing out this ambiguity. We retain language-model generation for contextual forecasting to use the LLM's ability to interpret textual information when producing forecast values. This choice is supported by CiK, where several LLMs generating forecasts as text outperform the evaluated LLM-based models with numerical forecasting heads, such as UniTime and Time-LLM [1, Section 5.3]. Our expert thus generates timestamp–value pairs as tokens, which we parse into numerical trajectories.
By contrast, for numerical forecasting, strong forecasts are already available from TSFMs so the numerical decoder lets TeeMoE refine their quantiles directly, without generating new trajectories token by token and incurring the associated cost of autoregressive generation. We ultimately preserve the prediction mechanism suited to each capability rather than force both through the same output head.
Figure 1 now labels the language-model head's outputs as “Forecasts and analytical answers,” and Section 3.3 explains this rationale and how generated timestamp–value pairs form numerical trajectories.
Q2. Missing GIFT comparisons and OOT entries
We thank the reviewer for raising this comparison gap. The selected public TSLMs were evaluated through their documented forecasting interfaces, but their measured runtimes project (far) beyond the stated 1,000 H100 GPU-hour budget per model on full GIFT-Eval (Appendix E.1). At 371,330 windows and 25 samples per window, each model must generate over nine million trajectories, which was not feasible given our compute. The same bottleneck applies to our standalone native and analysis experts, which generate forecasts autoregressively and also exceed this budget.
The full TeeMoE model avoids this cost by selecting the numerical decoder for GIFT requests and refining TSFM forecasts directly. Combining three adapters within one backbone does not require running three separate forecasting procedures, so the composed model can complete the evaluation even when the two standalone language-based experts cannot.
The aggregation expert is trained specifically for numerical forecasting so its strong GIFT performance is expected and provides the specialist reference we aim to retain. Matching that performance after composition shows that adding contextual forecasting and analysis does not sacrifice this capability. The additional experts extend what the model can do, rather than being intended necessarily to improve the aggregation expert on its own task.
To fill the missing comparisons, we have now evaluated all four selected public TSLMs and both previously OOT experts on a common GIFT subset, alongside TeeMoE and its numerical baselines. We selected one window from each of the 97 dataset/frequency/horizon categories without using prediction quality, preserving the original forecast horizons. Each language-based model generates five forecast samples per window, and numerical models use their quantile forecasts on the same windows.
| Model | Mean MASE rank ↓ | MASE ↓ | CRPS ↓ |
|---|---|---|---|
| OpenTSLM | 21.167 | 7.9393 | 1.9711 |
| TimeOmni-VL | 19.896 | 5.6514 | 1.0529 |
| TS-Reasoner | 18.734 | 1.5545 | 0.4239 |
| ChatTS | 16.276 | 0.8390 | 0.4010 |
| Native expert | 15.214 | 0.6163 | 0.1904 |
| Analysis expert | 14.620 | 0.5850 | 0.1857 |
| Aggregation expert | 6.823 | 0.5736 | 0.1062 |
| TeeMoE | 6.813 | 0.5735 | 0.1062 |
Mean MASE rank is computed within a common 22-system roster that includes the 13 constituent TSFMs, so these subset ranks are separate from the full-benchmark leaderboard ranks.
Both standalone language-based experts outperform the four external TSLMs on all three subset metrics. The analysis expert is particularly close to TeeMoE in geometric MASE (0.5850 versus 0.5735), although its CRPS and mean rank are substantially worse. TeeMoE retains the aggregation expert's performance and obtains the lowest geometric MASE and CRPS in the full subset roster. These results fill the OOT comparison gap while showing why retaining the numerical forecasting path is useful. We have added the subset comparison to Appendix E.1 and linked it from the captions of Tables 1–2.
Response to Review (Part 3 of 3)
References
[1] Williams et al. Context is Key: A Benchmark for Forecasting with Essential Textual Information. ICML, 2025. Section 5.3.
[2] Gao et al. UniTS: A Unified Multi-Task Time Series Model. NeurIPS, 2024.
[3] Guan et al. TimeOmni-1: Incentivizing Complex Reasoning with Time Series in Large Language Models. ICLR, 2026.
[4] Guan et al. TimeOmni-VL: Unified Models for Time Series Understanding and Generation. ICML, 2026.
[5] Xie et al. ChatTS: Aligning Time Series with LLMs via Synthetic Data for Enhanced Understanding and Reasoning. PVLDB, 18(8), 2025.
Response to Review (Part 1 of 3)
Weakness 1. Parameter-matched joint training
We thank the reviewer for identifying the parameter-count difference. To address it directly, we provide a shared rank-52 joint model, matching the three specialists' combined rank (4+32+16=52) over the same adapted matrices. Both settings use the same frozen backbone, training examples, number of training presentations, and output paths.
With adapter parameters matched, TeeMoE achieves 3.19% lower CiK RCRPS and 0.80 percentage points higher TSE accuracy, while joint training obtains a slightly better GIFT mean rank. TeeMoE thus remains competitive on numerical forecasting and performs better on the two language-based tasks without an advantage in total LoRA parameter count.
The comparison also retains a practical benefit: TeeMoE requires 30.756 H100 GPU-hours of training versus 58.657 for the joint model, a 47.6% reduction under these training recipes. Once trained, an individual expert can also be updated without retraining the other two. Together, these results support separate specialization as an effective way to build the generalist at matched adapter size.
We have replaced the rank-32 joint baseline in Section 5.3.2 with this parameter-matched rank-52 model and updated the joint-training appendix and training-cost table accordingly.
Weakness 2. Value of composition and specialist retention
We thank the reviewer for this observation. For a user who needs only numerical forecasting or only temporal analysis, training the corresponding expert is indeed a lower-cost option. Our GIFT and TSE results do not suggest that all three experts are necessary for either task alone. TeeMoE addresses the broader need for one model that performs strongly across numerical forecasting, contextual forecasting, and temporal analysis. Covering complementary time-series capabilities also motivates UniTS [2], TimeOmni-1 [3], TimeOmni-VL [4], and ChatTS [5]. For example, a demand-planning workflow may require a baseline forecast, a forecast conditioned on an announced promotion, and analysis of recent trends. TeeMoE's top-three results across the three benchmarks, together with the specialist comparisons, show that this broader coverage can be achieved while retaining the experts' performance.
Within that generalist, the reviewer is right that top-1 selection already preserves most of the GIFT and TSE performance. We do not claim a substantial soft-mixing gain on those two benchmarks. MoE lets us develop capabilities with different training requirements separately and combine them in a shared backbone; however, the contribution is the resulting generalist model and training recipe, rather than a new MoE framework whose principal claim is additional gains from expert interaction. We therefore believe that preserving strong specialist performance across the combined capabilities is valuable even where mixing does not improve each specialist further.
CiK indeed shows a clear additional benefit: soft composition reduces RCRPS from 0.12296 to 0.11479, a 6.65% reduction. To assess the consistency of this gain, we performed a paired bootstrap analysis of the existing predictions for all 355 instances across 71 task types. Each resample draws task types with replacement, keeping their five paired instances together and retaining the benchmark weights. Across 100,000 resamples, the 95% interval for the absolute reduction is [0.00211, 0.01482], entirely above zero. The improvement also remains positive when any single task type is omitted.
We have revised Section 5.3.2 to distinguish these findings: strong performance and specialist retention across all three capabilities, with additional predictive gains from composition most clearly supported on CiK. The paired analysis is included in the appendix. We appreciate the suggestion to make the value of the generalist and the more specific benefit of soft mixing clearer.