- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes PTEF, a lightweight framework for long-term time series forecasting using pretrained text embeddings. The method converts numerical time series into text, processes them using a frozen, layer-pruned text embedding model, and combines the resulting embeddings with the original time series through residual connections. A small CNN and linear prediction head are then used for forecasting. The authors evaluate PTEF on five benchmarks under few-shot and full-shot settings, reporting competitive few-shot performance with relatively few trainable parameters. Additional experiments investigate the effects of text embeddings, layer pruning, and different model components.
Critical strengths
The paper is generally clear and easy to follow.
The proposed framework is relatively simple and achieves competitive few-shot forecasting results with a small number of trainable parameters.
The authors provide several ablation studies on text embeddings, layer pruning, and forecasting heads.
Critical weaknesses
-
Limited novelty. The main idea is highly related to LETS-C (ACL 2025), which already uses pretrained text embeddings for time series modeling. Most other components, including RevIN, channel independence, layer pruning, and CNN-based prediction, are also existing techniques. The paper does not clearly establish a substantial methodological contribution beyond adapting these ideas to forecasting.
-
Potential input truncation. The look-back window is 336, while the selected MiniLM embedding model supports only 256 tokens. Since numerical values may require multiple tokens, a substantial portion of the input could be truncated. The paper does not adequately investigate this issue or clarify how much historical information the embedding model actually receives.
-
Inconsistent parameter reporting. The claimed 0.094M trainable parameters appear to correspond to the shortest prediction horizon. According to the output head formulation, the parameter count should increase significantly for longer horizons, potentially exceeding 1.5M at horizon 720.
-
Insufficient experiments. The evaluation lacks stronger and more recent forecasting baselines, such as iTransformer and TimeMixer. The experiments are also limited to Weather and four closely related ETT datasets. Moreover, the full-shot results are not particularly competitive, with PTEF performing worse than LLM4TS and Time-LLM in average ranking.
-
Incomplete ablation studies. Although adding text embeddings improves performance, the experiments do not establish that the improvements come from pretrained language knowledge. Comparisons with randomly initialized encoders or random frozen embeddings are missing. The no-embedding baseline also changes the timestamp encoding, making the comparison less controlled.
-
Insufficient efficiency and statistical analysis. The reported fast inference time mainly reflects cached embeddings rather than complete end-to-end inference for new inputs. Direct latency comparisons with lightweight forecasting models are missing. In addition, most results rely on a fixed seed, and the limited statistical analysis does not adequately support the small improvements reported in some few-shot settings.
Questions for authors
see weaknesses
3 comments
Thank you for the additional experiments and clarifications.
The parameter counts are now clarified, and the random embedding experiment provides useful additional evidence. However, the improvements over random embeddings are relatively small in most settings, so the benefit of pretrained language representations remains limited. More importantly, the reported end-to-end inference time (0.084s) is higher than that of LLM4TS (0.024s), which weakens the claimed efficiency advantage.
Several other concerns also remain unresolved. The authors acknowledge that approximately two-thirds of the input is truncated, but do not investigate its impact on forecasting performance. The limited novelty relative to LETS-C and the lack of comparisons with recent forecasting baselines have not been adequately addressed. The additional confidence intervals are also insufficient to establish the significance of the reported improvements.
Overall, while the response addresses some technical questions, my main concerns about novelty, experimental validation, and practical efficiency remain. I therefore maintain my original score of 2 .
Thank you for the feedback, apologies for not reply to your other comments we were running additional experiments.
- We agree PTEF builds directly on LETS-C, which established that frozen text embeddings fused with raw values via addition are useful time-series representations; we do not claim the embedding-plus-fusion idea as new, and will state this explicitly. Our contribution is to test whether this transfer survives the move to long-term forecasting, and how much of the encoder is needed. Three points go beyond LETS-C:
- Forecasting, not labelling - Forecasting maps each channel to a full future sequence rather than a class label. This requires per-channel encoding, horizon-dependent output heads, and handling a 336-step window under a 256-token budget (we now report this truncation and its effect, see question 2).
- How much encoder is needed - Our layer-retention study shows that keeping 1 of 6 MiniLM layers retains most of full-encoder accuracy at 17% of the MACs validating that numerical tokens requires less processing depth. LETS-C uses complete encoders and does not study this.
- Data-scarce forecasting - With 10% training data, PTEF ranks first against LLM-based forecasters (Time-LLM, LLM4TS) and numerical baselines.
We will revise the introduction and related work to position PTEF as an empirical study of efficient text-embedding transfer for forecasting that extends LETS-C, rather than as a new representation method.
- We did run further experiments to include the few shot for TimeMixer and iTransformer, we will revise the paper accordingly and include these 2 additional baselines + random embedding experiment (question 5 table). | Dataset | PTEF (ours) (MSE/MAE) | TimeMixer (MSE/MAE) | iTransformer (MSE/MAE) | |----------|----------|----------|----------| | Weather | 0.234 / 0.269 | 0.252 / 0.283 | 0.270 / 0.302 | | ETTh1 | 0.523 / 0.493 | 0.757 / 0.602 | 0.718 / 0.578 | | ETTh2 | 0.390 / 0.416 | 0.453 / 0.449 | 0.434 / 0.447 | | ETTm1 | 0.396 / 0.405 | 0.561 / 0.496 | 0.507 / 0.474 | | ETTm2 | 0.277 / 0.325 | 0.306 / 0.347 | 0.295 / 0.343 | | Avg. Rank | 2.000 / 2.200 | 9.000 / 9.200 | 8.200 / 8.400 |
- We acknowledge that PTEF is not competitive with LLM4TS and Time-LLM in the full-shot setting, its primary claim is data efficiency in few-shot forecasting.
-
Inference time on the same hardware per batch = 128 on ETTh2, Tout = 96 including the computation of the embedding of a new input: PTEF’s latency is 0.084 s, versus 0.012 s for DLinear, 0.017 s for PatchTST, and 0.024 s for LLM4TS. PTEF is therefore slower because of text-embedding extraction.
-
We experimented with 4 seeds below the full statistical analysis and it will be included in the revised version of the paper. MSE: | Dataset | MSE Average | 95% Confidence Interval | |----------|----------:|----------| | Weather | 0.2340 | [0.2336, 0.2343] | | ETTh1 | 0.5267 | [0.5167, 0.5367] | | ETTh2 | 0.3880 | [0.3758, 0.4002] | | ETTm1 | 0.3955 | [0.3941, 0.3968] | | ETTm2 | 0.2770 | [0.2764, 0.2776] |
MAE:
| Dataset | MAE Average | 95% Confidence Interval |
|---|---|---|
| Weather | 0.2685 | [0.2680, 0.2690] |
| ETTh1 | 0.4958 | [0.4908, 0.5008] |
| ETTh2 | 0.4135 | [0.4055, 0.4216] |
| ETTm1 | 0.4055 | [0.4049, 0.4061] |
| ETTm2 | 0.3249 | [0.3245, 0.3254] |
Dear reviewer, Thank you for the feedback.
-The number of trainable parameters for the different horizons:
Due the limited sequence length of the encoder, part of the input is truncated (2/3), it's one of the limitations of these approach as it consumes a considerable amount of tokens.
Inference time per batch = 128 on ETTh2, Tout = 96: 0.084 second for the whole pipeline jo cached embeddings. LLM4TS 0.024 seconds.
Confidence Interval analysis for the ETTm1 | Horizon | MSE Avg. (95% CI) | MAE Avg. (95% CI) | |----------|----------|----------| | 96 | 0.344 [0.343, 0.344] | 0.375 [0.374, 0.376] | | 192 | 0.371 [0.369, 0.374] | 0.391 [0.389, 0.392] | | 336 | 0.404 [0.402, 0.407] | 0.410 [0.409, 0.411] | | 720 | 0.464 [0.461, 0.468] | 0.447 [0.445, 0.449] |