- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes TSLM-ICL, a training-free method for selecting demonstrations for time-series language models. It combines time-series token embeddings with a pooled text embedding and uses optimal transport to rank candidate examples. Experiments on five benchmarks and six frozen models show improvements over random, text-only, and adapted time-series retrieval baselines. The ablations suggest that temporal features account for most of the gains, while optimal transport provides greater benefits on multivariate tasks.
Critical strengths
-
The method provides a practical, training-free approach to demonstration selection for time-series language models. It preserves token-level temporal information and accommodates multivariate sequences of varying lengths without updating the target model.
-
The evaluation covers five benchmarks and six models with different input representations, showing overall gains over the compared retrieval baselines. The ablations help identify where these gains arise: temporal features contribute most, while token-level optimal transport provides additional benefits on multivariate tasks.
Critical weaknesses
The variance-based motivation for OT in §4.2 conflates loss of distributional geometry with loss of scalar variance. For unit-normalized tokens, Eq. (14) gives . Thus, the pooled mean before further normalization exactly determines the scalar variance, and Euclidean comparison remains sensitive to this norm information, unlike cosine similarity. Eqs. (3)–(4) quantify reconstruction residuals, but do not establish that this scalar energy is unavailable from the mean or requires token-level comparison to recover. Moreover, Eq. (7)’s premise—identical means but different scalar variances—is unattainable under these assumptions.
For example, consider uniform distributions on
\qquad B=\left\\{\frac{(1,0,1)}{\sqrt2},\frac{(1,0,-1)}{\sqrt2}\right\\}.$$ Both have mean $(1/\sqrt2,0,0)$ and scalar variance $1/2$, yet every cross-set distance equals one, giving $W_1(A,B)=1$. OT therefore distinguishes geometry beyond the mean and scalar variance.Questions for authors
-
Given Eq. (14), the pooled mean before further normalization exactly determines the scalar variance. How do the authors reconcile this with §4.2’s claim that pooling followed by Euclidean comparison discards that variance? Please revise the energy-loss interpretation of Eqs. (3)–(4) and the unattainable premise of Eq. (7), distinguishing scalar variance from distributional geometry.
-
Appendix C.3 provides validation for exact OT, but the method uses entropically regularized transport. Beyond the example in Figure 10, can the authors report the mean, upper quantiles, and maximum cross-modal transport mass across retrieved pairs at (), to substantiate the claim that cross-modal leakage is negligible in practice?
4 comments
Q2: Appendix C.3 provides validation for exact OT, but the method uses entropically regularized transport. Beyond the example in Figure 10, can the authors report the mean, upper quantiles, and maximum cross-modal transport mass across retrieved pairs at (ε = 0.03), to substantiate the claim that cross-modal leakage is negligible in practice?
Yes. Using the method's own Sinkhorn solver at ε = 0.03, we computed γ_ε for every query's top-8 retrieved demonstrations over all 39 subtasks. We report the cross-modal mass, i.e., the share of γ_ε's total mass that sits on time series↔text entries.
Cross-modal transport mass (% of total mass), top-8 retrieved pairs, ε = 0.03:
| Benchmark | Pairs | Mean | p90 | p99 | Max |
|---|---|---|---|---|---|
| TSExam | 5,968 | 0.003% | 0.013% | 0.025% | 0.51% |
| TSR-Bench | 33,000 | 0.0006% | 0.0001% | 0.021% | 0.05% |
| Time-MQA | 48,000 | 0.0002% | <0.0001% | 0.007% | 0.13% |
| MTBench | 110,184 | 0.0003% | <0.0001% | 0.009% | 0.15% |
| TimeOmni | 32,432 | 0.001% | 0.0007% | 0.025% | 0.17% |
| All | 229,584 | 0.0005% | 0.0001% | 0.020% | 0.51% |
| Random pairs (reference) | 96,000 | 0.058% | 0.035% | 1.27% | 27.2% |
Leakage on retrieved pairs is negligible. Even the worst retrieved pair moves 0.51% of the mass across modalities, and the text point keeps at least 99.5% of its mass α on the other sample's text point. Cross-modal entries contribute at most 1% of any retrieved pair's OT score (mean 0.001%). Leakage is larger only on random, dissimilar pairs (max 27%), i.e., pairs that are never retrieved, so the modality-preserving structure of Proposition 1 holds where it affects retrieval. This table and its discussion are now in Appendix C.3 (new Table), with a one-line pointer in §4.4 (TSLM-ICL), where Proposition 1 is introduced.
We thank the reviewer again for the careful and constructive review. It led us to sharpen the motivation in §4.2 around covariance rather than scalar variance, and to quantify the entropic plan's cross-modal leakage on every retrieved pair. We hope these revisions and the new results address the reviewer's concerns. The reviewer rated the paper as informative ("I learned something interesting or useful"), and we believe this reflects the potential of the work. With the soundness concerns now resolved, we kindly ask the reviewer to reconsider their assessment.
Thank you for providing the requested leakage statistics. However, the reported pair counts are inconsistent with the evaluation population specified in the paper. Tables 6–7 report 19,611 samples across 39 subtasks, so retrieving eight demonstrations for every query yields 156,888 directed query–demonstration pairs, rather than 229,584. Specifically, the expected versus reported counts are 24,000 versus 48,000 for Time-MQA, 68,640 versus 110,184 for MTBench, and 25,280 versus 32,432 for TimeOmni. Appendix B.2 also states that retrieval results are shared across all six models.
A related inconsistency already appears in Appendix C.3: it reports approximately 115,000 selected pairs at , whereas 19,611 queries would yield 78,444 pairs.
What exact query populations and counting procedures were used for these two validations? Please provide the query counts per subtask, clarify whether additional samples or repeated records were included, and reconcile the counts with Tables 6–7. If the leakage statistics were computed on a different population, please report them for the documented evaluation set.
We apologize for the confusion and thank the reviewer for engaging in the discussion and for the careful check. The difference in counts comes only from scope. The validations were run on all 39 subtasks we prepared, in addition to 8 subtasks (together 28,698 queries) which are excluded from the evaluation in the paper, as described in Appendix D:
- the forecasting and imputation subtasks of MTBench, Time-MQA and TimeOmni, where outputs are numeric sequences rather than answers that can be scored by accuracy
- Time-MQA open-ended QA, whose answers are free-form
These subtasks were omitted because at k=0 their answers are numeric sequences (forecasting, imputation) or free-form text (open-ended QA), which cannot be scored by accuracy, and because the models perform poorly on them, often producing malformed or wrong-length outputs that are truncated by token limits, unrelated to ICL.
These 8 subtasks account exactly for the differences the reviewer identified in Time-MQA, MTBench and TimeOmni, and for the ≈115k pairs at k = 4 in Appendix C.3 (28,698 × 4). For consistency with Tables 6–7, we now report both validations on the 39 evaluated subtasks.
Shared retrieval. Yes. To ensure a fair comparison, all six TSLMs receive exactly the same selected demonstrations for each query, so pairs are counted once per query, not once per model.
Cross-modal transport mass (% of total mass), top-8 retrieved pairs, ε = 0.03, 39 evaluated subtasks (19,611 queries):
| Benchmark | Pairs | Mean | p90 | p99 | Max |
|---|---|---|---|---|---|
| TSExam | 5,968 | 0.003% | 0.013% | 0.025% | 0.51% |
| TSR-Bench | 33,000 | 0.0006% | 0.0001% | 0.021% | 0.05% |
| Time-MQA | 24,000 | <0.0001% | <0.0001% | <0.0001% | 0.02% |
| MTBench | 68,640 | 0.0004% | <0.0001% | 0.016% | 0.15% |
| TimeOmni | 25,280 | 0.001% | 0.001% | 0.026% | 0.17% |
| All | 156,888 | 0.0006% | 0.0001% | 0.022% | 0.51% |
| Random pairs (reference) | 78,000 | 0.067% | 0.042% | 1.47% | 27.2% |
The conclusions are unchanged. The worst retrieved pair moves 0.51% of the mass across modalities, and cross-modal entries contribute at most 1% of any retrieved pair's OT score. In Appendix C.3, the condition now holds for all 78,444 top-4 pairs, for every top-1 pair, and for 99.99% of top-8 pairs. We will update Table [ot_leakage] and Appendix C.3 accordingly.
We thank the reviewer for the careful reading and for recognizing the practical, training-free design, which preserves token-level temporal information across variable lengths and channel counts, and the breadth of the evaluation and ablations. We address each point below.
We would like to thank again for pointing this out, this constructive feedback is essential for the improvements and the correctness of our work, below we address this concern.
We agree. For ℓ2-normalized tokens, ‖ē‖² + σ² = 1, so the unnormalized mean fully determines the scalar variance σ², and a Euclidean comparison of means is not blind to it. Pooling does lose the distributional geometry: the covariance Σ and, beyond it, the full token distribution. The given example shows this well. A and B have the same mean and the same σ² = 1/2, but different covariances (spread along e₂ vs. e₃), and W₁(A, B) = 1.
Thanks to the shared comments, we revised the first part of §4.2 around this distinction:
The main claims of Remark 2 and Proposition 1 are unchanged, and so are their proofs in the appendix. Remark 2 still states that W₁ is lower-bounded by the pooled Euclidean distance and is strictly positive whenever the two token distributions differ. Its proof only uses that W₁ is a metric on empirical distributions, so it never relied on the scalar variance. We only restated the example premise of Eq. (7) with covariances, which makes it attainable. Proposition 1 (the dual embedding space, which keeps transport mass within each modality) does not involve the variance at all.