acceptodds
Under review as a conference paper at ICLR 2027

Instance-Adaptive Prompts as Context for Time-Series Foundation Models

Abstract

Longer histories can improve time-series foundation models (TSFMs), but require substantially higher inference cost. We therefore ask whether contextual information can be provided more efficiently through a compact set of learned token embeddings. We introduce PaCTS, which generates a small set of instance-adaptive latent prompts in the form of continuous embedding tokens conditioned on the visible context. These prompts serve as compact context surrogates for frozen TSFMs. PaCTS constructs them from instance-specific global statistics and further refines them with segment-level temporal information, capturing both global characteristics and local temporal variations. The prompt module is jointly trained and deployed across heterogeneous time series with the frozen backbone. Extensive experiments demonstrate the effectiveness of prompts as context, consistently improving forecasting across context lengths and model architectures. With a shorter input context, PaCTS can outperform the same frozen backbone using double context while requiring substantially less inference computation. Compared with weight-space adaptation methods, PaCTS achieves stronger improvements and better out-of-distribution generalization.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-series foundation model, post-training, prompt tuning, parameter-efficient adapation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_UbYbOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper investigates whether a small number of input prompt tokens can improve frozen time-series foundation models. PaCTS combines shared learnable prompts with low-rank, instance-adaptive prompts generated from input statistics, and refines them through cross-attention over segment statistics with relative positional information. The prompts are prepended to the original patch tokens. The model continues to process the entire visible history; it does not encode earlier observations excluded from the context window. A separate prompt module is trained for each backbone and context-length setting and shared across datasets. The experiments post-train on the training splits of GIFT-Eval, evaluate its 97 test configurations, and transfer to TIME, which the manuscript states has no dataset overlap with GIFT-Eval. At a context length of 4,096, normalized MASE/CRPS for Chronos-2 improve from 0.708/0.496 to 0.697/0.480; PaCTS also outperforms the zero-shot backbone with a context length of 8,192. The paper includes results for additional backbones, component ablations, multivariate inference, and inference costs. The GIFT-Eval results evaluate post-training on temporally held-out portions of the same datasets, whereas TIME tests transfer of the prompt module across datasets.

Critical strengths
  1. Practical improvements in accuracy and inference cost. Tables 1–2 (pp. 6–7) show improvements in both point and probabilistic forecasting while adding only 153,286 parameters to Chronos-2, approximately 0.128% of its parameter count. Compared with the zero-shot model at a context length of 8,192, PaCTS at 4,096 reduces FLOPs, peak GPU memory, and latency by approximately 49.3%, 18.4%, and 36.8%, respectively. The comparison and measurement conditions are sufficiently clear to motivate PaCTS as a lightweight post-training approach.
  2. The evaluation extends beyond a single backbone or aggregate score. The incremental ablations in Table 3 show successive benefits from static prompts, adaptive generation, and segment refinement. Results on two additional backbones and across history lengths and forecast horizons provide evidence about applicability. Transfer to TIME without further tuning and multivariate inference after univariate training are also valuable. The explicit distinction between datasets used for adaptation and datasets used to evaluate transfer makes the results easier to interpret.
Critical weaknesses
  1. Limited novelty and insufficient isolation of the distinctive contribution. The ten descriptors in Table 5 (p. 15) are conventional. Time-LLM uses statistics in textual prompts, Gen-P-Tuning adapts frozen time-series backbones through input-dependent prompts, and CoSPOT constructs basis prompts from spectra. Although backbones and tasks differ, PaCTS’ potential advance lies in combining global low-rank prompts, ordered segment refinement, and shared training across datasets. Table 3 adds information and capacity together, leaving the contributions of basis composition and cross-attention unresolved. A simple encoder using identical global and ordered segment statistics, with matched prompt-token and parameter counts, is missing. Appendix C also uses different training schedules and batch sizes without establishing comparable data exposure or tuning budgets. Given the MASE margin of 0.697 versus 0.703 in Table 1, the gains cannot be clearly attributed to the architecture rather than statistical conditioning or optimization protocols.
  2. Uncertain limits of statistical conditioning and transfer gains. Histories with different forecasting-relevant dynamics can share identical conditioning descriptors, including global and ordered segment statistics, positions, and masks, and therefore receive identical prompts. Forecasts can still differ because the backbone receives the original patches, but the paper does not test for negative transfer when the descriptors miss relevant dynamics. TIME MASE/CRPS improve only from 0.664/0.558 to 0.661/0.555 (Table 7), without seed dispersion or paired dataset-level uncertainty. Frequencies and horizons sharing a dataset are not independent observations. Claims about replacing longer contexts should remain limited to the evaluated accuracy–cost trade-off: prompts cannot access additional history, and Appendix Tables 13–14 include slightly worse short-context CRPS, such as 0.484 versus 0.481 for PatchTST-FM.
Questions for authors
  1. Suggested comparison. What gains do the low-rank basis and cross-attention provide over a simple prompt generator using identical global and ordered segment statistics, with matched token counts, parameters, training data, and budgets? Please provide existing controls or add this comparison.
  2. Protocol clarification. Do PaCTS, LoRA, and full fine-tuning match training records, task and horizon sampling, loss normalization, and data exposure? Please report training costs and validation search ranges, and clarify whether the advantage persists with matched budgets and sampling.
  3. Existing results. Please report all three seeds, dataset-level paired error changes, win/loss rates, and uncertainty on TIME/GIFT-Eval, accounting for dependencies among frequencies and horizons within datasets. Are the small average TIME gains stable and broadly distributed?
4 comments
AuthorsOpenReview ↗

Response to Reviewer UbYb (1/3)

We thank reviewer UbYb for the insightful comments. In the following we address the concerns and provide answers to the questions.

Novelty and positioning (Weakness 1)

We agree that existing methods also explore input-space latent prompting. We would like to further clarify our contributions and differences from existing methods in three aspects.

Research target. PaCTS aims to improve a frozen TSFM effectively and efficiently with compact context derived from its own available history, rather than with additional information from other channels (Gen-P-Tuning) or from vision and text (UniCast, CoSPOT). Time-LLM uses input statistics as a text prefix for an LLM rather than constructing latent prompts for a frozen TSFM in our method. To the best of our knowledge, PaCTS is the first method that treats soft prompts as a summary instruction and compact context to improve frozen TSFMs in general.

Training across heterogeneous datasets. To improve frozen TSFMs in general, we train and evaluate a single prompt module jointly on heterogeneous datasets, without dataset-specific adaptation. One module thus serves all datasets without per-dataset training or storage, matches the general-purpose use of TSFMs, and transfers to datasets unseen in training (TIME, see below). In contrast, Gen-P-Tuning, UniCast and CoSPOT train a separate model for each dataset or task.

Architecture design. To make instance-adaptive prompts work across diverse datasets with different context lengths and forecast horizons, we design a statistics-conditioned low-rank prompt generator that explicitly extracts high-level information from the available history, and a cross-attention-based segment refinement that adds finer statistics of ordered segments for long inputs. The same-information control below further shows that this design makes better use of the statistics than a generic generator.

We will add these contribution clarifications in the main paper.

Prompt module architecture ablations (Weakness 1 and Question 1)

To demonstrate the effectiveness of the proposed prompt generation architecture, we conduct experiments by replacing the low-rank generator and the segment cross-attention of PaCTS with a simple MLP generator that receives the same global and ordered-segment statistics, with a matched number of parameters and the same training settings. As shown in the following table (over three seeds), the statistics alone already improve over zero-shot, and the PaCTS architecture adds a further gain on top of the same information and parameter budget. Together with Table 3, the results show that each of the three components of the prompt generator contributes: the shared prompts, the instance-adaptive generation and the ordered-segment refinement, whose segment information helps even with a generic MLP. The low-rank basis and cross-attention design then makes better use of the statistical information than a generic MLP. We will add these new ablations to Table 3.

Method (Chronos-2, 4,096)Trainable parametersMASECRPSΔ MASEΔ CRPS
Zero-shot00.7080.496——
Simple generator (same statistics)152,9790.701 ± 0.0020.484 ± 0.002−0.99%−2.42%
PaCTS153,2860.697 ± 0.0010.480 ± 0.001−1.55%−3.23%
AuthorsOpenReview ↗

Response to Reviewer UbYb (2/3)

Training protocol of the baselines (Weakness 1 and Question 2)

All methods are trained on the same GIFT-Eval training data and train / validation / test splits. In Table 1, full fine-tuning and LoRA follow the official Chronos-2 fine-tuning recipe, which we adopt to avoid reimplementation errors and to use the configuration recommended by the backbone authors.

To further match the protocols, we additionally train full fine-tuning and LoRA within the PaCTS pipeline, matching the training records, task and horizon sampling, loss normalization and data exposure. The settings and the checkpoints are selected by validation loss. As shown in the following table, both baselines perform similarly to their official-recipe results and remain behind PaCTS. Under this matched training budget, training on one GPU takes 4.7 h for PaCTS, 5.6 h for full fine-tuning and 4.7 h for LoRA, with a peak GPU memory of 32.5 GiB, 48.9 GiB and 40.2 GiB, respectively. We also train both baselines within the PaCTS pipeline on TimesFM-2.5 and PatchTST-FM r1 (see our response to Reviewer qCro), where PaCTS also outperforms them.

We will add these results and discussions in the paper.

Chronos-2 (mean ± std over 3 seeds)4,096 MASE4,096 CRPS8,192 MASE8,192 CRPS
Zero-shot0.7080.4960.7040.491
Full FT, official recipe (Table 1)0.703 ± 0.0000.487 ± 0.0000.698 ± 0.0010.482 ± 0.001
LoRA, official recipe (Table 1)0.703 ± 0.0000.493 ± 0.0000.699 ± 0.0000.488 ± 0.000
Full FT, PaCTS pipeline0.703 ± 0.0000.494 ± 0.0020.699 ± 0.0010.484 ± 0.001
LoRA, PaCTS pipeline0.703 ± 0.0010.490 ± 0.0020.698 ± 0.0010.485 ± 0.002
PaCTS0.697 ± 0.0010.480 ± 0.0010.695 ± 0.0010.478 ± 0.001

Identical descriptors and negative transfer (Weakness 2 and Question 3)

We understand the reviewer's concern that time series with the same descriptors receive the same prompt. The prompt, however, is an additive, gated input and the backbone still processes the raw patches, so their forecasts differ. Moreover, in practice, identical descriptors for different histories are rare: among the 371,330 GIFT-Eval test windows, only 1,305 (0.35%) share their descriptors with a window of a different history, all from two datasets (bitbrains_fast_storage and car_parts) and mostly all-zero or sparse intermittent series, while none of the 111,177 TIME test windows shares the same descriptors. Even on bitbrains_fast_storage, PaCTS still improves over zero-shot Chronos-2 (−1.11% MASE and −6.40% CRPS in the table below).

To examine negative transfer, we report the change of each GIFT-Eval dataset relative to zero-shot Chronos-2, where the 97 configurations are grouped into 23 source datasets (frequency variants and subsets of the same data, e.g. the M4 subsets, are grouped). Each cell is the change relative to zero-shot Chronos-2, as a geometric mean over the configurations of the source dataset and averaged over three seeds.

PaCTS improves both MASE and CRPS on 16 of the 23 datasets, compared with 12 for full fine-tuning and 13 for LoRA. We do observe negative transfer on a few datasets: MASE increases by more than 1% on bizitobs_application, bizitobs_service and car_parts (a single configuration), and full fine-tuning and LoRA also degrade on bizitobs_service and car_parts. These datasets differ from the rest of the training data: the two bizitobs datasets are the only GIFT-Eval datasets sampled every 10 seconds, so their daily period (8,640 steps) exceeds the 4,096-step context, and car_parts consists of sparse intermittent demand series. We will add this discussion of negative transfer to the paper.

GIFT-Eval datasetPaCTS Δ MASE (%)PaCTS Δ CRPS (%)
covid_deaths−16.76−30.80
bizitobs_l2c−10.18−10.60
us_births−6.87−4.63
m4−4.72−4.83
loop_seattle−2.68−2.46
m_dense−1.98−2.73
solar−1.72−1.28
bitbrains_fast_storage−1.11−6.40
sz_taxi−0.93−1.34
kdd_cup_2018−0.82−0.57
restaurant−0.76−0.90
hospital−0.49−4.46
ett1−0.41−0.24
bitbrains_rnd−0.21−34.46
jena_weather−0.19−0.27
temperature_rain−0.11−0.09
hierarchical_sales+0.01−0.31
electricity+0.09−0.73
ett2+0.18+0.38
saugeen+0.33+0.29
bizitobs_service+2.89+3.61
car_parts+5.04+4.66
bizitobs_application+10.04+21.06
Datasets improved16 of 2318 of 23
AuthorsOpenReview ↗

Response to Reviewer UbYb (3/3)

Uncertainty and dataset-level results on TIME (Weakness 2 and Question 3)

To show that the gains of PaCTS on TIME are stable and broadly distributed, we report the standard deviation over seeds, dataset-level changes and win/loss rates.

As shown in the following table, full fine-tuning and LoRA underperform zero-shot on TIME, whereas PaCTS improves over it by more than the standard deviation over three seeds. Each of the three seeds of PaCTS improves over zero-shot on both metrics on both benchmarks.

At the dataset level (changes relative to zero-shot Chronos-2, geometric mean over the configurations of each dataset, averaged over three seeds; the 98 TIME tasks are grouped into 37 source datasets in the same way), PaCTS improves MASE and CRPS each on 21 of the 37 datasets, better than full fine-tuning (20 / 16) and LoRA (20 / 19). PaCTS is also the only method that improves over zero-shot on average across datasets, and has the smallest worst-case degradation. At the configuration level, PaCTS improves both metrics on 64 of the 97 GIFT-Eval and 45 of the 98 TIME configurations, compared with 47 / 32 for full fine-tuning and 50 / 32 for LoRA. These results show that the improvements of PaCTS are stable and broadly distributed.

The per-dataset changes of PaCTS on GIFT-Eval are given above, and those on TIME below:

BenchmarkMethod (context)MASECRPS
GIFT-EvalZero-shot (4,096)0.7080.496
GIFT-EvalFull FT (4,096)0.703 ± 0.0000.487 ± 0.000
GIFT-EvalLoRA (4,096)0.703 ± 0.0000.493 ± 0.000
GIFT-EvalPaCTS (4,096)0.697 ± 0.0010.480 ± 0.001
TIMEZero-shot (4,096)0.6640.558
TIMEFull FT (4,096)0.665 ± 0.0000.560 ± 0.001
TIMELoRA (4,096)0.666 ± 0.0010.560 ± 0.000
TIMEPaCTS (4,096)0.661 ± 0.0010.555 ± 0.000
TIME (37 datasets)Datasets improved (MASE)Datasets improved (CRPS)Dataset-level change vs. zero-shot (MASE)Dataset-level change vs. zero-shot (CRPS)Largest per-dataset MASE increase
Full FT (4,096)2016+0.16%+0.37%+14.59%
LoRA (4,096)2019+0.41%+0.39%+19.73%
PaCTS (4,096)2121−0.40%−0.59%+5.15%
TIME datasetPaCTS Δ MASE (%)PaCTS Δ CRPS (%)
Housing_Inventory−17.38−11.44
Vehicle_Supply−9.03−2.72
Vehicle_Sales−3.73−5.59
Australia_Solar−3.63−2.11
US_Term_Structure−1.88−1.25
epf_electricity_price−1.76−1.59
Commodity_Import−1.70−13.31
OpenElectricity_NEM−1.32+0.82
Commodity_Production−1.26−1.93
EWELD_Load−1.07−6.56
Crypto−1.00−0.38
SG_PM25−0.80−1.56
azure2019−0.52−0.82
Port_Activity−0.50−1.27
Oil_Price−0.46−0.69
NE_China_Wind−0.38+0.38
Finland_Traffic−0.31+2.15
CPHL−0.31−0.70
WUI_Global−0.23+0.31
current_velocity−0.22−0.53
Online_Retail_2_UCI−0.05−0.26
SG_Weather+0.05−0.33
MetroPT-3+0.06−0.03
SG_Carpark+0.13+0.34
Auto_Production_SF+0.26+0.56
Smart_Manufacturing+0.35+0.34
Water_Quality_Darwin+0.39−1.85
Global_Influenza+0.70+3.70
Supply_Chain_Location+0.81+0.70
Coastal_T_S+0.95+0.91
Supply_Chain_Customer+1.28+0.92
ECDC_COVID+1.45+2.57
Uncertainty_1M+1.58+2.29
JOLTS+1.86+1.73
Global_Price+3.89+3.63
Job_Claims+4.55−0.41
US_Labor+5.15+2.66
Datasets improved21 of 3721 of 37

Claims about replacing longer contexts (Weakness 2)

We agree that the prompts cannot access history beyond the context window, and we will rephrase the contribution as an accuracy–cost trade-off: comparable or better accuracy than zero-shot inference at twice the context length, at substantially lower inference cost.

We thank reviewer UbYb again. We hope the answers and experiments address the concerns and questions of the reviewer.

Dear Reviewer UbYb, thank you again for reviewing our paper. As the discussion period is ending, we would greatly appreciate it if you could let us know whether our responses and the revised paper address your concerns. We are happy to clarify any remaining questions.

Reviewer_Pq9YOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PaCTS, which provides compact context surrogates for frozen time series foundation models through instance-adaptive latent prompts. The method generates adaptive prompts from global statistics of the visible context and refines them using segment-level statistics and multi-head cross-attention. The adaptive component and its refinement are combined, gated, and added to a shared learned prompt component before prepending the resulting prompt tokens to the time series patch embeddings. Only the prompt module is trained, while the backbone remains frozen, and inference requires no further parameter updates. Experiments on GIFT-Eval show improved forecasting performance across multiple backbone architectures, while direct evaluation on TIME demonstrates out-of-distribution generalization without additional tuning.

Critical strengths
  1. It is valuable to explore how to improve forecasting while keeping pre-trained TSFM weights frozen, and this serves as a promising adaptation path.
  2. Time series heterogeneity is highly prominent, and the motivation to tailor the method specifically to these characteristics is well-justified.
Critical weaknesses
  1. The supervised adaptation of PaCTS within the training intervals of the GIFT-Eval dataset invalidates the zero-shot setting. Therefore, a direct comparison with the baseline model's zero-shot performance is unfair.
  2. The claim of replacing long contexts lacks sufficient validation, as there is no empirical guarantee that the available history at each forecasting start point strictly exceeds 8,192. Thus, it remains unproven whether this approach can truly substitute actually available extended historical contexts.
Questions for authors
  1. Would it be possible to include comparisons under both the LoRA and full fine-tuning settings for TimesFM-2.5 and PatchTST-FM r1? Comparing directly against the zero-shot baseline in Figure 5 seems unfair.
  2. Could the authors filter the evaluation data so the available history at each forecasting start point exceeds 8,192, and re-compare Chronos-2 zero-shot, full fine-tuning, LoRA, and PaCTS?
  3. Is it feasible to test whether a prompt module trained on one context length can be directly applied to other context lengths without additional training?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Pq9Y

We thank reviewer Pq9Y for the insightful comments. In the following we address the concerns and provide answers to the questions.

Comparison with full fine-tuning and LoRA on TimesFM-2.5 and PatchTST-FM r1 (Weakness 1 and Question 1)

We would like to clarify that Figure 5 is intended to show that, on each backbone, PaCTS reaches the accuracy of the frozen TSFM with doubled context length at substantially lower inference cost.

To address the reviewer's request, we further provide full fine-tuning and LoRA baselines on TimesFM-2.5 and PatchTST-FM r1, trained with the same data and pipeline as PaCTS. The results are in the following tables. As on Chronos-2, PaCTS outperforms both full fine-tuning and LoRA.

TimesFM-2.5, 4,096MASECRPS
Zero-shot (4,096)0.7170.514
Zero-shot (8,192)0.7090.506
Full fine-tuning0.7100.511
LoRA0.7120.512
PaCTS0.706 ± 0.0020.507 ± 0.001
TimesFM-2.5, 8,192MASECRPS
Zero-shot (8,192)0.7090.506
Zero-shot (16,384)0.7060.503
Full fine-tuning0.7080.503
LoRA0.7040.508
PaCTS0.700 ± 0.0020.500 ± 0.002
PatchTST-FM r1, 4,096MASECRPS
Zero-shot (4,096)0.7160.490
Zero-shot (8,064)0.7040.481
Full fine-tuning0.7070.488
LoRA0.7050.490
PaCTS0.701 ± 0.0010.484 ± 0.000

We will add the above results and discussions in the paper.

Experiments on long-context configurations (Weakness 2 and Question 2)

We agree with the reviewer that the prompt cannot provide all the information contained in a longer context. Instead, PaCTS makes better use of the statistical information in the available history, reaching accuracy comparable to that with doubled context at substantially lower inference cost.

To show the improvement on data whose history strictly exceeds 8,192 steps, we select the GIFT-Eval configurations in which every test window has more than 8,192 steps of available history. This yields 36 configurations from 10 datasets: bizitobs_l2c/5T, electricity/15T and /H, ett1/15T and /H, ett2/15T and /H, jena_weather/10T, kdd_cup_2018/H (short and medium), loop_seattle/5T, m_dense/H, saugeen/D (short) and solar/10T (all three terms unless noted). The results on Chronos-2 are shown in the following table.

With half of the context, PaCTS improves over zero-shot Chronos-2 at 8,192 on both metrics, whereas full fine-tuning at 4,096 improves MASE only marginally, and LoRA at 4,096 is worse on both. This is consistent with Figure 3: the gains of weight-space adaptation concentrate on configurations with shorter histories, while the prompt remains effective when long histories are available.

Method (context)MASECRPSΔ MASE vs. zero-shot (8,192)Δ CRPS vs. zero-shot (8,192)
Zero-shot (8,192)0.7000.512——
Full FT (4,096)0.698 ± 0.0010.512 ± 0.001−0.29%0.00%
LoRA (4,096)0.703 ± 0.0000.517 ± 0.001+0.43%+0.98%
PaCTS (4,096)0.687 ± 0.0030.501 ± 0.002−1.86%−2.15%

We will add these experiments and discussions in the paper.

Generalize across context lengths (Question 3)

To test whether the prompt module generalizes across context lengths, we apply the module trained at 4,096 directly to twice the context (8,192; 8,064 for PatchTST-FM r1) without retraining. As shown in the following table, although slightly weaker in some settings, it recovers most of the gain of a module trained natively at the longer context, and clearly outperforms zero-shot inference. A single prompt module can therefore be reused at twice its training context without retraining.

BackboneZero-shot (8,192) MASEZero-shot (8,192) CRPSPaCTS 4,096 → 8,192 MASEPaCTS 4,096 → 8,192 CRPSPaCTS trained at 8,192 MASEPaCTS trained at 8,192 CRPS
Chronos-20.7040.4910.698 ± 0.0010.479 ± 0.0010.695 ± 0.0010.478 ± 0.001
TimesFM-2.50.7090.5060.700 ± 0.0010.499 ± 0.0000.700 ± 0.0020.500 ± 0.002
PatchTST-FM r10.7040.4810.694 ± 0.0010.478 ± 0.0010.692 ± 0.0020.476 ± 0.001

We will add the results and discussions in the paper.

We thank reviewer Pq9Y again. We hope that our answers and experiments address the concerns and questions of the reviewer.

Thank you for the additional experiments. The comparisons with LoRA and full fine-tuning on both TimesFM-2.5 and PatchTST-FM r1, the evaluation on windows with more than 8,192 steps of available history, and the transfer results across context lengths address my main concerns. I have raised my score accordingly.

We sincerely thank the reviewer for the positive feedback and for raising the score. We will incorporate the additional comparisons with full fine-tuning and LoRA, the evaluation on configurations with more than 8,192 steps of history, and the context-length generalization results into the revised paper.

Reviewer_qCroOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes PaCTS, a lightweight adaptation method for frozen time-series foundation models. A small set of continuous prompt tokens is generated from a shared learned component, a low-rank component conditioned on global time-series statistics, and a segment-level refinement module. The prompt module is trained jointly across heterogeneous datasets and applied without test-time parameter updates. Experiments on GIFT-Eval and transfer evaluation on TIME show improvements in point and probabilistic forecasting.

Critical strengths
  1. Important practical motivation. Increasing the historical context of a TSFM can substantially increase inference cost. PaCTS offers an appealing alternative by introducing a small number of learned tokens while keeping the pretrained backbone frozen.

  2. A structured and lightweight prompt-generation mechanism. The decomposition into shared prompts, global-statistics-conditioned prompts, and segment-level refinement provides a reasonable way to combine cross-series regularities with instance-specific temporal information. The low-rank prompt basis also limits the number of trainable parameters.

Critical weaknesses
  1. The novelty relative to existing prompt-learning and parameter-efficient adaptation methods needs clarification. The method combines shared prompt tokens, a statistics-conditioned low-rank prompt generator, and cross-attention-based segment refinement. Each component is reasonable, but the overall design is closely related to existing input-space prompt tuning and conditional prompt-generation approaches.

  2. The baseline comparisons are incomplete across backbones and adaptation paradigms. The evaluation on Chronos-2 includes five useful adaptation baselines: full fine-tuning, LoRA, LayerNorm-only tuning, BitFit, and linear probing. However, the evaluation on PatchTST-FM and TimesFM-2.5 compares PaCTS primarily with zero-shot inference, without the corresponding weight-space adaptation baselines or comparable prompt-based alternatives.

Questions for authors
  1. How does PaCTS compare with existing time-series prompt-learning approaches under matched backbones, context lengths, and training data?

  2. Are the reported improvements statistically significant across the 97 GIFT-Eval configurations, and how consistently does PaCTS improve individual datasets?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qCro (1/2)

We thank reviewer qCro for the insightful comments. In the following we address the concerns and provide answers to the questions.

Novelty relative to existing prompt-learning and parameter-efficient adaptation methods (Weakness 1)

We agree that existing methods also explore input-space latent prompting. PaCTS differs from them in three respects.

Research target. PaCTS aims to improve a frozen TSFM effectively and efficiently with compact context derived from its own available history, rather than with additional information from other channels (Gen-P-Tuning) or from vision and text (UniCast, CoSPOT). To the best of our knowledge, PaCTS is the first method that treats soft prompts as a summary instruction and compact context to improve frozen TSFMs in general.

Training across heterogeneous datasets. To improve frozen TSFMs in general, we train and evaluate a single prompt module jointly on heterogeneous datasets, without dataset-specific adaptation. One module thus serves all datasets without per-dataset training or storage, matches the general-purpose use of TSFMs, and transfers to datasets unseen in training. In contrast, Gen-P-Tuning, UniCast and CoSPOT train a separate model for each dataset or task.

Architecture design. To make instance-adaptive prompts work across diverse datasets with different context lengths and forecast horizons, we design a statistics-conditioned low-rank prompt generator that explicitly extracts high-level information from the available history, and a cross-attention-based segment refinement that adds finer statistics of ordered segments for long inputs. Table 3 and our answer to Q1 demonstrate the effectiveness of these designs.

We will add these contribution clarifications in the main paper.

Comparison with existing time-series prompt-learning methods (Question 1)

Since neither of these methods reports results on current TSFMs trained across heterogeneous datasets, we reimplement the latent prompt generators of CoSPOT and Gen-P-Tuning based on the PaCTS pipeline.

The results are provided in the following table. PaCTS outperforms both baselines on both metrics at both context lengths. We attribute the gap to the conditioning signal: low-frequency amplitudes and patch embeddings carry no explicit summary of how level and scale change across the window, which PaCTS provides through its global and ordered-segment statistics.

We will add these baselines in Table 1 in the main paper.

GIFT-Eval, Chronos-2 (mean over 3 seeds)4,096 MASE4,096 CRPS8,192 MASE8,192 CRPS
Zero-shot0.7080.4960.7040.491
CoSPOT0.7070.4900.7030.489
Gen-P-Tuning0.7020.4840.7010.482
PaCTS0.697 ± 0.0010.480 ± 0.0010.695 ± 0.0010.478 ± 0.001
AuthorsOpenReview ↗

Response to Reviewer qCro (2/2)

Comparison with baseline methods on TimesFM-2.5 and PatchTST-FM r1 (Weakness 2)

We would like to clarify that Figure 5 is intended to show that, on each backbone, PaCTS reaches the accuracy of the frozen TSFM with doubled context length at substantially lower inference cost.

To address the reviewer’s request, we further provide full fine-tuning and LoRA baselines on TimesFM-2.5 and PatchTST-FM r1, trained with the same data and pipeline as PaCTS. The results are in the following tables. As on Chronos-2, PaCTS outperforms both full fine-tuning and LoRA.

TimesFM-2.5, 4,096MASECRPS
Zero-shot (4,096)0.7170.514
Zero-shot (8,192)0.7090.506
Full fine-tuning0.7100.511
LoRA0.7120.512
PaCTS0.706 ± 0.0020.507 ± 0.001
TimesFM-2.5, 8,192MASECRPS
Zero-shot (8,192)0.7090.506
Zero-shot (16,384)0.7060.503
Full fine-tuning0.7080.503
LoRA0.7040.508
PaCTS0.700 ± 0.0020.500 ± 0.002
PatchTST-FM r1, 4,096MASECRPS
Zero-shot (4,096)0.7160.490
Zero-shot (8,064)0.7040.481
Full fine-tuning0.7070.488
LoRA0.7050.490
PaCTS0.701 ± 0.0010.484 ± 0.000

We will add the above results and discussions in the paper.

Significance and per-configuration consistency (Question 2)

To test whether the improvements are significant, for each configuration we compute the ratio of the error of a method to that of zero-shot Chronos-2, and the overall change is the geometric mean of these ratios over the 97 configurations minus one. Since the configurations of one source dataset (e.g. electricity at 15T, H, D and W, or the M4 subsets) are not independent, we use a dataset-level bootstrap: we resample the source datasets with replacement, keeping all configurations of a sampled dataset together, recompute the overall change, and repeat this 10,000 times. The 95% confidence interval is given by the 2.5th and 97.5th percentiles. As shown in the following table, PaCTS is the only method whose improvement over zero-shot Chronos-2 is significant on both metrics, with both intervals entirely below zero.

Method (4,096)95% CI of Δ MASE95% CI of Δ CRPS
Full FT[−1.90%, +0.23%][−5.23%, +0.41%]
LoRA[−1.70%, −0.04%][−1.99%, +0.18%]
PaCTS[−3.20%, −0.01%][−7.57%, −0.05%]

For consistency, as shown in the following table, PaCTS improves both metrics on 64 of the 97 configurations compared with zero-shot Chronos-2 as in Table 1, outperforming full fine-tuning (47) and LoRA (50):

Method (4,096)Δ MASEΔ CRPSConfigurations improved on both metrics (of 97)
Full FT−0.71%−1.81%47
LoRA−0.71%−0.60%50
PaCTS−1.55%−3.23%64

We will add the above results and discussions in the paper.

We thank reviewer qCro again. We hope the answers and experiments address the concerns and questions of the reviewer.

We sincerely thank the reviewer for the reply and for keeping the positive rating. We are glad that our responses have addressed most of the concerns, and we would be happy to clarify any remaining ones.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of the revision

We thank all reviewers for the constructive comments. We have uploaded a revised paper, with all changes marked in blue. The main changes are summarized below.

Main paper

  • Contributions and positioning (Reviewers qCro, UbYb). We clarify the three contributions in Section 1: improving frozen TSFMs with compact context built from the available history, a single prompt module trained jointly across heterogeneous datasets, and the statistics-conditioned low-rank generator with ordered-segment refinement. Section 2 now positions PaCTS against Time-LLM, UniCast, CoSPOT and Gen-P-Tuning.
  • Time-series prompt-learning baselines (Reviewer qCro). CoSPOT and Gen-P-Tuning, reimplemented on Chronos-2 within the PaCTS pipeline, are added to Table 1 and discussed in Section 4.2.
  • Generator architecture ablation (Reviewer UbYb). Table 3 adds a matched-size MLP generator that receives the same global and ordered-segment statistics.
  • Accuracy–cost trade-off (Reviewer UbYb). The Abstract, Section 1 and Section 4.2 now describe the prompts as compact context that achieves comparable or better accuracy than zero-shot inference with twice the context, at substantially lower inference cost, rather than as a replacement for longer history.
  • Long-history configurations (Reviewer Pq9Y). Section 4.2 reports the results on the 36 GIFT-Eval configurations whose history exceeds 8,192 steps.
  • Other backbones (Reviewers qCro, Pq9Y). Section 4.2 summarizes the comparison with full fine-tuning and LoRA on TimesFM-2.5 and PatchTST-FM r1 and the transfer across context lengths.
  • Limitations (Reviewer UbYb). Section 5 discusses the visible-history limitation and the negative transfer on a few datasets.

Appendix D (new results)

  • Uncertainty over seeds (Reviewer UbYb). Table 8 reports the standard deviation over three seeds on GIFT-Eval and TIME.
  • Fine-tuning baselines under the PaCTS pipeline (Reviewer UbYb). Table 9 reports full fine-tuning and LoRA trained within the PaCTS pipeline, together with training time and peak GPU memory.
  • Significance and consistency (Reviewers qCro, UbYb). Table 10 reports dataset-level bootstrap confidence intervals and per-configuration consistency.
  • Dataset-level results and negative transfer (Reviewer UbYb). Tables 11 and 12 report dataset-level results on GIFT-Eval and TIME, together with the analysis of negative transfer and identical descriptors.
  • Long-history configurations (Reviewer Pq9Y). Table 14 details the configurations with history longer than 8,192 steps.
  • Fine-tuning baselines on other backbones (Reviewers qCro, Pq9Y). Table 22 compares PaCTS with full fine-tuning and LoRA on TimesFM-2.5 and PatchTST-FM r1.
  • Generalization across context lengths (Reviewer Pq9Y). Table 23 reports the transfer of the prompt module to twice the training context on all three backbones.

We thank the reviewers again for the time and the valuable feedback. We hope that the revision and our responses address the concerns, and we would be happy to answer any further questions during the discussion period.

Dear reviewers,

Thank you again for reviewing our work.

As the discussion period is close to its end, we would like to know if there is any remaining concern. We are available to provide any additional clarification.

Thank you!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.