acceptodds
Under review as a conference paper at ICLR 2027

TTC4TS: Scaling Test-Time Compute for Time Series Forecasting

Abstract

Test-time compute has become an important paradigm in computer vision and language modeling, where additional computation at inference time enables models to explore multiple candidate solutions and select or refine the most reliable one. However, time series forecasting models typically produce predictions through a single forward pass, leaving them vulnerable to distribution shifts and non-stationarity that frequently arise in real-world temporal data. We propose Time Test Compute for Time Series (TTC4TS), an architecture‑agnostic framework that improves the predictive performance of frozen forecasting models at inference time. Given a pretrained backbone, the generator produces multiple candidate forecast trajectories using augmentation, retrieval, adapter. The synthesizer then selects or combines these candidates using aggregation, classification, or candidate-scoring and ranking methods to produce the final forecast. By increasing the number of candidates, TTC4TS scales test-time compute without retraining the forecasting backbone. Beyond improving point-forecast accuracy, TTC4TS can exploit candidate forecasts for uncertainty quantification, which supports improved interval forecasts. We also theoretically analyze when additional test-time computation improves forecasting Experiments across diverse datasets and backbones demonstrate consistent gains in accuracy and calibration. Code is provided in the supplementary materials.

Primary area:
learning on time series and dynamical systems
Keywords:
Time Series Forecasting, Test Time Compute, Test Time Adaptation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_7hGROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TTC4TS, an algorithm that gives better forecasts from some pre-trained time series foundational model without retraining. Instead of just asking the model for one single forecast, it asks for up to 80 slightly different ones, produced by giving the model slightly altered versions of the same data each time (adding noise, stretching it, flipping it, etc.) Then, a second component combines all of them together into a final forecast (either by an average, by a learned weighted average learned from past data or by a small network trained to pick the best combination of the forecasts for each case).

The main thesis is that, as it happens with LLMs, spending more compute at prediction time can lead to better forecasts. The method is tested with 6 pretrained models on 6 datasets. The authors report a reduction of the forecast error (using the fixed weighted average algorithm) of 6.4% on average, with improvements on the original model in 420 of the 432 tested settings.

Critical strengths
  • Improving an existing model without retraining it is a useful practical goal, and the method works with any model, which is a plus.
  • The authors report negative results openly, including failed components, the cost in time, and the fact that a small subset of candidates matches the full set. This made the paper easy to evaluate.
  • The experiments are careful not to use information from the future, a common problem in forecasting papers. I did not find this happening here.
Critical weaknesses

The paper should be rejected because: (1) It does not establish that the proposed method is really worth its cost, which is about 60 times more computation per final forecast; (2) the paper overstates the evidence; and (3) the method reduces to known techniques, with its own results showing that the "test-time compute" idea is not what produces the stated gains in the end.

Reasons for rejection

  • The practical value of the framing is not well stablished. The complete method runs the model 61 times per forecast, this makes the smallest model go from 43 ms to about 2600 ms. It is also never compared against other possible ways of spending that compute, such as a bigger model on its own like Chronos-Bolt-base, which takes 74 ms or an ensemble of different models.

  • The evidence makes the size of the obtained gains and the ranking between model variants hard to trust. For example, on the hourly EET data just 4 non-overlapping test forecasts at the longest horizon and about 30 at the shortest are used. The "420 of 432" claim rests on the reuse the same test data across horizons, seeds and models, and no confidence intervals are reported.

  • Furthermore, the contribution reduces to some good known techniques: The experiments show that, in fact, the proposed test time compute does not really help (for example, the proposed test-time classifier is worse than using the original model alone, giving 23% higher error) and what works best in the end is just a classical fixed weighted average (Bates & Granger, 1969) obtained by an ensemble selection algorithm (Caruana et al. (ICML 2004), not cited). This is probably showing a fundamental shortcoming of the proposed idea: test-time compute works well for LLMs because many of the proposed answers can be checked or compared before it is returned (think on a solution to a math or programing problem), but in forecasting, a prediction can only be checked after the future arrives, which is never available at test-time.

Questions for authors
  • If 13 forecasts match 80, what is the case for the full method, and how does it differ from standard forecast combination (Bates & Granger, 1969) and ensemble selection (Caruana et al., 2004)?
  • Could you report the actual error values and compare against a larger model, an average of several models, and the models' own probabilistic forecasts, at the same computational cost? This would address the second weakness.
  • Could you report confidence intervals over the test forecasts and evaluate on a longer test period?
Reviewer_FQ3ROpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes TTC4TS, a framework for improving frozen time series forecasting models through additional test-time computation. Instead of producing a single forecast, the framework generates multiple candidate predictions using data augmentation, residual adapters, and historical retrieval. These candidates are then combined or selected through various synthesis strategies, including uniform averaging, learned weights, ensemble learning, and candidate scoring.

The paper also analyzes when additional candidates can improve forecasting performance, highlighting the roles of candidate diversity and aggregation error. The framework is further extended to uncertainty quantification using conformal prediction. Experiments across multiple datasets and forecasting backbones demonstrate consistent improvements over the original frozen models. The results also suggest that effective aggregation is important, as increasing the number of candidates does not always improve prediction accuracy.

Critical strengths

(1) An interesting and potentially useful research direction.

Exploring test-time compute for time series forecasting is a worthwhile direction, especially with the increasing use of pretrained forecasting models. The proposed generate-and-synthesize framework offers a systematic way to improve predictions without updating the backbone at test time. I think this general perspective could motivate further research.

(2)Extensive experiments with some useful findings.

The authors conduct a fairly comprehensive evaluation across multiple forecasting models, datasets, and synthesis strategies. The reported improvements are generally consistent. I also appreciate that the paper goes beyond reporting accuracy gains and attempts to explain when additional candidates are useful. In particular, the finding that more candidates do not necessarily lead to better forecasts is meaningful. The authors also report negative results for some synthesis strategies rather than presenting only favorable comparisons.

Critical weaknesses

(1)The methodological novelty is somewhat limited: My main concern is that the proposed framework largely combines existing techniques rather than introducing a substantially new method. Data augmentation, residual adaptation, retrieval-based forecasting, and ensemble learning are all well-established ideas. The strongest synthesis strategies mainly rely on weights or ensemble structures learned offline, making the approach quite similar to conventional forecast ensembling. The paper presents TTC4TS as a new test-time compute paradigm, but I am not fully convinced that it offers a fundamentally different inference mechanism. The theoretical analysis is reasonable, but much of it builds on known properties of convex combinations and ensemble prediction. I would expect clearer evidence of what is genuinely new beyond organizing these existing techniques into a unified framework.

(2)The necessity of the complete framework is not well established: This is one of my biggest concerns. Appendix G.2 reports that learned weighting over the original forecast and augmentation candidates alone achieves a relative MSE of 0.935, compared with 0.936 for the full framework. Although the full framework performs slightly better in MAE, the MSE results suggest that the additional adapter and retrieval components do not provide a clear overall benefit under this synthesis strategy. The authors acknowledge this limitation, which I appreciate. However, it still raises an important question: what do these additional components actually contribute? The experiments show that combining different generators can be useful, but they do not convincingly establish why the complete framework is preferable to a much simpler alternative.

(3)The computational overhead is substantial, and the accuracy–cost trade-off is not convincing: Since scaling test-time compute is the central theme of the paper, I think computational efficiency deserves more attention. The measurements in Appendix G.5 show a substantial increase in inference latency for the complete framework. However, the overall accuracy improvements are relatively modest compared with this additional cost. The candidate-budget experiments mainly examine prediction accuracy as the number of candidates increases. They do not establish whether TTC4TS provides a favorable accuracy–computation trade-off against simpler methods under matched computational budgets.

(4)The generalization and distribution-shift claims need stronger support: The paper motivates TTC4TS partly by the challenges of non-stationarity and distribution shifts in time series forecasting. However, the experiments mainly demonstrate average accuracy improvements on standard forecasting benchmarks. There is limited direct evidence that the proposed method is particularly robust to distribution shifts or changing temporal patterns. Another concern is that the learned synthesis weights are fitted separately for each backbone, dataset, and forecasting horizon. This raises questions about how well the method would work on unseen datasets or when the deployment distribution differs from the fitting data.

Questions for authors

(1) What is the main methodological difference between TTC4TS and conventional forecast ensembling?

Since the best-performing synthesis methods largely use offline-learned weights, I would like to understand what makes the proposed approach fundamentally different from existing augmentation-based ensembles.

(2) Are all three candidate-generation mechanisms necessary?

Appendix G.2 shows that augmentation-only learned weighting achieves essentially the same MSE as the full framework. Could the authors explain the additional value of the adapter and retrieval components? Are there particular datasets or situations where these components are clearly beneficial?

(3) Can the authors provide a more convincing accuracy–cost comparison?

The complete framework introduces substantial inference overhead. How does TTC4TS compare with simpler ensembles under the same latency or computational budget? I think this is essential for supporting the test-time scaling claim.

(4) How well does the method generalize beyond the fitting distribution?

Since the synthesis weights are fitted for individual datasets and forecasting horizons, would the same weights remain effective under distribution shifts or on unseen datasets? Is there direct evidence that TTC4TS improves robustness to non-stationarity, as suggested in the introduction?

Reviewer_uJUrOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

TTC4TS proposes to improve a frozen time-series forecaster by generating alternative forecasts through input transformations, offline-fitted bounded residual adapters, and aligned historical demonstrations. Selection or convex aggregation then produces the final point prediction without test-query parameter updates. The paper also analyzes candidate-set geometry and aggregation error, and studies conformally calibrated prediction intervals derived from candidate information. Across six backbones, six datasets, four horizons, and three seeds, the manuscript reports a geometric-mean relative MSE of 0.936 and improvements in 420/432 configurations for learned-weight synthesis. Additional evaluations examine synthesis rules, candidate budgets, generator families, and uncertainty estimators.

Critical strengths
  1. A useful modular framework with explicit causal information boundaries. The framework combines several ways of improving a deployed forecaster while keeping its parameters fixed. The declared point-forecast protocol distinguishes offline auxiliary fitting from inference and restricts retrieval and retrospective scoring to information available at the relevant forecast origin.

  2. Careful analysis of the limits of candidate expansion. The theory separates improvements in attainable convex aggregation from aggregation error and distinguishes population-risk oracles from unavailable test-label selection. The observation that densifying an endpoint-inclusive grid along a fixed affine output-adapter path does not enlarge its convex hull is particularly informative. The analysis explicitly states its assumptions and avoids claiming unconditional monotonic improvement for deployed methods.

  3. Broad diagnostics and transparent negative findings. The study examines multiple backbones, synthesis rules, generator families, and uncertainty constructions. It reports unfavorable outcomes, including classifier failures, severe undercoverage of raw candidate quantiles, and substantial measured inference costs. These disclosures improve the paper’s informativeness and help identify the limitations of different framework components.

Critical weaknesses
  1. Central numerical results require reconciliation. Table 2 reports a TimesFM learned-weight relative MSE of 0.868, whereas the six corresponding dataset values in Table 6 geometrically average to approximately 0.919. Section 5.2/Table 11 also reports approximately 0.942 for full-family Uniform aggregation, compared with 0.965 in Tables 1 and 3 under apparently matching settings. These differences exceed rounding error. Refitting a synthesizer cannot by itself explain differences for Uniform, which has no fitted synthesizer. Any changes in candidate pools, fitted generators, or evaluation protocols need to be identified before these comparisons can be assessed reliably.

  2. The named baseline variants are insufficiently specified. Table 1 explicitly describes its comparators as matched adaptation/retrieval variants, but the manuscript does not adequately document their method-specific modifications, update policies, and tuning procedures. Appendix G.1 describes shared TTC4TS component controls that do not clearly map to the displayed rows. A clear correspondence between each named row and its actual implementation and fitting protocol is needed to determine what the comparisons establish about the cited methods.

  3. The benefit–cost case for the complete generator under the strongest synthesis rules remains unclear. As Appendix G.2 acknowledges, augmentation-only learned weighting reaches 0.935 relative MSE versus 0.936 for the complete pool, while the complete pool has slightly better MAE. The measured full-pool learned-weight implementation executes 61 wrapper calls and takes approximately 60–70 times Frozen latency in Table 9’s ETTh1, horizon-96, batch-one setting; some calls are explicitly acknowledged as redundant. Full-pool greedy aggregation achieves a stronger 0.919 MSE ratio, but a corresponding augmentation-only greedy comparison is not reported. Such a comparison, with measured cost, would clarify the additional value of adapters and retrieval under the strongest synthesizer.

  4. The candidate-budget protocol needs a precise, consistent specification. Budget-specific fitting is explicitly disclosed and is valid within the paper’s scope. However, the exact candidate subsets and reused generator parameters at each budget are unclear. Equation (6) and Algorithm 2 order the generator families differently, without establishing whether that order determines the budget subsets. Appendix F.1 specifies 40 updates for budget-specific fits, whereas Appendix D.4.1 refers to a 160-update budget study. These details should be reconciled to make the curves reproducible and their interpretation unambiguous.

  5. The uncertainty-calibration protocol is incompletely specified. Section 4 states that a separate calibration block is used, but the manuscript does not give its chronological boundaries, the number of calibration origins, or its relationship to point-predictor selection and uncertainty-head fitting. These details are needed to assess the stated separation and reproduce the calibration results from the manuscript alone. Separately, the scale head’s training objective and complete fitting configuration are unspecified, limiting reproduction of the scale-based comparisons. This is a documentation concern, not evidence of leakage; the temporal-dependence limitations are appropriately acknowledged.

Questions for authors
  1. Clarification/correction: Can you reconcile the TimesFM learned-weight results in Tables 2 and 6 and the full-pool Uniform results in Tables 1/3 and 11? Please identify any differences in candidate pools, fitted generators, evaluation origins, or aggregation procedures.

  2. Clarification: For each named baseline variant in Table 1, what implementation, modifications, fitting data, update policy, and tuning procedure were used? How do these rows correspond to the shared-component controls described in Appendix G.1?

  3. Additional evidence affecting the assessment: Can the full-pool greedy result be compared with augmentation-only greedy aggregation using the same fitting protocol, with the corresponding measured inference costs? This would help establish the incremental contribution of adapters and retrieval under the strongest synthesizer.

  4. Clarification: What exact candidates are used at each budget in Figure 5, which generator parameters are reused, and are subsets constructed as prefixes of an ordered pool? Please also clarify the 40-update versus 160-update descriptions of budget-specific fitting.

  5. Clarification: What are the exact chronological blocks and sample counts for uncertainty-head fitting, model selection, and conformal calibration? Please specify the scale-head training objective and fitting procedure, and explain how calibration is separated from the relevant fitting and selection decisions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.