acceptodds
Under review as a conference paper at ICLR 2027

Label-Free Combination of Zero-Shot Time-Series Foundation Models

Abstract

Zero-shot time-series foundation models (TSFMs) forecast unseen series without training, and since no single model dominates, practitioners pool several. On a new domain, however, no target label is available to decide how to combine them or which of them to include. We argue that both decisions follow from one observation, that without labels a member's error can only be contained, not corrected. Averaging distributions rather than quantiles contains it, and on 97 GIFT-Eval and 98 TIME configurations the quantile mean trails the linear opinion pool (LOP) on nearly all 120 subsets of seven TSFMs, while the best setting of every other rule family lands within half a percent of LOP. Which members enter the pool matters far more, yet a subset chosen in advance, even with labels from other sources, recovers only part of the gain of choosing with the target's labels, so selection has to happen inside the combination, one forecast at a time. We propose HuberLOP, which replaces the equal weights of LOP with Huber weights computed from each member's distance to a robust center of the pool. It improves on LOP on both benchmarks, and when three naive forecasters join the pool its conservative variant degrades by 0.9% on both benchmarks, where LOP degrades by 14.0% on GIFT-Eval and 12.0% on TIME. With no training and one threshold, it is a simple yet effective baseline for label-free TSFM combination. Code is available at https://anonymous.4open.science/r/ICLR27-TSFM.

Primary area:
foundation or frontier models, including LLMs
Keywords:
time-series foundation models, forecast combination, zero-shot forecasting, probabilistic forecasting, linear opinion pool, robust aggregation, ensemble methods, label-free model selection

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_L1b6OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This work investigates training-free, label-free ensemble adaptation for time series foundation models (TSFMs). The authors challenge conventional quantile aggregation approaches, such as mean and median averaging, and propose HuberLOP, a robust extension of the linear opinion pool (LOP). HuberLOP penalizes disagreement from models whose forecasts deviate substantially from the collective center. The results demonstrate consistent improvements over equally weighted LOP and other ensemble approaches, including quantile mean and median aggregation.

Critical strengths

Practical and relevant problem. The paper addresses an important practical question: how to effectively combine forecasts from different TSFMs. Given the diversity of model architectures, pretraining data, and capabilities, developing more effective ensemble methods is valuable to the time series forecasting community.

Extensive empirical evaluation. The authors evaluate the proposed method on two prominent benchmarks, TIME and GIFT-EVAL, providing evidence of its performance across a range of forecasting settings.

Simple and generalizable approach. HuberLOP is a plug-and-play method that is training-free, parameter-free, and adaptable to arbitrary sets of forecasting models. This simplicity makes it easy to integrate into existing forecasting pipelines without additional training or supervision.

Critical weaknesses

Reliance on collective agreement: The method assumes that forecasts close to the collective center are more reliable, penalizing models that disagree with the majority. However, a model may be more accurate than most other models in particular scenarios, and the proposed method does not explicitly identify or account for such cases.

Limited empirical justification for the motivation: The paper makes strong claims about the need for a label-free and training-free ensemble method, but provides insufficient empirical justification for these claims. In particular, the authors do not compare the proposed method with context-based approaches such as Synapse and TimeCopilot, which may also support label-free adaptation. Comparisons with their ensemble strategies would help establish the value of HuberLOP relative to existing alternatives.

Marginal improvements over simple baselines: The improvements over basic approaches such as quantile averaging and quantile median appear relatively modest. Although the results in Table 2 suggest a more substantial difference between LOP and HuberLOP, the overall practical significance of the improvements requires further investigation, particularly against stronger, context-based alternatives.

Insufficient justification for excluding context-based adaptation: The paper argues that ensemble weights should be determined solely from member forecasts in a label-free and training-free manner, but this design choice is not sufficiently justified. The claim that no useful supervision is available when a new series arrives overlooks the potential of historical context for backtesting, in-context learning, or model adaptation. The authors acknowledge some of these possibilities but do not sufficiently establish why they are inadequate for the problem considered.

Limited comparison with existing methods and computational overhead: The paper does not provide empirical comparisons with relevant context-based approaches, such as Synapse and TimeCopilot, including their ensemble strategies where applicable. These methods may offer alternative ways to combine forecasts without requiring externally labeled data. Furthermore, the lack of a computational overhead analysis makes it difficult to assess whether HuberLOP provides a meaningful practical advantage over these alternatives; this might even uncover HuberLOP's strength.

Questions for authors

Definition of labeled data: Could the authors clarify what constitutes labeled data in their setting and distinguish it from contextual information, such as sampling frequency, domain, and sequence length? The current terminology may confuse readers about what information is available during adaptation.

Consistency of reported results: Why do the results in Table 2 for a pool size of seven differ from those in Figure 2 for the same pool size? Is this discrepancy due to Figure 2 evaluating every fifth forecast window, or are there other differences in the evaluation protocol?

5 comments
AuthorsOpenReview ↗

Author's Response to Reviewer L1b6 (1/3)

We sincerely thank the reviewer for your insightful comments, which have helped us improve the paper quality significantly. Below, we are more than willing to clarify the reviewer's concerns one by one.

Q1: Definition of labeled data: Could the authors clarify what constitutes labeled data in their setting and distinguish it from contextual information, such as sampling frequency, domain, and sequence length? The current terminology may confuse readers about what information is available during adaptation.

We appreciate the reviewer for providing a valuable discussion point. This distinction is central to our method, because HuberLOP_con never scores a forecast and therefore needs no labels of the target series. Labeled data are observed values used to score the members' forecasts, from the forecast horizon or from a held-out part of the context, whereas metadata such as frequency, domain, and length are not labels. The lookback window is model input and becomes labels only when part of it is held out for scoring, as in backtesting. We added this definition to Section 3.1.

W1: Reliance on collective agreement: The method assumes that forecasts close to the collective center are more reliable, penalizing models that disagree with the majority. However, a model may be more accurate than most other models in particular scenarios, and the proposed method does not explicitly identify or account for such cases.

We appreciate the reviewer for providing a crucial discussion point. From the current forecasts alone, no rule can tell a correct minority from a wrong one, so we chose the side that keeps the combined forecast from collapsing when bad models enter. Every agreement-based rule shares this limitation, including trimmed [3] and depth pools [4], and Synapse [1] states it in its Limitations section, "This self-referential loop risks confirmation bias, potentially favoring consensus models over correct outliers." When one TSFM is shifted by one context standard deviation and is wrong, HuberLOP_con has the lowest CRPS of nine robust rules, about 5% below LOP on GIFT-Eval and TIME. HuberLOP_con therefore follows the majority. If the disagreeing model is correct, the combination gives up what it would have added, and if it is wrong, it can pull the combination only a little, whereas under LOP a wrong model keeps its full share however far it goes. We discuss this limitation in Section 5 (Appendices C.2 and C.4).

Q2: Consistency of reported results: Why do the results in Table 2 for a pool size of seven differ from those in Figure 2 for the same pool size? Is this discrepancy due to Figure 2 evaluating every fifth forecast window, or are there other differences in the evaluation protocol?

Thanks for your comments, which have helped improve the clarity of our paper. We redrew Figure 2 using all forecasts, and it now matches Table 2. As the reviewer suspected, Figure 2 scored every fifth forecast while normalizing by seasonal naive scores computed on all forecasts. The conclusions of Section 4.3 are unchanged.

AuthorsOpenReview ↗

Author's Response to Reviewer L1b6 (2/3)

W2: Limited empirical justification for the motivation: The paper makes strong claims about the need for a label-free and training-free ensemble method, but provides insufficient empirical justification for these claims. In particular, the authors do not compare the proposed method with context-based approaches such as Synapse and TimeCopilot, which may also support label-free adaptation. Comparisons with their ensemble strategies would help establish the value of HuberLOP relative to existing alternatives.

W4: Insufficient justification for excluding context-based adaptation: The paper argues that ensemble weights should be determined solely from member forecasts in a label-free and training-free manner, but this design choice is not sufficiently justified. The claim that no useful supervision is available when a new series arrives overlooks the potential of historical context for backtesting, in-context learning, or model adaptation. The authors acknowledge some of these possibilities but do not sufficiently establish why they are inadequate for the problem considered.

Thanks for your thoughtful feedback, which has helped us better position our paper compared to the related works. Scoring members on the context complements a label-free rule rather than replacing it. Backtest weights and Synapse with a backtest history need twice the TSFM calls and gain significantly only on clean pools, and for a new series from an unseen domain there is no way to confirm in advance that the pool is clean. With naive forecasters or a member failing after the backtest, HuberLOP_con has lower error than both on GIFT-Eval and TIME, and on fev-bench [5], a third benchmark, it beats both under failure (Tables R2 and R3). Our reimplementation of Synapse [1] without history reads the same forecasts as HuberLOP_con, yet its error is more than 20% higher on GIFT-Eval and TIME, both with naive forecasters and when a member fails (Table R1). With a backtest budget, HuberLOP_con + backtest gives the lowest error in Table R2, so the two approaches are complementary (Appendices C.7, C.10, and C.13).

Synapse with archived history needs scored past forecasts of the same series, which a new series lacks, and under failure HuberLOP_con has 4 to 9% lower CRPS than it (Appendix C.12). Models adapted by in-context learning or fine-tuning can join our pool as members, and the ensemble of TimeCopilot [2] is the quantile median, which Table R1 includes.

Table R1: With the same forecasts as input, HuberLOP_con is more robust than Synapse and the TimeCopilot ensemble. CRPS on GIFT-Eval and TIME and scaled quantile loss on fev-bench (↓), in that order in each cell. "Failure" means that one TSFM outputs its last observation after the weights are set. None of these methods runs a backtest. The TimeCopilot ensemble is the quantile median of the members' quantiles, and Synapse is our reimplementation started from uniform weights.

MethodNaive addedFailure
TimeCopilot ensemble [2]0.4754 / 0.5557 / 0.52440.4693 / 0.5492 / 0.5096
Synapse [1]0.5908 / 0.6796 / 0.57670.5972 / 0.6648 / 0.5733
HuberLOP_con (ours)0.4701 / 0.5504 / 0.51670.4670 / 0.5469 / 0.5077

Table R2: When a member fails after the weights are set, HuberLOP_con beats both backtest-based methods with half the TSFM calls, and HuberLOP_con + backtest, which multiplies backtest weights by those of HuberLOP_con, gives the lowest error. Error when one TSFM outputs its last observation after the weights are set (↓).

MethodBacktestTSFM callsGIFT-EvalTIMEfev-bench
Synapse [1]✓140.47110.55590.5132
Backtest weights✓140.47740.56310.5296
HuberLOP_con (ours)✗70.46700.54690.5077
HuberLOP_con + backtest (ours)✓140.46590.54640.5065
AuthorsOpenReview ↗

Author's Response to Reviewer L1b6 (3/3)

W5: Limited comparison with existing methods and computational overhead: The paper does not provide empirical comparisons with relevant context-based approaches, such as Synapse and TimeCopilot, including their ensemble strategies where applicable. These methods may offer alternative ways to combine forecasts without requiring externally labeled data. Furthermore, the lack of a computational overhead analysis makes it difficult to assess whether HuberLOP provides a meaningful practical advantage over these alternatives; this might even uncover HuberLOP's strength.

We thank the reviewer for this suggestion, which indeed highlights a strength of HuberLOP_con. HuberLOP_con adds only 3.6 ms to the 123.5 ms of TSFM inference on TIME, needs no labels or stored histories, and is the most accurate of these methods when naive forecasters enter (Table R3). Backtesting runs every TSFM a second time, stored histories need per-series bookkeeping that a new series or model lacks, and the TimeCopilot agent also calls an LLM five or more times per run (Appendices C.9 and C.12).

Table R3: HuberLOP_con needs no labels, no stored history, and no extra TSFM calls, yet it is the most accurate when naive forecasters are added. Requirements and average time per forecast on TIME, and CRPS with three naive forecasters added (↓), with GIFT-Eval before TIME. Times add the combination time to 123.5 ms per inference pass, and the backtest rows assume two passes, with compiled exact-mixing Synapse. Bold marks no requirement or the best CRPS.

MethodTSFM callsLabelsTime (ms)CRPS, naive added
TimeCopilot ensemble [2]7No123.60.4754 / 0.5557
Synapse [1]14Yes≈ 2500.4759 / 0.5579
Backtest weights14Yes≈ 2500.4736 / 0.5571
HuberLOP_con (ours)7No127.00.4701 / 0.5504

W3: Marginal improvements over simple baselines: The improvements over basic approaches such as quantile averaging and quantile median appear relatively modest. Although the results in Table 2 suggest a more substantial difference between LOP and HuberLOP, the overall practical significance of the improvements requires further investigation, particularly against stronger, context-based alternatives.

Thanks for letting us know your concern regarding performance. HuberLOP_con improves on simple baselines even on clean pools, and the gap widens when bad models enter, on fev-bench as well. On the clean pool it is significantly better than the quantile mean, by about 2% on GIFT-Eval and TIME, and than the quantile median on all three benchmarks. These clean-pool gaps are modest because, when every member is sound, a label-free rule has little to correct, and its larger value appears when bad models enter, which no one can rule out on a new series. With naive forecasters, its error rises by less than 1% on GIFT-Eval and TIME and under 3% on fev-bench, less than the median's everywhere and a fraction of the quantile mean's (Table R4).

Table R4: HuberLOP_con degrades the least when three naive forecasters are added. Error (↓) and its increase (%) from the clean pool. Bold marks the best results.

MethodGIFT-EvalTIMEfev-benchIncrease (%)
Quantile mean0.62710.72050.760131.98 / 29.75 / 46.26
LOP0.53080.61400.606313.98 / 11.98 / 19.55
Quantile median0.47540.55570.52441.75 / 1.52 / 3.89
HuberLOP_con (ours)0.47010.55040.51670.93 / 0.87 / 2.79

Once again, we appreciate the reviewer highlighting valuable discussion points in our study. We hope our response has clarified the contents of the paper, and we are more than willing to engage in further discussion regarding any aspects that remain unclear.


References

[1] "Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models.", In Transactions on Machine Learning Research, 2026

[2] "TimeCopilot.", In arXiv preprint arXiv:2509.00616v3, 2025

[3] "Trimmed Opinion Pools and the Crowd's Calibration Problem.", In Management Science, 2014

[4] "Probabilistic Forecast Aggregation with Statistical Depth.", In European Journal of Operational Research, 2026

[5] "fev-bench: A Realistic Benchmark for Time Series Forecasting.", In Advances in Neural Information Processing Systems (NeurIPS), Evaluations & Datasets Track, 2026

Reviewer_hHtWOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies how to combine forecasts from frozen Time Series Foundation Models (TSFMs) without labels or additional training. It compares quantile aggregation and distribution pooling on both GIFT-Eval and TIME datasets, and examines all 120 subsets containing at least two of seven models under three main combination rules. The proposed HuberLOP method down weights forecasts that deviate from a robust center, assigning one weight to each model's predictive CDF (with a conservative variant using a lower threshold and the minimum weight across quantiles). The experiments examine how the choice of models, the pooling rule and the presence of poor forecasts affect accuracy and robustness.

Critical strengths
  1. The pool-composition analysis is substantial and the strongest part of the paper. Evaluating every multi-model subset under the three main rules helps separate the choice of models from the choice of combiner. The paper also makes the information available to each method explicit and evaluates context backtesting with its extra inference cost.
  2. The robustness tests support a practical use case for conservative Huber pooling. Table 4 shows much less degradation than uniform LOP when naive forecasters are added. The threshold sweep explains the trade-off on clean pools, and the trimming and displaced-member comparisons show where existing robust rules also work well.
Critical weaknesses
  1. I am less convinced by the added value of the weighting rule than by the pool-composition study. Table 2 shows roughly a 0.3% CRPS gain over uniform LOP on each benchmark and almost no difference from depth pooling. The authors acknowledge these close scores. Table 3's larger gains also change the pool size, so they do not isolate the benefit of weighting. Table 5 gives better CRPS with per-forecast weights in seven of eight cases, but fixed weights have lower MASE in all four contaminated cases, and the conservative variant worsens clean GIFT-Eval CRPS from 0.4642 to 0.4657. Although the conservative variant's robustness gains are substantial, but the advantage over the strongest existing rules needs more evidence than these small, mixed differences.

  2. The main missing comparison is depth-weighted pooling under contamination. Taylor's method is nearly tied with HuberLOP on the clean pool, and Appendix B.7 evaluates ten depth-based settings. However, its results are missing from both the added-naive-forecaster and displaced-member tests. Since robustness is the main reason to prefer the proposed weighting, I would like to see this comparison. Appendix B.1 already tests trimmed opinion pools under contamination; the remaining gap is specifically depth pooling.

Questions for authors
  1. How does depth-weighted pooling perform in the contamination and displacement tests, using the same forecasts and settings chosen before these tests? If the comparison is not feasible, what prevents it?
  2. Could you report paired confidence intervals for the differences from depth pooling, trimmed pooling and fixed weights, grouping configurations from the same source dataset when resampling? How should we interpret the CRPS gains alongside the MASE losses in Table 5?
  3. lines 200-201: the bound of 1/K applies to the change in the CDF at a fixed value. I don't think it means an outlier only widens the forecast without shifting its quantiles. Please try to qualify this statement.
6 comments
AuthorsOpenReview ↗

Author's Response to Reviewer hHtW (1/3)

We sincerely thank the reviewer for your insightful comments, which have helped us improve the paper quality significantly. Below, we are more than willing to clarify the reviewer's concerns one by one.

W2: The main missing comparison is depth-weighted pooling under contamination. Taylor's method is nearly tied with HuberLOP on the clean pool, and Appendix B.7 evaluates ten depth-based settings. However, its results are missing from both the added-naive-forecaster and displaced-member tests. Since robustness is the main reason to prefer the proposed weighting, I would like to see this comparison. Appendix B.1 already tests trimmed opinion pools under contamination; the remaining gap is specifically depth pooling.

Q1: How does depth-weighted pooling perform in the contamination and displacement tests, using the same forecasts and settings chosen before these tests? If the comparison is not feasible, what prevents it?

We thank the reviewer for this suggestion, which has helped us test our method against its closest competitor. With the same forecasts and the ten depth settings of Appendix B.6 (B.7 in the submission) fixed in advance, HuberLOP_con beats every depth setting in the contamination test and is never significantly worse than any of them when one member is displaced. With the three naive forecasters of Table 4, its error rises less than that of the most robust depth setting on GIFT-Eval, TIME, and fev-bench [1], a third benchmark we added, by about half on the first two (Table R1). In the displacement test, it is significantly worse in none of the 256 comparisons with the ten depth settings and six trimmed pools on any benchmark (Appendices C.2, C.3, and C.13).

Table R1: HuberLOP_con degrades the least when the three naive forecasters of Table 4 are added, ahead of all ten depth settings. Error (↓) and its increase (%) from the clean (curated) pool. Least and most robust are the depth settings with the largest and smallest increase among Taylor's ten. Bold marks the best results.

MethodGIFT-EvalTIMEfev-benchIncrease (%)
LOP0.53080.61400.606313.98 / 11.98 / 19.55
Depth pool, least robust [2]0.52290.60060.594411.96 / 9.81 / 17.79
Depth pool [2]0.50600.58690.57008.87 / 7.33 / 12.91
Depth pool, most robust [2]0.47860.55900.52902.02 / 1.83 / 4.07
HuberLOP_con (ours)0.47010.55040.51670.93 / 0.87 / 2.79

W1: I am less convinced by the added value of the weighting rule than by the pool-composition study. Table 2 shows roughly a 0.3% CRPS gain over uniform LOP on each benchmark and almost no difference from depth pooling. The authors acknowledge these close scores. Table 3's larger gains also change the pool size, so they do not isolate the benefit of weighting. Table 5 gives better CRPS with per-forecast weights in seven of eight cases, but fixed weights have lower MASE in all four contaminated cases, and the conservative variant worsens clean GIFT-Eval CRPS from 0.4642 to 0.4657. Although the conservative variant's robustness gains are substantial, but the advantage over the strongest existing rules needs more evidence than these small, mixed differences.

Thanks for letting us know your concern regarding performance. The weighting rule adds value at a fixed pool size, and most of all when bad models enter. Holding the pool fixed, HuberLOP beats LOP on all 57 subsets of three or more TSFMs that contain Moirai-base and beats the quantile median and the trimmed pool on all 42 without it on GIFT-Eval and TIME (Appendix C.5). Close clean-pool scores are expected, because a label-free rule limits how far one model moves the combined forecast and a clean pool has little to limit. With naive forecasters, HuberLOP_con also leaves significantly fewer forecasts more than 10% worse than the most robust depth setting on all three benchmarks, 9.4% against 16.7% on GIFT-Eval (Appendix C.11). Table 5 is addressed in Q2-1 and Q2-2.

AuthorsOpenReview ↗

Author's Response to Reviewer hHtW (2/3)

Q2-1: Could you report paired confidence intervals for the differences from depth pooling, trimmed pooling and fixed weights, grouping configurations from the same source dataset when resampling?

We thank the reviewer for this suggestion, which has helped us establish the reliability of our results more rigorously. With paired intervals over source datasets, HuberLOP_con is never significantly worse on the clean pool and is significantly better than every deployable baseline of Table R2 once naive forecasters enter. The clean-pool gap to the fixed weights of Table 5 that the reviewer noted, 0.4657 against 0.4642 on GIFT-Eval, is not significant, and on TIME and fev-bench [1] HuberLOP_con is even lower than those weights, which look ahead and cannot be deployed (Q2-2). With naive forecasters, HuberLOP_con is 5 to 9% below the depth and trimmed pools on all three benchmarks and significantly below even the hindsight fixed weights on GIFT-Eval and TIME (Table R2, Appendix C.1).

Table R2: HuberLOP_con is never significantly worse on the clean pool and is the best once naive forecasters are added. CRPS on GIFT-Eval and TIME and scaled quantile loss on fev-bench (↓). Parentheses give the method's error minus that of HuberLOP_con, with paired 95% confidence intervals over source datasets. The fixed weights are those of Table 5. Bold marks the best results and differences in favour of HuberLOP_con.

MethodGIFT-EvalTIMEfev-bench
Clean pool
Depth pool [2]0.4648 (−0.0009 [−0.0031, 0.0012])0.5468 (0.0012 [−0.0001, 0.0025])0.5049 (0.0022 [−0.0023, 0.0086])
Trimmed pool [3]0.4657 (0.0000 [−0.0014, 0.0017])0.5466 (0.0010 [0.0000, 0.0021])0.5047 (0.0020 [−0.0017, 0.0075])
Fixed weights, hindsight0.4642 (−0.0015 [−0.0046, 0.0014])0.5461 (0.0004 [−0.0010, 0.0018])0.5043 (0.0016 [−0.0022, 0.0095])
HuberLOP_con (ours)0.46570.54560.5027
Three naive forecasters added
Depth pool [2]0.5060 (0.0360 [0.0247, 0.0508])0.5869 (0.0365 [0.0214, 0.0533])0.5700 (0.0533 [0.0294, 0.0966])
Trimmed pool [3]0.5036 (0.0336 [0.0225, 0.0475])0.5805 (0.0301 [0.0181, 0.0431])0.5644 (0.0477 [0.0262, 0.0864])
Fixed weights, hindsight0.4733 (0.0032 [0.0012, 0.0056])0.5541 (0.0037 [0.0009, 0.0065])0.5190 (0.0023 [−0.0028, 0.0140])
HuberLOP_con (ours)0.47010.55040.5167

Q2-2: How should we interpret the CRPS gains alongside the MASE losses in Table 5?

We appreciate the reviewer for providing a valuable discussion point. MASE scores only the median, and with naive forecasters, fixed weights that can actually be deployed are worse than HuberLOP_con on both metrics. Of the CRPS gain of per-forecast weights over the hindsight weights, 62–81% comes from the outer 0.1 and 0.9 levels, which MASE does not see. The fixed weights of Table 5 average each member's weight over all forecasts, later ones included, whereas fixed weights from earlier forecasts only, which a deployed system can compute, are worse than HuberLOP_con in both metrics on all three benchmarks, by up to 5.47% in CRPS and 2.19% in MASE (Table R3, Appendices C.6 and C.13.3).

Table R3: With three naive forecasters, HuberLOP_con beats the deployable causal fixed weights in both CRPS and MASE on all three benchmarks. Results with the three naive forecasters added (↓), with SQL in place of CRPS on fev-bench. Fixed weights average the weights of HuberLOP_con over all forecasts (hindsight, Table 5) or over earlier ones only (causal). Bold marks the best among methods available at deployment.

MethodAvailable at deploymentGIFT-Eval CRPSGIFT-Eval MASETIME CRPSTIME MASEfev-bench SQLfev-bench MASE
Fixed weights, hindsightNo0.47330.69180.55410.65590.51900.6577
Fixed weights, causalYes0.49720.70910.56060.66130.52780.6656
HuberLOP_con (ours)Yes0.47010.69360.55040.65730.51670.6617
AuthorsOpenReview ↗

Author's Response to Reviewer hHtW (3/3)

Q3: lines 200-201: the bound of 1/K applies to the change in the CDF at a fixed value. I don't think it means an outlier only widens the forecast without shifting its quantiles. Please try to qualify this statement.

Thanks for your comments, which have helped improve the clarity of our paper. We replaced the sentence with Proposition 1, which shows that under LOP an outlier with enough weight drags the outermost quantile on its side, and that Huber weights limit this effect by shrinking the outlier's share. If members with total weight π deviate arbitrarily and G pools the rest, the pooled τ-quantile is at least G⁻¹((τ − π)/(1 − π)) for τ > π and at most G⁻¹(τ/(1 − π)) for τ ≤ 1 − π. With seven members, one outlier has π = 1/7, so the upper bound vanishes at the 0.9 level, which follows an upward outlier. A Huber weight falls with distance, so π shrinks and both bounds approach the quantiles of G at every level where they are unique. We revised Sections 1 and 3.2 accordingly (proof in Appendix A.8).

Once again, we appreciate the reviewer highlighting valuable discussion points in our study. We hope our response has clarified the contents of the paper, and we are more than willing to engage in further discussion regarding any aspects that remain unclear.


References

[1] "fev-bench: A Realistic Benchmark for Time Series Forecasting.", In Advances in Neural Information Processing Systems (NeurIPS), Evaluations & Datasets Track, 2026

[2] "Probabilistic Forecast Aggregation with Statistical Depth.", In European Journal of Operational Research, 2026

[3] "Trimmed Opinion Pools and the Crowd's Calibration Problem.", In Management Science, 2014

Reviewer_LW5wOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies label-free and training-free aggregation of zero-shot time-series foundation models, where the combiner observes only the current predictive quantiles. It proposes HuberLOP, which computes Huber weights from each model’s standardized deviation from a robust cross-model center and uses these weights in a linear opinion pool. A conservative variant takes the minimum weight across quantile levels. Experiments involve seven TSFMs on GIFT-Eval and TIME, with analyses of aggregation rules, model subsets, forecast-dependent weighting, and contamination by naive forecasters. The main findings are that pool composition substantially affects performance, improvements over uniform LOP are small on curated pools, and the conservative variant substantially reduces degradation under the tested contamination settings.

Critical strengths
  1. A practical aggregation method with minimal deployment requirements. HuberLOP requires neither labeled calibration data nor additional calls to the underlying forecasting models. Its forecast-dependent weights allow a model to be discounted locally without excluding it everywhere. The conservative variant shows substantial robustness in the reported contamination experiment: CRPS degradation is approximately 0.9% on both benchmarks, compared with 14.0% and 12.0% for uniform LOP.
  2. Broad empirical analysis within the forecast-only setting. The study covers two benchmarks, seven TSFMs, all 120 multi-model subsets, and a broad collection of quantile-based, trimmed, depth-based, angular, and radial aggregation settings. Comparisons with fixed weights and context-backtesting strategies further help identify when the proposed approach is useful. This empirical characterization is a meaningful contribution independently of the proposed weighting rule.
Critical weaknesses
  1. The methodological contribution over existing robust aggregation methods is limited. The method combines standard Huber reweighting, MAD-based scaling, and weighted linear opinion pooling. Existing trimmed opinion pools [1] and statistical-depth methods [2] already use the current forecasts’ centrality or outlyingness to perform robust aggregation without historical accuracy labels. The distinctive contribution is therefore the particular construction and collapse of quantile-level Huber weights, rather than label-free adaptive aggregation itself. Although the paper acknowledges and compares these predecessors, it does not establish a sufficiently substantial conceptual or theoretical advance over them. The “select forecasts rather than models” framing also does not clearly distinguish the method from existing forecast-dependent robust aggregation.
  2. The evidence for a consequential advantage over strong alternatives remains limited. On curated pools, HuberLOP improves CRPS over uniform LOP by only approximately 0.29% and 0.25%, and it does not consistently outperform the depth-weighted and trimmed baselines in Table 2. On the best fixed subsets, Table 9 reports essentially identical performance to LOP on GIFT-Eval and only a very small difference on TIME. Confidence intervals or source-group-aware significance analyses are not provided to establish the reliability of these small differences. Under contamination, the conservative variant is promising, but the quantile median already limits degradation to 1.8% and 1.5%. The stress tests mainly involve naive forecasters or an artificially displaced member, leaving correlated model failures and a correct minority insufficiently examined. Moreover, the broad set of forecast-only baselines evaluated on curated pools is not reported equally comprehensively under contamination. These limitations make it difficult to determine whether the proposed rule offers a substantial, general advantage over established robust alternatives. Finally, the paper does not directly compare against source-trained FFORMA-style combiners. Although these methods fall outside the strict training-free setting, they can be applied to unseen series without target labels or mandatory retraining.

[1] Victor Richmond R. Jose, Yael Grushka-Cockayne, and Kenneth C. Lichtendahl, Jr. Trimmed opinion pools and the crowd’s calibration problem. Management Science, 60(2):463–475, 2014. [2] James W. Taylor. Probabilistic forecast aggregation with statistical depth. European Journal of Operational Research, 328(2):460–476, 2026.

Questions for authors
  1. Statistical reliability: Can you provide paired confidence intervals for the CRPS differences against uniform LOP and the strongest depth-based and trimmed baselines, accounting for configurations derived from the same source dataset? This would help establish whether the curated-pool gains are reliable.
  2. Dependence on pool composition: Beyond the best-subset results in Table 9, how does HuberLOP compare with strong robust baselines across subsets excluding Moirai-base? This would clarify whether its benefit extends beyond attenuating the weakest member of the selected pool.
  3. Robustness relative to the closest competitors: What are the contamination results for the strongest depth-based settings from Appendix B.7? Do the proposed variants remain competitive when several members share a bias or when the most accurate member disagrees with the majority?
  4. Scope of the deployment claim: Should the recommendation be understood strictly as applying to forecast-only aggregation? FFORMA-style combiners can be trained on source datasets and applied to new series without target labels or mandatory retraining. Please clarify this distinction when motivating the setting and discussing learned alternatives.
7 comments
AuthorsOpenReview ↗

Author's Response to Reviewer LW5w (1/3)

We sincerely thank the reviewer for your insightful comments, which have helped us improve the paper quality significantly. Below, we are more than willing to clarify the reviewer's concerns one by one.

Q1: Statistical reliability: Can you provide paired confidence intervals for the CRPS differences against uniform LOP and the strongest depth-based and trimmed baselines, accounting for configurations derived from the same source dataset? This would help establish whether the curated-pool gains are reliable.

We thank the reviewer for this suggestion, which has helped us establish the reliability of our results more rigorously. HuberLOP_con is never significantly worse than any baseline of Table R1 on the clean (curated) pool, and it is significantly better than all of them once bad models enter. As requested, we computed paired 95% confidence intervals by resampling source datasets (Table R1), and we added a third benchmark, fev-bench [5], now in the main tables, on which HuberLOP_con has the lowest clean-pool error of all 36 settings. Small clean-pool gaps are expected, because a label-free rule limits how far one model moves the combined forecast and a clean pool has little to limit. The gain of HuberLOP over LOP that the reviewer cited is positive on both benchmarks and significant on TIME, 0.25% [0.12, 0.38]. With the three naive forecasters of Table 4, HuberLOP_con is significantly better than every comparator, 1.12%, 0.96%, and 1.46% below the strongest, the quantile median, and rises the least of all 36 settings on all three benchmarks (Appendices C.1, C.2, and C.13).

Table R1: HuberLOP_con is never significantly worse on the clean pool and is the best once naive forecasters are added. CRPS on GIFT-Eval and TIME and scaled quantile loss on fev-bench (↓). Parentheses give the method's error minus that of HuberLOP_con, with paired 95% confidence intervals over source datasets. Best is the most accurate of Taylor's ten depth settings or of the six trimmed settings in each condition. Bold marks the best results and differences in favour of HuberLOP_con.

MethodGIFT-EvalTIMEfev-bench
Clean pool
LOP0.4657 (0.0000 [−0.0036, 0.0035])0.5483 (0.0027 [0.0006, 0.0047])0.5071 (0.0044 [−0.0016, 0.0143])
Depth pool, best [2]0.4648 (−0.0009 [−0.0031, 0.0012])0.5464 (0.0007 [−0.0001, 0.0017])0.5040 (0.0013 [−0.0019, 0.0060])
Trimmed pool, best [1]0.4657 (0.0000 [−0.0014, 0.0017])0.5466 (0.0010 [0.0000, 0.0021])0.5041 (0.0014 [−0.0003, 0.0040])
Quantile median0.4672 (0.0015 [0.0005, 0.0023])0.5474 (0.0018 [0.0010, 0.0027])0.5048 (0.0021 [0.0010, 0.0040])
HuberLOP_con (ours)0.46570.54560.5027
Three naive forecasters added
LOP0.5308 (0.0608 [0.0448, 0.0818])0.6140 (0.0636 [0.0392, 0.0908])0.6063 (0.0895 [0.0503, 0.1628])
Depth pool, best [2]0.4786 (0.0085 [0.0056, 0.0117])0.5590 (0.0086 [0.0055, 0.0127])0.5290 (0.0123 [0.0065, 0.0267])
Trimmed pool, best [1]0.4779 (0.0079 [0.0047, 0.0119])0.5575 (0.0071 [0.0042, 0.0102])0.5281 (0.0114 [0.0065, 0.0226])
Quantile median0.4754 (0.0053 [0.0031, 0.0078])0.5557 (0.0054 [0.0029, 0.0081])0.5244 (0.0077 [0.0040, 0.0159])
HuberLOP_con (ours)0.47010.55040.5167

W2: Under contamination, the conservative variant is promising, but the quantile median already limits degradation to 1.8% and 1.5%.

Thanks for letting us know your concern regarding performance. HuberLOP_con is better than the median both on the clean pool and under contamination, and fewer of its forecasts degrade sharply. With the naive forecasters, its error rises by 0.93% and 0.87% on GIFT-Eval and TIME, about half the median's 1.75% and 1.52%, and by 2.79% against 3.89% on fev-bench (Appendices C.2 and C.13). Its error is significantly lower than the median's on all three benchmarks, both on the clean pool and with the naive forecasters (Table R1). Fewer of its forecasts also lose more than 10%, 9.4% against 13.7% on GIFT-Eval (Appendix C.11), and when a member fails after the weights are set, its error stays below the median's on all three benchmarks (Appendices C.10 and C.13).

AuthorsOpenReview ↗

Author's Response to Reviewer LW5w (2/3)

Q2: Dependence on pool composition: Beyond the best-subset results in Table 9, how does HuberLOP compare with strong robust baselines across subsets excluding Moirai-base? This would clarify whether its benefit extends beyond attenuating the weakest member of the selected pool.

We appreciate the reviewer for providing a valuable discussion point. HuberLOP keeps its edge over the robust baselines without Moirai-base, so its benefit is not only the down-weighting of that member. Over the 42 subsets of three to six TSFMs without Moirai-base, HuberLOP has the lowest average error on all three benchmarks and beats the quantile median and the trimmed pool on nearly every subset (Table R2). The robustness of HuberLOP_con does not depend on Moirai-base either, as its error rises by less than 1% on GIFT-Eval and TIME and 2.5% on fev-bench when naive forecasters join the six remaining members, against 23% for LOP on fev-bench. The best fixed subsets of Table 8 (Table 9 in the submission) are chosen with target labels, whereas HuberLOP recovers 54% of their CRPS gain on GIFT-Eval without choosing any model. A pool chosen in advance with labels of other datasets is at most 0.22% better than the better of our two variants, and on fev-bench HuberLOP_con is below the profiled pools of three and five members (Table 3 in Section 4.3, Section 4.4, Appendices C.2, C.5, and C.13).

Table R2: Without Moirai-base, HuberLOP still has the lowest error on all three benchmarks. Average error over the 42 subsets of three to six TSFMs (↓). Win rate is the fraction of the 42 subsets on which HuberLOP has a lower error than that method (1.00: better on every subset). Bold marks the best results.

MethodGIFT-EvalWin rateTIMEWin ratefev-benchWin rate
LOP0.46700.690.54881.000.50700.88
Depth pool [2]0.46700.740.54850.880.50620.69
Depth pool, squared weights [2]0.46730.830.54840.620.50600.64
Trimmed pool [1]0.46941.000.54941.000.50720.98
Quantile median0.46971.000.54971.000.50740.98
HuberLOP (ours)0.4669–0.5483–0.5059–

Q3: Robustness relative to the closest competitors: What are the contamination results for the strongest depth-based settings from Appendix B.7? Do the proposed variants remain competitive when several members share a bias or when the most accurate member disagrees with the majority?

Thanks for your thoughtful feedback, which has helped us better position our paper compared to the related works. HuberLOP_con degrades the least when bad models enter, ahead of all ten depth settings of Appendix B.6 (B.7 in the submission), and it stays first or second when several members share a bias. With the three naive forecasters, its error rises by less than half as much as that of the most robust depth setting on GIFT-Eval and TIME (Table R3). When one to three members share a shift, it ranks first of nine robust rules in 15 of 18 cases and second in the rest, and when a member silently falls back to its last value after the weights are set, its error rises by less than 0.3% on average on GIFT-Eval and TIME (Figure 5). Without labels, a correct member that disagrees with the majority looks the same as a wrong one, so every agreement-based rule moves away from it, as Synapse [4] also notes. HuberLOP_con therefore follows the majority. If the disagreeing member is correct, the combination gives up what it would have added, and if it is wrong, it can pull the combination only a little, whereas under LOP a wrong member keeps its full share however far it goes. We state this trade-off as a limitation in Section 5 (Appendices C.2, C.4, and C.13).

Table R3: HuberLOP_con degrades the least when the three naive forecasters are added, ahead of all ten depth settings. Error (↓) and its increase (%) from the clean pool. Least and most robust are the depth settings with the largest and smallest increase among Taylor's ten. Bold marks the best results.

MethodGIFT-EvalTIMEfev-benchIncrease (%)
LOP0.53080.61400.606313.98 / 11.98 / 19.55
Depth pool, least robust [2]0.52290.60060.594411.96 / 9.81 / 17.79
Depth pool [2]0.50600.58690.57008.87 / 7.33 / 12.91
Depth pool, most robust [2]0.47860.55900.52902.02 / 1.83 / 4.07
HuberLOP_con (ours)0.47010.55040.51670.93 / 0.87 / 2.79
AuthorsOpenReview ↗

Author's Response to Reviewer LW5w (3/3)

Q4: Scope of the deployment claim: Should the recommendation be understood strictly as applying to forecast-only aggregation? FFORMA-style combiners can be trained on source datasets and applied to new series without target labels or mandatory retraining. Please clarify this distinction when motivating the setting and discussing learned alternatives.

Thanks for your comments, which have helped improve the clarity of our paper. Yes, our recommendation is for forecast-only aggregation, and its per-forecast weights keep HuberLOP_con robust when a member fails after deployment. FFORMA [3] needs no target labels either, but it learns its weights on source data, must be retrained when the pool changes, and cannot weight newly added members. We compared HuberLOP_con with the best of eight FFORMA settings on each benchmark, a choice that favours FFORMA, trained on GIFT-Eval and applied unchanged to TIME and fev-bench.

HuberLOP_con outperforms FFORMA when a member fails after training. When one TSFM outputs only its last observation, the error of HuberLOP_con rises by at most 1% on average, against 3.5 to 6% for FFORMA (Table R4). On clean pools, HuberLOP_con matches FFORMA delivered through LOP without any training and beats FFORMA's own quantile-mean combination in all eight comparisons on GIFT-Eval and TIME, significantly in five. We have clarified this in the introduction and Table 1 (Appendices C.8 and C.10).

Table R4: HuberLOP_con is nearly unaffected when a TSFM fails after the weights are set, while FFORMA degrades more than LOP. CRPS, or SQL on fev-bench (↓), averaged over the seven TSFMs failing in turn, and its rise (%) from no failure.

MethodGIFT-EvalTIMEfev-benchIncrease (%)
LOP0.48300.56570.53513.70 / 3.17 / 5.51
FFORMA [3]0.48430.56470.53564.47 / 3.46 / 6.01
HuberLOP_con (ours)0.46700.54690.50770.28 / 0.23 / 1.00

W1: The methodological contribution over existing robust aggregation methods is limited. The method combines standard Huber reweighting, MAD-based scaling, and weighted linear opinion pooling. Existing trimmed opinion pools [1] and statistical-depth methods [2] already use the current forecasts' centrality or outlyingness to perform robust aggregation without historical accuracy labels. The distinctive contribution is therefore the particular construction and collapse of quantile-level Huber weights, rather than label-free adaptive aggregation itself. Although the paper acknowledges and compares these predecessors, it does not establish a sufficiently substantial conceptual or theoretical advance over them. The "select forecasts rather than models" framing also does not clearly distinguish the method from existing forecast-dependent robust aggregation.

We agree that each component is standard, and we regard this as a strength. This makes HuberLOP_con practical, and it differs from depth and trimmed pools in the property that matters when bad models enter, since a member's weight keeps falling as it moves away and no number to cut is fixed in advance. Depth weights follow the members' ranks, so they stop cutting a member once it is outermost, and trimmed pools must fix how many members to cut. HuberLOP_con is therefore significantly better than all ten depth and six trimmed settings when naive forecasters enter, on fev-bench as well (Tables R1 and R3). By Proposition 1, a member can move the pooled quantiles only as far as its weight share allows, so this falling weight is what limits a bad member's damage. Our study over all 120 subsets, which the reviewer recognized in Strength 2, shows why this matters, because the pool moves accuracy far more than the rule yet cannot be vetted on a new series. The rule needs no labels, no training, and no extra TSFM calls, and it adds 3.6 ms to the 123.5 ms of TSFM inference (Appendix C.9). Section 3.3 now notes that these pools also weight each forecast.

Once again, we appreciate the reviewer highlighting valuable discussion points in our study. We hope our response has clarified the contents of the paper, and we are more than willing to engage in further discussion regarding any aspects that remain unclear.


References

[1] "Trimmed Opinion Pools and the Crowd's Calibration Problem.", In Management Science, 2014

[2] "Probabilistic Forecast Aggregation with Statistical Depth.", In European Journal of Operational Research, 2026

[3] "FFORMA: Feature-based forecast model averaging.", In International Journal of Forecasting, 2020

[4] "Synapse: Adaptive Arbitration of Complementary Expertise in Time Series Foundational Models.", In Transactions on Machine Learning Research, 2026

[5] "fev-bench: A Realistic Benchmark for Time Series Forecasting.", In Advances in Neural Information Processing Systems (NeurIPS), Evaluations & Datasets Track, 2026

Comments on the paper

2 comments
AuthorsOpenReview ↗

Response Summary to All Reviewers

Dear Reviewers and Area Chair,

We sincerely thank all the reviewers for their constructive feedback. We are grateful that they found our label-free and training-free combination practical (LW5w, L1b6), our evaluation over all 120 pools of seven TSFMs a contribution in its own right (LW5w, hHtW, L1b6), and the conservative variant robust to bad models (LW5w, hHtW).

HuberLOP_con is a practical, robust default for combining zero-shot TSFMs on a new series. It needs no labels, no training, and no extra TSFM calls, and its combination step takes a few milliseconds next to the inference of the models. On a new series no one can check in advance whether every model in the pool is sound, so a combination rule should lose little when the pool is clean and hold up when it is not. The added results show that HuberLOP_con does both.

A third benchmark confirms these results. We added fev-bench to the main results in Section 4, and there HuberLOP_con has the lowest error of all 36 combination settings we compare, on the clean pool and with bad models.

The main concerns and our responses are as follows.

  • The clean-pool results are reliable (LW5w, hHtW, L1b6). With paired confidence intervals over source datasets, HuberLOP_con is never significantly worse than any label-free baseline on any of the three benchmarks.
  • It is the most robust when bad models enter, ahead of every depth-based setting (LW5w, hHtW). With three naive forecasters in the pool, its error rises by less than 1% on GIFT-Eval and TIME, while that of LOP rises by 12 to 14%, and every one of Taylor's ten depth settings degrades more than HuberLOP_con.
  • It is also the most robust when a member fails after deployment (LW5w, L1b6). If one model silently breaks after the weights are set, rules that fix their weights in advance, FFORMA and backtest weights included, keep its share and degrade several times more than HuberLOP_con, which reads the current forecasts (new Figure 5).
  • Context-based methods complement it rather than replace it (L1b6). Backtesting doubles the TSFM calls and leaves a failing member its share, and HuberLOP_con + backtest, which multiplies the two weights, gives the lowest error. The ensemble of TimeCopilot is the quantile median, which HuberLOP_con beats.
  • Its simplicity is deliberate (LW5w, L1b6). HuberLOP_con is built from standard robust estimators, and Proposition 1 shows that discounting an outlying member's share bounds how far it can move the pooled quantiles. Like every rule that relies on agreement, it cannot single out a correct minority without labels, which we state in Section 5.

We have uploaded the revised manuscript, with the passages our responses refer to in blue and the new analyses collected in Appendix C.

  • fev-bench as a third benchmark in Tables 2–5 and Figures 1–5 (details in Appendix C.13)
  • Definition of labels and Proposition 1 in Section 3, Figure 2 redrawn on all forecasts and pool selection on fev-bench (Table 3) in Section 4.3, and member failures with a robustness summary in Section 4.4 (Figure 5)
  • Paired confidence intervals in Appendix C.1, robustness tests in Appendix C.2–C.6, C.10, and C.11, and context-based methods, FFORMA, and cost in Appendix C.7–C.9 and C.12

Once again, thank you for your time and effort in reviewing our work. We look forward to your continued feedback.

Best regards, Authors

AuthorsOpenReview ↗

Summary of the Discussion

Summary of the Discussion

Dear Area Chair and Reviewers,

We thank all reviewers for a constructive discussion that clearly strengthened the paper.

Our paper offers a practical and robust default for combining zero-shot TSFMs, grounded in well-established robust statistics and backed by an extensive empirical study. HuberLOP_con needs no labels, no training, and no extra TSFM calls. We evaluate it on three benchmarks, over every pool of seven TSFMs, against label-free baselines including all depth and trimmed settings, and under bad models, shared shifts, and member failures after deployment. The study shows that pool composition drives accuracy yet cannot be checked on a new series, and HuberLOP_con matches the strongest label-free baselines on curated pools and degrades the least once bad models enter. All three reviewers recognized this practical value and breadth, and LW5w described the empirical study as a meaningful contribution in its own right.

Each review after the discussion.

  • hHtW kept a positive rating, noting that our response "substantially addressed" their concerns and that the expanded evaluation, including fev-bench, makes the practical contribution clearer.
  • LW5w wrote that the paired confidence intervals and new comparisons "strengthen the empirical evidence and address several" of their original concerns. We answered the remaining ones by showing that HuberLOP_con matches the best depth setting on curated pools with one fixed setting, that it is significantly better than the strongest baselines once bad models enter, and that the correct-minority case the reviewer raised is rare and costs little.
  • L1b6 asked for comparisons with Synapse, TimeCopilot, and backtest weights, a cost analysis, a definition of labeled data, and a consistent Figure 2, and all of these are now in the revised manuscript. HuberLOP_con adds only a few milliseconds to inference, is more robust when a member fails after deployment, and gives the lowest error when combined with backtest weights.

We hope this summary helps the final assessment.

Best regards,

The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.