acceptodds
Under review as a conference paper at ICLR 2027

Better on Average, Worse Locally: Interval Guarantees for Online Forecast Adaptation

Abstract

An adaptive forecaster can improve total error while worsening a recent interval. We study a causal guard that bounds the eventual excess MAE of every interval of issued multi-horizon forecasts relative to a frozen reference, including intervals whose targets are not yet observed. For absolute loss, accounting jointly for the predictions that share a target makes each interval's feasible set a translated ball—exactly the forecasts that keep the guarantee for every completion of the pending targets—and under regular horizon- feedback the history compresses into two statistics and at most active constraints. A constructed example separates this exact accounting from any independent certificate by a factor of order . Replays on temperature, Exchange Rate, multivariate transit, Weather and electricity, including modern online adapters, show that unguarded adaptation often lowers mean error while a single interval absorbs a regression that grows –-fold with aggressiveness. The certificate removes it at a mean-MAE cost that is negligible or negative on temperature and material, but tunable through the allowance, on transit, Weather and electricity. Cheap independent guards match the exact guard in error: on the audited streams exact accounting accepts – more of each candidate's correction without changing realized error, so the separation, genuine in the worst case, is operationally inert. We contribute an exact characterization of what every-interval protection allows and a measured account of what it costs.

Primary area:
transfer learning, meta learning, and lifelong learning
Keywords:
online time series forecasting, test-time adaptation, delayed feedback, safe adaptation, baseline guarantees

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_DyegOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper studies an online adaptive forecasting problem.

Critical strengths
Critical weaknesses

A major weakness for this paper is clarity. The exposition is extremely obfuscated and uses imprecise language, unexplained jargon, and incoherent logical structure. This paper does not clearly state the problem setup or clearly describe the methodology or results.

The lack of clarity begins in the abstract.

“We study a causal guard that bounds the eventual excess MAE of every interval of issued multi-horizon forecasts relative to a frozen reference, including intervals whose targets are not yet observed”

“Weather and electricity, including modern online adapters, show that unguarded adaptation often \emph{lowers} mean error while a single interval absorbs a regression that grows 99--4343-fold with aggressiveness.”

What are guarded predictions? What is aggressiveness? Neither of these are ever defined.

The following main paragraphs describing contributions in the introduction. Many of these terms - e.g., "prefix constraint," "H-step feedback," and "unguarded adaptation" are never clearly explained.

“(i) Geometry. A prefix constraint is exactly a translated ℓ1 ball— necessary and sufficient for every completion of the pending targets—and every-interval protection is an intersection of such balls, so any guard that keeps the guarantee, including the cheap independent one, must issue forecasts inside this set”

“State and separation. With regular H-step feedback, completed history compresses into two scalars and at most Hconstraints.”

“Measurement. Unguarded adaptation exposes a systematic failure: in 57 of 117 temperature record–allowance cells it lowers average MAE while a single interval absorbs an excess of up to 494×its burst allowance (median 36×), so an averaged report calls it an improvement”

Questions for authors
Reviewer_4vPrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

This paper addresses a operational vulnerability in online test-time adaptation (TTA) for time-series forecasting, i.e., an adaptive model can reduce global average error over a deployment stream while inflicting severe, concentrated regressions over local contiguous intervals.

Critical strengths
  1. The authors demonstrate exceptional scientific honesty by strictly separating exploratory from pre-registered experiments and transparently reporting that their exact guard yields no realized MAE advantage.
  2. The paper derives an exact translated ℓ1\ell_1-ball feasible geometry and an O(H2N)O(H^2N)-space running-minimum compression over unbounded real targets without requiring stationarity or uncertainty models.
Critical weaknesses
  1. The paper is obscure and hard to understand. It basically fails to clearly state its notable contributions and methodology. Furthermore, many existing test-time adaptation methods are not discussed in this paper.
  2. The theoretical premise is a "fixed nominal budget", but Section 3.1 admits that the number of tokens, parameter counts and budgets can vary. However, the paper identifies control measures that are identical to the budget, so the claim that "the framework explains the empirical results" does not hold.
Questions for authors
  1. Why does the paper focus mainly on MAE instead of MSE? What is the reason?
  2. The survey of related work in the paper is incomplete. Why can the method proposed in this paper cover all adapters, including online ones?
  3. While guarding is nearly free on noisy or over-aggressive spectral updates on ETT, its cost on strong modern adapters in multivariate domains is steep. Can you explain these phenomena
Reviewer_WX31OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

This paper considers multihorizon prediction problems in which we receive two different forecasts, a baseline btb_t and a second forecast ctc_t. It then develops methods for constructing a convex combination of these predictions that is guaranteed to obtain loss no worse than the baseline up to some given error budget. These guarantees are designed to hold over any local time interval.

Critical strengths
  • Multihorizon prediction is a challenging and interesting problem. Developing ensembling methods for combining multiple predictors is also of broad interest.
Critical weaknesses
  • This paper is very poorly written. In my view, even if the content of this paper was strong (which unfortunately I don't think it is), in the current research landscape it should be rejected for this reason alone.
  • This paper develops methods that don't appear to use the data at all and instead consider the worst case value of the outcomes. I don't know why this is of interest. Indeed, I would expect standard adaptive regret algorithms that provide similar guarantees over local time intervals and actually use the signal from the data to outperform the methods proposed here.
Questions for authors

I don't have any questions for the authors.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.