- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
This paper proposes LIRA, a post-hoc correction method for frozen time-series forecasting models. The key observation is that forecasting residuals contain recurring patterns associated with target cycle positions and forecast leads, which can persist after backbone training and transfer to future predictions. LIRA corrects forecasts through two components. A recurring error lookup estimates systematic residual patterns. A channel-shared linear head then predicts the remaining residuals from the observed history and original forecasts. The method also supports online adaptation by updating lookup statistics using newly observed residuals without modifying the backbone or remainder head. Experiments with five task-specific models and four pretrained foundation models report consistent forecasting improvements.
Critical strengths
- Consistent empirical improvements across different models, including backbones & FMs.
- A simple framework for post-hoc correction. LIRA separates residual correction into recurring error estimation and input-conditioned remainder prediction. It can be applied to both frozen backbones and FMs, and its online adaptation only requires updating historical statistics without gradient optimization.
Critical weaknesses
The insights of the two components are not sufficiently established.
The core idea of LIRA is that forecasting residuals remains learnable structures, and that these structures include two parts, one can be captured by recurring errors determined by channel and cycle position, the other one can be learned from the input window and the prediction. However, neither theoretical nor empirical explanations are well established for: why such structures can be learned by post-hoc designs but cannot be learned by the model itself. The paper tries to use Figure 6 to make the explanations, but the experimental design and analysis are not sufficient or clear. As for the lookup component, it mainly brings two sources of new information, channel id & calendar position. Appendix C.10 shows that the improvement of LIRA is much less for calendar-aware backbones, which partially support the importance of calendar position. More experiments are needed to confirm which design contributes to the improvements on these backbones. As we go deeper in Figure 6a, "lookup + raw head" is a weired setting, where the target of raw head is real error ( without lookup) but inference adds the output of raw head and lookup. The training output space and inference output space are different. More surprisingly, it brings a huge improvement over backbone, lookup only and raw head, which means it does not ruin the prediction, but combines the benefit of the two designs. And the gap to the full LIRA is quite small. Such results need explanation. The only valid explanation that occurs to me is that, the outputs / residual error space of lookup and raw head are orthogonal, which means the outputs will not interfere with each other. As for the head component, it needs explanation about why head component can learn information that the backbone itself fails to learn. The head component uses only the original input the backbone forecast, introducing no additional predictive information. Figure 6b further strengthen my question, as more than half the MSE gain can be recovered by adding a scalar bias learned by head component.
Questions for authors
- Statistics on the learned and .
- Statistics on how orthogonal are between the output of lookup and the output of raw head / head component, as well as the scalar expectation of the two.
- A series experiments with settings.
- Head input (3): input only, forecast only, input + forecast.
- head output (2): vector, scalar offset only
- training recipe (2): frozen backbone + learnable linear head, joint fine-tuning. For joint fine-tuning, the data flow keeps the same. Take forecast only + joint fine-tuning as an example, it is the same as appending a residual linear layer.
- Analysis on the aforementioned results.
6 comments
Response to Reviewer tkJc (2/4)
w3, q1, q2 — Combining the lookup and head, the performance gap, and remainder learning (2/2)
2. Remainder learning improves the combined shape correction
The raw target includes recurring errors that the lookup also corrects, whereas LIRA trains the head on the remainder, . To locate the additional gain, we decompose the difference in combined MSE gain into offset and shape contributions.
Entries are differences between LIRA and Lookup + raw head, in percentage points normalized by the original backbone MSE.
| Aggregation | Offset gain difference (pp) | Shape gain difference (pp) |
|---|---|---|
| Mean over 100 settings | −0.06 | +0.49 |
The additional gain is concentrated in shape, with a small reduction in offset gain.
To separate the training-target effect from weight-decay selection, we compared the targets at each fixed weight decay, holding the backbone forecasts, lookup, α, initialization, batch order, and calibration procedure fixed. The table reports the mean relative MSE reduction over Lookup + raw head, using its MSE as the denominator. The last column selects weight decay separately for each target on validation data.
| Weight decay | 0 | 0.001 | 0.01 | 0.1 | 0.3 | Selected on validation |
|---|---|---|---|---|---|---|
| LIRA MSE gain (%) | 0.65 | 0.51 | 0.44 | 0.35 | 0.21 | 0.45 |
The mean gain is positive at every tested weight decay and decreases as regularization strengthens. Thus, the improvement cannot be attributed solely to weight-decay selection; changing the training target also contributes.
3. q1. Applied α and γ coefficients
The comparisons above use calibrated lookup and head corrections. To show how strongly each component is applied, we report the lookup coefficients α and head coefficients γ below.
These coefficients are fitted and shrunk on validation data, then fixed for testing. They are defined per channel and lead group, with upper bounds of 1.5 for and 1 for .
| Coefficient | Mean | Median | 25th–75th percentiles | Zero coefficients (%) |
|---|---|---|---|---|
| α cycle | 0.81 | 0.85 | 0.56–1.09 | 5.1 |
| α lead | 0.76 | 0.85 | 0.30–1.10 | 11.9 |
| α interaction | 0.86 | 0.90 | 0.57–1.20 | 7.1 |
| γ raw head | 0.63 | 0.77 | 0.22–1.00 | 10.2 |
| γ remainder head | 0.63 | 0.76 | 0.25–1.00 | 7.3 |
The coefficients allow correction strength to vary across groups. A zero coefficient disables the corresponding component within that group.
Thank you for your careful review. We have revised RQ2 accordingly in the revised paper.
Response to Reviewer tkJc (3/4)
w1, w4, q3, q4 — What the head learns, and whether the backbone needs further training (1/2)
You asked why a lightweight head can recover errors that a more expressive backbone has not learned. Our experiments show that a head can correct predictable residual error with the backbone fixed, with much of its MSE gain coming from window-specific level corrections.
Why a post-hoc correction can help. Let denote the backbone's complete prediction-time input, , and . For a correction based on prediction-time information , define . For each fixed channel and lead, expectations are over forecast windows with all fitted parameters held fixed:
$ \mathrm{MSE}(r)-\mathrm{MSE}(r-u) = \mathbb E[m(Z)^2]-\mathbb E[(u-m(Z))^2].$
The first term is the potential gain from predictable residual error; the second is the error of the learned correction relative to the ideal correction .
- Lookup. With comprising channel, cycle position, and lead, the lookup can provide information beyond when cycle position is not determined by . Even a predictor optimal for alone can then leave phase-dependent errors.
- Head. With and , the head adds no information beyond , but can remain nonzero. If , no correction using these inputs can reduce expected MSE. A positive expected head-only gain therefore requires input-predictable residual error.
Even when a backbone can represent such corrections, training on finite data does not guarantee that its fitted forecasts equal on future windows. The observed head-only gains provide empirical evidence that some remaining error is recoverable with a frozen backbone, although they do not isolate its specific cause.
For the head's additional gain after lookup correction , the same identity applies to , with .
The role of the scalar offset. The scalar component in Figure 6b (revised Table 7) shifts all leads by an offset that varies across windows and channels. Its contribution indicates that much of the head's MSE gain comes from correcting forecast levels. Part 2 separately evaluates heads trained directly to predict these offsets.
Further backbone training. Part 2 isolates the head's contribution and compares frozen with joint training. For DLinear, an input-only linear head with channel-specific calibration also improves forecasts. Here, we separately compare complete LIRA with backbone-only fine-tuning of the same checkpoints using training and validation targets. The evaluation covers 35 dataset–backbone combinations. Entries report mean MSE gains (%) relative to the original backbone; † denotes online updates.
| Backbone | Fine-tuning | Fine-tuning† | LIRA | LIRA† |
|---|---|---|---|---|
| PatchTST | 0.70 | 0.56 | 6.15 | 7.77 |
| iTransformer | 0.39 | 0.64 | 5.36 | 6.70 |
| DLinear | 5.93 | 7.66 | 10.45 | 11.72 |
| TimeMixer | 0.45 | −0.82 | 4.00 | 5.80 |
| TimeBase | 0.20 | 0.26 | 10.76 | 11.86 |
| Mean | 1.53 | 1.66 | 7.34 | 8.77 |
Under these procedures, complete LIRA yields larger average gains than backbone-only fine-tuning for every listed backbone.
Response to Reviewer tkJc (1/4)
Thank you for your thoughtful feedback and recognition of our consistent gains, simple post-hoc design, and online adaptation. We have added coefficient and output statistics and comparisons of head inputs and training strategies.
w3, q1, q2 — Combining the lookup and head, and remainder learning (1/2)
You asked why a raw-residual head works well with the lookup and why this combination performs nearly as well as LIRA. We should have more clearly distinguished why the corrections work together from what remainder learning adds.
1. Understanding the combined gains and the remaining performance gap
As you noted, the raw head is trained on r=Y−Y^, although inference combines it with the lookup. In Lookup + raw head, the channel–lead-group coefficients γ are fitted and weight decay is selected using validation error after lookup correction; the head's training target remains the raw residual. Even the naive sum, which keeps the head and coefficients from raw-head-only unchanged, improves on either correction alone.
1.1. Where does the performance difference arise?
Let B be the lookup correction, u=γP the applied head correction, and G the MSE gain (%) over the backbone.
G(B+u)=G(B)+G(u)−C∥r∥22⟨B,u⟩×100.
All rows below share the lookup. G(u) applies each method's calibrated head alone, without refitting; C is the interaction term in percentage points (pp). A positive C subtracts from the sum of the standalone gains, while a negative C adds to it. The inner product and norm are computed over all test origins, leads, and channels for each seed and setting then averaged.
Exact orthogonality in every run would yield 2.89 + 4.17 = 7.06% for the unchanged corrections. The naive sum retains 85% of this additive benchmark; the remaining 15% (about 1.04 pp) is the interaction penalty. It therefore still improves on either correction alone.
Post-lookup γ fitting and weight-decay selection reduce this penalty from 1.043 to 0.439 pp, raising the combined gain from 6.013% to 6.359%. With LIRA, C becomes −0.318 pp. Compared with Lookup + raw head, its standalone head gain falls by 0.334 pp, but the interaction contribution improves by 0.757 pp, yielding 0.423 pp more combined gain. The interaction improvement outweighs the standalone loss. This is consistent with training the head for its role after lookup correction; the controlled comparison in Part 2 evaluates the target change.
1.2. What do the two corrections address, and why does the raw head also combine effectively with the lookup?
To examine your orthogonality hypothesis, we decompose each correction into its mean over the H leads within each window and channel (offset) and the remainder (shape). These components are orthogonal by construction, so the lookup–head interaction splits exactly into offset and shape contributions.
The lookup uses centered patterns and contributes mainly to shape, while most of the head's gain comes from window-specific offsets. These results support the view that the two corrections primarily address different error components.
q2. Output means and alignment with the lookup. We compute means and cosines over all test origins, leads, and channels for each seed and setting, then average across seeds and settings. The raw head is the model selected for Lookup + raw head.
The lookup mean is approximately 0.000012; output means use standardized forecast units. The counts indicate weak alignment in many settings. (∣cos(B,P)∣<0.1 in 60/100 and 68/100). Together with the offset/shape results, this helps explain why a raw head already captures much of the combined benefit. However, weak alignment can nevertheless leave a meaningful interaction contribution, whose effect on performance is quantified using the applied correction u=γP above.