- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
SCoRe is a plug-and-play module that learns an additive correction to the output of a frozen backbone. It targets entities that lack direct history under a forthcoming context, such as a product with no historical records for an upcoming festival. The method has four modules. In Semantic Residual Memory, an LLM describes each forecasting scenario jointly from its history, covariates, known future conditions and entity attributes. The top-K historical cases are retrieved by cosine similarity of description embeddings, together with their normalized backbone residuals. A frozen BGE encoder provides normalized embeddings for downstream conditioning. Conditional Response Transport (CRT) warps each retrieved residual in time around calendar anchors, then applies a bounded scaling and a low-rank shape increment. Dual-branch Utility Estimation (DUE) predicts the directional gain of each transported candidate, computes G_q exactly, and approximately maximizes the estimated utility with a fixed number of Frank-Wolfe iterations starting from zero. It does this in two branches. Reference branch D uses a historical utility network and a query gate, and branch S uses a separately pretrained utility network adapted on recent fully observed windows. Regularized Correction Blending (RCB) mixes the two branches with one mixing coefficient shared across queries. The coefficient is fitted through an analytical solution on OOF corrections from rolling validation folds, and it is used only if it passes a development-set risk check. The main experiments add SCoRe to eight backbones, DLinear, TimeXer, iTransformer, PatchTST, TFT, TS-RAG, RAFT and Chronos-2, on the five EPF subsets, EDF and Bike Sharing. SCoRe lowers MSE in 53 of the 56 backbone and dataset combinations and MAE in 53, with average relative reductions of 8.31% and 5.10%. On M5 it is added to four backbones and lowers MSE by 2.73% to 13.85%. The paper also gives a progressive ablation on EDF and Bike Sharing, several mechanism analyses and three M5 case studies. In a real-world industrial retail setting, ACC rises from 67.44% to 71.45%.
Critical strengths
The correction pipeline is clearly specified and guards against temporal leakage. Eq. (4) gives the realized error reduction of the combined correction. Candidates are therefore combined through their interactions in G_q instead of being scored separately, and zero correction stays available. Eq. (8) is the correct analytical solution of Eq. (7). The temporal protocol is explicit. Lines 309-310 state that a historical case is eligible for retrieval only after its complete forecast horizon has been observed. The mixing coefficient is fitted on rolling OOF predictions from three chronological folds. Activation is decided on a later development interval. If a check fails, the method reverts to branch D or to the backbone, and all model parameters are fixed before testing (lines 263-297). Output correction leaks easily, and this protocol handles the main risks, so the design supports reading the results as out-of-sample.
The paired gains are broad and consistent. SCoRe is added to eight backbones of different kinds, including two retrieval-augmented forecasting methods, TS-RAG and RAFT, and the pretrained forecasting model Chronos-2. Lines 316-319 state that both versions use the same base-model checkpoint and input configuration. In Table 1, MSE falls in 53 of the 56 backbone and dataset combinations and stays the same at the reported precision in the other three. The largest MAE increase is 1.19%, for TS-RAG on NP. Each step of the progressive ablation in Table 2 also lowers all four metrics. These consistent gains are the main evidence that SCoRe works as a plugin.
Critical weaknesses
The experiments do not isolate the paper's central claim. The title, abstract and contributions credit the gains to semantic retrieval, which finds potentially transferable cases across entities and contexts and so "bridges the experience gap". The evidence for this has four gaps.
First, the main text has no experiment that replaces semantic retrieval with numerical or random retrieval while keeping CRT, DUE and RCB fixed. Table 2 adds modules one at a time, but only for one backbone and two datasets, and its rows match the TimeXer results in Table 1. Figure 4 compares semantic and numerical retrieval for a single M5 query without learned adaptation, and lines 446-447 say it measures retrieval quality rather than complete model performance. The description embeddings also condition CRT through its encoders, as described in lines 206-207 and 221-222. Replacing the retrieval alone would therefore not show whether the LLM descriptions are needed.
Second, the motivating case is never tested directly. Lines 041-043 report that about 45% to 50% of products have no historical records for the same festival. No result is split by whether the target entity has history under the target context. The paper also does not report what share of retrieved references come from other entities or other contexts.
Third, the public versions of the seven evaluation settings in Table 1 each contain one target series with covariates. EPF has one hourly price series and two covariates per subset. EDF is the national electricity demand of Panama, and Bike Sharing counts hourly bicycle rentals across the whole Washington, D.C. system. These settings can test transfer across contexts within one series. They test transfer across entities only if the residual memory pools several EPF subsets or datasets, and the main text does not say whether it does. M5 is the clearest public test of the motivating case, yet Table 3 covers only four backbones and reports only aggregate results.
Fourth, the mechanism analyses are indirect. Figure 3(b) shows that the utility networks rank the same transported candidates better than text scores within each query. This fits the paper's point that semantic relevance does not imply error reduction, but it does not show that semantic retrieval finds better candidates than numerical retrieval. After temporal alignment, the Pearson correlation between aligned residuals and e_q is still low, 0.04 on BS and 0.05 on EDF in Fig. 3(a).
Taken together, the results show that the full framework helps. They leave open how much of the gain comes from semantic retrieval and conditional residual transfer and how much comes from extra inputs and SCoRe training. That attribution is the paper's main claim.
The benchmark tables lack a baseline with the same information, and they report no uncertainty. Tables 1 and 3 compare SCoRe only with the original backbone. The internal comparisons in Table 2, Fig. 3(c) and Fig. 6 cannot replace a comparison with another correction method that uses the same information and fitting data. SCoRe adds trainable parts, adapts branch S on recent fully observed windows, fits its mixing coefficient on rolling validation folds, and decides activation on a later development interval (lines 247-249 and 263-284). The encoders for query and source conditions use text, known calendar features, historical statistics and base forecasts (lines 221-222). Each backbone, in contrast, uses the inputs supported by its architecture (lines 158-159). The main text does not say whether the backbones receive the same inputs and fitting data. TFT already uses known future covariates and still gains 7.30% to 19.50% in MSE, so extra information does not explain everything, but its share still needs to be measured. Useful baselines include a learned residual correction model trained on the same inputs and data, residual averaging with numerical retrieval, and the backbone refit on the same recent data. Adaptive forecasting methods that the paper already cites would also be fair comparisons. Examples are SOLID (Chen et al., 2024), which fine-tunes the prediction layer on contextually similar samples, and the test-time adaptation method TAFAS (Kim et al., 2025). Tables 1 and 3 also give no standard deviations or significance tests, and three random seeds are mentioned only in Section 4.4. Several gains are below 1%. For Chronos-2 on DE, MSE goes from 0.237 to 0.236, and for TS-RAG on BE it goes from 0.340 to 0.338.
Questions for authors
Which windows fill the residual memory? Lines 211-213 say that fitting and selection precede the forecast windows of the historical forecasts B_j^hist. For windows inside the backbone's training period, is the backbone refit on a rolling basis so that the stored residuals are out-of-sample? For a given query, does the historical text index M hold only cases whose forecast origin comes before it? Candidate rankings are precomputed (lines 205-206), and invalid candidates are masked without replenishment (lines 213-215). Later cases could then take top-K slots and shrink K_q. Does the set of eligible cases grow during testing, and what is a typical K_q?
Does retrieval cross EPF subsets or datasets? What share of retrieved references comes from a different entity or a different context than the query?
Which LLM generates the descriptions, and what does the domain metadata D_q in Eq. (1) contain, for example dataset names or absolute dates? Each description includes "a qualitative hypothesis about future trends" (lines 201-202). The paper says this hypothesis uses only available inputs and serves as a retrieval cue, not a numerical forecast (lines 202-203). The description embeddings, however, also condition CRT (lines 206-207 and 221-222). For public benchmarks whose periods an LLM may have seen during pretraining, such as M5 and EPF, how do you rule out that the hypothesis draws on remembered outcomes? An ablation that removes the hypothesis from the description would show how much it contributes.
What inputs does each backbone receive? Do DLinear, PatchTST and iTransformer get the known future covariates that SCoRe uses? Are all backbones trained up to the same cutoff as the SCoRe components?
Eq. (10) is a Laplace-type loss with location ĝ_qj and scale σ_qj, so ĝ_qj estimates a conditional median of g_qj. The utility in Eq. (4) is linear in g_q, so its expected value depends on the conditional mean of g_q. Why did you choose this loss, and do the results change with a squared-error or Gaussian loss?
Which M5 series, aggregation level, splits and horizon were used? If you used a subset, how was it chosen? Why does Table 3 include only four backbones, and how was the query in Case 1 selected?
Was the ACC gain from 67.44% to 71.45% measured in an offline back-test or an online test? Over what period, and on how many products? Was the production model retrained on the same data?
How much does SCoRe add to the cost of the backbone? Please report the number of LLM calls and the time spent generating descriptions, along with the SCoRe training and inference time.