acceptodds
Under review as a conference paper at ICLR 2027

The Price of Not Forgetting: A Covering-Based Memory-Error Analysis for Continual Time-Series Forecasting under Regime Drift

Abstract

A forecaster running on a live stream faces an old dilemma: adapting to a new regime can erase what it knew about the old one. The usual escape is to freeze what has been learned and grow a fresh expert per regime - forgetting becomes impossible by construction, but the guarantee is bought with memory, and nothing says how much, or when to stop growing. We ask what can be certified about that bargain. The regimes a forecaster serves form a small metric space of distributions, and it is the intrinsic dimension of that space, not the raw channel count, that governs how error falls as experts are added. Covering arguments turn expert allocation into a budgeting problem with explicit upper certificates: for the regimes observed, and, under stated assumptions, across the whole condition space. On one narrow, fully specified class of drifts we also match the rate from below over a limited range of budgets - a conditional match, not universal optimality, and our slack terms are upper bounds rather than proved error floors. Three design rules follow: how large an adapter must be, when adding experts stops paying, and which regimes to keep in one pass. On five real datasets, freezing gives exactly zero measured forgetting - because old-regime parameters never move - yet does not uniformly out-learn methods that tolerate some. Low-rank adapters help only behind a bottleneck; a cross-channel drift statistic repairs a real blind spot in a controlled test but not on real streams; and the picture of fine-tuning sliding smoothly along a Wasserstein geodesic does not survive measurement. What we offer is a set of explicit certificates and their conditions - not a free lunch, but a priced one.

Primary area:
learning on time series and dynamical systems
Keywords:
Continual Learning, Time series, Deep learning, Theory

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_Ww3MOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper addresses an important problem, namely, the memory-error tradeoff in continual time-series forecasting. A rigorous theoretical framework is proposed. It models regime-specific forecast laws as a Wasserstein metric space, derives nearest-center clamping certificates by covering arguments, and proves a matching memory exponent for a restricted anchored-translation class with public condition nets and a finite expert codebook.

Critical strengths
  1. The memory cost of eliminating parameter interference is a practical and interesting question. The paper usefully distinguishes preserving old experts from obtaining good forecasts under a fixed resource budget.

  2. The mathematical statements carefully separate finite-regime certificates, restricted minimax results, and scheme-specific adapter costs. The explicit distinction between upper offsets and lower error floors substantially improves interpretability.

Critical weaknesses
  1. This paper lacks sufficient illustration about the theoretical results or the problem setup. It is hard to understand some of the concepts, definitions, and theorems. Moreover, it lacks enough motivation and connections for the proposed theoretical results, making it hard to capture the core contribution of this work. The authors should provide a more detailed description of the setup, the problem, potential practical contributions, and the connection between theoretical results and practical implementations.

  2. The connection between the theoretical error and forecasting utility remains unresolved. Sections 3.1-3.3 bound W2 distances between regime-level forecast laws, whereas Tables 5-8 evaluate paired prediction-target MSE. Matching an unconditional output distribution does not preserve the correspondence between inputs and targets: permuting forecasts within a regime can preserve the law while changing prediction error. Section 5 acknowledges this mismatch, but it affects the interpretation of the central memory-error claim. Please define precisely whether the laws are conditional on the full forecasting input, establish a loss-comparison result under appropriate coupling assumptions, or experimentally evaluate the actual Wasserstein quantity certified by the theorems alongside MSE.

  3. There are some writing problems or presentation errors in this manuscript. For example, Fig.2 is improperly clipped, Tab.7 has an extremely large size of font, and the font size in Fig.1 is too small. These problems should be addressed before the submission of this manuscript.

  4. The rank-adapter rule is not established for the implemented architecture. Design Principle 4 and Proposition 6 require an explicit realization of the leading functional displacement subspace by rank-r matrix updates. Low functional drift rank does not imply low parameter-update rank. Although the paper correctly states this proviso, no such realization is demonstrated for the MoE backbone. Moreover, Appendix M.4 estimates rank from regime-mean shifts rather than Definition 1’s optimal-map displacement family. The conditional theorem may be valid, but the measured proxy does not establish its applicability. Please construct the realization for the adapted layer, bound any approximation error, and justify the link between the measured and theoretical ranks.

  5. The experiments do not test the central memory–error tradeoff under a binding budget. Section 4.1 uses a single chronological 70/30 transition, and Table 6’s zero forgetting follows from retaining the untouched regime-A expert. Section 5 explicitly states that no experiment forces eviction or revisits an evicted regime. Therefore, the study does not establish how covering-based retention controls error when the number of encountered regimes exceeds the expert budget. Please evaluate recurring regime sequences with fewer retained experts than regimes, compare covering-based retention against recency and random retention at equal total memory, and report both current-regime and revisited-regime errors. Moreover, the experiments are conducted on small-scale datasets, and additional experimental results and analysis on large-scale real-world datasets should be provided.

Questions for authors

see weaknesses

2 comments
AuthorsOpenReview ↗

Response to Reviewer Ww3M

Thank you for a detailed review.

W1 (illustration, motivation). Yes, we added a plain statement of the problem and a map of the results (Section 3 opener).

  • Regimes arrive one at a time, at most kk frozen experts fit in memory, and each regime is served by its nearest stored expert.
  • Theorem 1 gives error vs. budget, Theorem 5 when more experts stop paying, and Theorem 6 which regimes to keep.
  • Each theorem now has a one-sentence reading above it. Figure 1 is redrawn to show this mechanism.
  • The budget experiment (Appendix T) gives a numerical case for Theorems 1 and 6; on ETTh1 and Electricity the result is consistent with saturation at k⋆ = 1 (Theorem 5).

W2 (W2 vs. MSE). Yes, we took your loss-comparison option; see General Response (GR) 2. Under conditional laws and a fit bound uniform over inputs, Theorem 1's certificate (in d⋆d_\star) bounds RMSE.

W3 (Figure 1, Figure 2, Table 7). Fixed. Figure 1 is redrawn as a mechanism diagram with all text at about 9 pt. Figure 2 has uncut axis labels and the key values on the bars. Table 7 is no longer scaled down.

W4 (rank rule on the implemented backbone). On ETTh1, the adapted layer stops improving near the measured displacement rank (new Appendix S).

  • The paper states the condition (p. 6, ll. 282-285) and calls the M.4 rank a proxy (p. 23).
  • For the adapted layer, the realization is explicit. The adapter is a rank-rr update ΔW=UV⊤\Delta W=UV^\top, so it changes the forecast by ΔWh(x)∈col(U)\Delta W h(x)\in\mathrm{col}(U), an rr-dimensional subspace of R96\mathbb R^{96}.
  • We measure the change that full fine-tuning makes in that space, y^ft(x)−y^A(x)\hat y_{\rm ft}(x)-\hat y_A(x).
  • On ETTh1, where fine-tuning helps, this change has rank 11-15 at τ = 0.05. Its top-8 part keeps 91% of fine-tuning's gain, and its top-16 part 95%. The adapter's error stops improving at r ≈ 8-16.
  • On ETTh2 and Exchange the rank is 5-6 and 3-4, close to the proxy kdrift=4k_{\rm drift}=4, but fine-tuning does not improve on the frozen model there, so the rank sweep is flat.
  • This links the measured rank to the adapted layer; it does not identify it with Definition 1.

W5 (binding budget; scale). Yes, we ran the experiment you describe (GR 3c, Appendix T).

  • It has 30 regimes, fewer experts than regimes, and covering vs. recency, random and first-k retention at equal memory. Evicted regimes are scored on held-out windows, which is what a revisit of an evicted regime costs. The regimes are consecutive rather than recurring, and we report mean and worst excess over all regimes, not current and revisited errors separately.
  • Covering in an error-calibrated metric lowers the worst-regime excess error on ETTh2 (k = 4: 0.36 vs. 0.88-1.67); on ETTm1, recency is as good or slightly better.
  • The covering radius tracks the mean held-out excess error (Spearman 0.77-0.90 on four of six series).
  • On scale, Electricity (321 channels) and Traffic (862 channels) are in Tables 5-6, on subsamples (Section 5 (i)).
  • New (Appendix U): the paper's setting reproduces the Electricity rows of Table 5 within 0.006.
  • With early stopping on all Electricity windows (10,203 training windows, three seeds), the three adapters have the lowest mean MSE_B of all methods (0.1926-0.1928 vs. replay 0.1936 ± 0.0014, fine-tune 0.1977), with zero forgetting; the margin over replay is within one s.d. of replay.
  • On Traffic with 8000 windows per split (three seeds), LwF is best (0.5489 ± 0.0036). The full-encoder adapter is second (0.5554 ± 0.0019), ahead of replay (0.5584), and the rank-16 adapter trails LwF by 0.014; both adapters have zero forgetting.
Reviewer_Ww3MOpenReview ↗

Response to the authors' reply

Thank you for providing detailed explanation and empirical results, and these results substaintially improve this work. But I think this paper requires another cycle of review since the modifications involve all aspects of theory, methods and experiments.

Reviewer_DbUsOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper studies growing expert architectures, a popular solution in continual learning, in the context of forecasting. The paper develops theoretical results to establish the distance between drifts and how big the novel experts should be. Experimental results in standard forecasting benchmarks show limited improvements inconsistent among the benchmarks.

Critical strengths

None.

Critical weaknesses
  • The paper is poorly written
    • I found the paper unclear at several points and hard to read
    • Sec.3 refers to Assumptions that are only formally stated in the Appendix. Other parts of the paper also refer to the Appendix.
    • the theory is currently a bit disconnected from the actual method. The paper could have provided a bit more motivation and a high-level map of what it's trying to do.
  • experiments:
    • benchmarks seem to be limited on two splits. Also, these splits are randomly set (70/30) and they do not correspond to actual distribution drifts
    • MSE is not sufficient in forecasting. The paper should report other metrics
    • several methods exist in the literature. However, the paper only compares against very simple continual learning baselines that are not even designed for forecasting
    • empirical results are not good.
Questions for authors
  • Fig.2 seems to be badly cropped
  • experts can achieve zero-forgetting only when task labels are provided. Are you working in a task-incremental setting? My understanding is that you are not, so I find the zero-forgetting claim unclear.
  • I would argue that forecasting is a task-free setting, where regimes cannot be clearly separated into well-defined and isolated task like we do in class-incremental. As such, isolated experts may not be an ideal solution. The results seen in the paper, such as showing that isolated experts are not necessarily better, may support this. I think the paper should discuss this. At the moment, reading the paper one would think that isolated experts are an ideal solution and we only need to optimize their memory cost.
2 comments
AuthorsOpenReview ↗

Response to Reviewer DbUs

Thank you for the review. Several points are already in the paper; we give the places and fix the rest.

Writing; assumptions in the Appendix; map of the paper. Fixed. Section 3 now opens with a plain statement of the problem and states Assumptions 3-5 in the main text. Each theorem has a one-sentence reading above it, and terms are explained in plain words where they first appear. The paper has a map (p. 3, ll. 157-161) and Figure 1, which is redrawn.

"splits are randomly set (70/30)" / "limited on two splits". The split is chronological, not random (p. 7, l. 362). Section 4.3 already uses 30 regimes per series (Table 7). Our correction: the boundary is the benchmark's train/test boundary, which the text called 70/30 (General Response, GR 3a). It is not an annotated drift, so the new budget experiment uses 30 consecutive regimes per series with held-out scoring (Appendix T).

MSE only. The early-stopping benchmark reports MAE (Table 15; Appendix U for Electricity and Traffic). The ordering matches MSE on ETTh2, ETTh1 and Exchange. On ILI the adapters' MAE (1.381-1.405) is slightly worse than fine-tune (1.379) and replay (1.344), though their MSE is lower.

Baselines. Added: replay and DER++ with a buffer the size of one adapter, LwF, a freeze-only row, budgeted freeze-and-grow, and the online forecasting methods FSNet and OneNet (Appendix X). With early stopping, the adapters have the lowest mean MSE_B among methods that train on B on ETTh2 and Exchange (freeze-only is lower still) and are within 0.010 of the best on ETTh1; on ILI, OneNet and then LwF are best (GR 4).

"empirical results are not good". With early stopping they improve (see Baselines above; GR 4). The paper claims a priced trade-off, not a win (abstract l. 026; Section 4.2). Table 8 (three seeds) shows the adapter beating replay on Exchange (0.083 ± 0.001 vs. 0.086 ± 0.003), tying on ETTh2 (0.512 ± 0.003 vs. 0.513 ± 0.008) and losing on ILI. Those runs over-train the adapter (Table 11).

Fig. 2. Fixed.

Task labels. Yes. Zero forgetting assumes the regime of each window is known; the paper states this setting (p. 8, l. 429), and the claim now says so where it is made. New: two routers that never see the label (GR 3b). Averaged over seeds, the adapters' forgetting stays between −0.022 and 0.047; on ILI it is 0.035-0.047, against 0.17-0.24 for fine-tune, EWC and replay.

Task-free forecasting; are isolated experts ideal? We agree that they are not ideal, and the paper's results show it. Replay has the best MSE_B on three of five datasets and is within 0.007 of the best on the other two (Table 5). A freeze-only model beat every adapted variant at the 30-epoch schedule (p. 8, l. 394). The revision adds a paragraph (Section 4.2, "Isolation is not an ideal solution"): the certificates price frozen experts when the regime is identifiable, not isolation in general.

Reviewer_DbUsOpenReview ↗

answer to rebuttal

I thank the authors for the rebuttal. I decided to keep my original score because the amount of changes is excessive and it results in essentialy a new paper. Evaluating it would be equivalent to a new review, which is not the goal of the rebuttal.

Reviewer_RGVROpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper operates in the domain of continual learning and time series forecasting. Continual learning is modeled as shifting regimes (distributional shifts that also cause the forecasting outputs to shift), while the solution space considered is the addition and freezing of experts for specific regimes (either entire networks or low rank components). In such a setting forgetting is not possible since every expert serves a single regime. However, as new regimes are introduced, the memory costs scale linearly.

The paper studies how much growth is required and when do the returns start diminishing by routing new regime inputs to the nearest expert which they term “clamping”. The key theoretic result is that the worst case Wasserstein error is bounded by the covering radius of the regime set already observed and their corresponding expert fit quality. Further, this error scales exponentially with the number of experts and the intrinsic dimensionality of the metric space formed by the regimes collectively. The authors derive upper certificates on this error for both the observed regimes, and extend it to the full condition space. Utilizing the upper certificate, the authors further derive a saturation point where additional experts yield diminishing improvements, and provide an eviction policy to remove experts not impactful. Finally, the authors also provide a design principle that ties the rank of the expert adapter to the defined functional rank of the regime providing insights into memory-performance trade-offs.

Experiments on five multivariate time series datasets show zero forgetting by construction for such a method. Additionally, on synthetically constructed regimes with controlled intrinsic dimensionality the estimation pipeline recovers the intrinsic dimensionality of the regimes. The results show that the covering dimensionality of the real world regimes is far lower than the number of variables in the time series datasets. Further, the authors validate the low rank adapter design principle where a lower rank adaptation of models without a bottleneck degraded the performance requiring a full rank adapter.

Critical strengths

A novel and well executed theoretical framing. Formulating the addition of experts as a covering problem and the relationship identified to the intrinsic dimensionality of the combined regimes served. The costs associated with growing experts scales with the complexity of the regimes and not with the dataset size, provides actionable insights to practitioners. Additionally, the theoretical claims made state the approximations and scope appropriately with valid self criticisms with the limitations section being thorough.

Critical weaknesses

The disconnect between theoretical contributions and empirical results is large, and the experiments conducted are poorly executed. As a result, it becomes difficult to asses the practical usefulness of theoretic contributions claimed. Every theorem operates in the Wasserstein 2 distance regime but the reported results are in mean squared errors (authors acknowledged). The regime split utilized and the general experiments are conducted poorly with over-fitted results reported. No justification for the regime splits used was provided, with the headline figure showing little to no forgetting. The baseline comparisons are narrow with no other methods utilizing freezing and growing with limited memory budget. The adapter rank derivation proposed by the theoretical derivations was not tested on the real world datasets. The streaming eviction principle derived has no empirical results supporting it. The drift detection mechanism proposed and the corresponding empirical results are not convincing with the performance close to chance, appropriate baselines should be utilized.

Questions for authors
  1. Can the authors provide the main results with the over-fitting corrected results?
  2. Why was the fixed rank adapter used for all the datasets?
  3. Why was the 70/30 split chosen? and why not consider more regimes in all the empirical results?
  4. Can the authors expand on the computational and memory costs of the full pipeline as compared to other freeze, and grow methods?
1 comment
AuthorsOpenReview ↗

Response to Reviewer RGVR

Thank you for the careful summary.

Theory in W2, results in MSE. Fixed under a fit bound uniform over inputs; see General Response (GR) 2. Under conditional laws, Theorem 1's certificate (in d⋆d_\star) bounds RMSE.

Over-fitting; Q1. Yes, we reran every method with early stopping, three seeds, MSE and MAE (GR 4, Appendix Q). The paper reports the over-training (Section 4.4; Table 11; Section 5 (iv)). The rerun improves the frozen adapter on all three paper datasets (e.g. ETTh2 0.513 → 0.475), consistent with Section 5 (iv).

Split; little forgetting in Figure 2; Q3. The split is chronological (p. 7, l. 362) and is the benchmark's train/test boundary (GR 3a). The baselines do forget: fine-tuning +0.215 on ILI and +0.069 on Exchange (Table 6). On ETTh2 forgetting is negative because B is easier. A single transition creates little forgetting on some datasets, so the budget experiment uses 30 regimes and scores evicted regimes (Appendix T, GR 3c).

Rank rule on real data; Q2. It was tested on real data, and we extended the test.

  • Table 11 (p. 26) uses real ETTh2 and Exchange: at one adapter epoch, rank 4 (= kdriftk_{\rm drift}) beats rank 16 and freeze-only (single seed, two ranks).
  • Rank 16 is at or above the measured kdriftk_{\rm drift} (1-11 at τ=0.05\tau=0.05, Table 7), except on Electricity at τ=0.01\tau=0.01 (21, Table 10).
  • New: r=kdriftr=k_{\rm drift} performs like r = 16 (ETTh2 0.474 vs. 0.475; Exchange 0.072 vs. 0.072; ILI 3.445 vs. 3.370) with 1/4 (ETTh2, Exchange) to 1/16 (ILI) of the parameters.
  • On ETTh1 the adapter stops improving at r ≈ 8-16, matching the measured rank 11-15 (Response to Ww3M, W4).

Streaming eviction. Yes, now evaluated. The budget experiment runs the one-pass rule of Appendix I as a retention policy. In the error-calibrated metric it nearly matches offline k-center from k ≈ 2-3. At k = 2, 4 and 8 it has lower worst-regime error than recency, first-k and random on ETTh2 (Appendix T, Table 20). At k = 1 it keeps the first regime by construction.

Detector; baselines. Added (Appendix W): four standard change statistics fail at the real cut in the same way ours does, and only cross-channel statistics detect a planted lead-lag change. The paper reports the detector as a negative result on real streams (Section 4.5; Table 13), and the contribution list says "but not on real streams" (p. 2). The four statistics use the same matched-lag protocol: raw-window MMD, a CUSUM-type mean shift, per-channel KS, and covariance change.

  • Synthetic streams, 20 per change type, with one planted change. On a lead-lag change that keeps every channel's marginal law, only the cross-channel statistics detect it: our cross-channel MMD reaches AUROC 1.00 and covariance change 1.00. The marginal statistics stay at 0.44-0.55.
  • Real 70% detection cut (ETTh2, Exchange, Electricity; ILI, n = 12, is too small to read). All six statistics have AUROC between 0.23 and 0.57.

The gap likely lies in the benchmark, which has no annotated change, not in the choice of statistic.

Q4 costs. The 7-channel backbone (ETTh2) has 4,155,300 parameters; one adapter has 30,208 (16 × (1792 + 96)), 0.73% of the backbone per regime. Appendix L (Table 3) gives the bit accounting. On one RTX 4090, training the adapter on ETTh2 takes 3.5 s versus 8.1 s for fine-tuning (mean of three seeds; from the run logs in the supplementary, data/extended_runs/; the PDF does not report times). Budgeted freeze-and-grow at equal memory (first-k) is in Appendix T.

Comments on the paper

2 comments
AuthorsOpenReview ↗

Summary of Revisions

Changes are in blue. Page, line, section, table and figure numbers in our responses refer to the submitted PDF; appendix letters O-X, Tables 14-24, Figure 8 and Proposition 8 refer to the revised PDF.

Where each concern is addressed:

  • Ww3M W1, DbUs (writing, map): Sec. 3 "The problem at a glance", Sec. 3.1 "Assumptions used below".
  • Ww3M W2, RGVR (W2 vs. MSE): Sec. 3.2, App. O.
  • Ww3M W3, DbUs (figures): Figure 1, Figure 2, Table 7.
  • Ww3M W4, RGVR (rank rule): Sec. 3.4, Sec. 4.4, App. S.
  • Ww3M W5, RGVR (binding budget, eviction): Sec. 4.3 "Binding budget", App. T, Figure 8.
  • Ww3M W5 (scale): App. U.
  • DbUs, RGVR (split): Sec. 4.1, App. M.1.
  • DbUs (MSE only, baselines), RGVR Q1 (over-fitting): Sec. 4.2 ("Isolation" paragraph), App. P-Q, X.
  • DbUs (task labels, task-free): Sec. 4.2, App. R.
  • RGVR Q2: App. Q, App. S. RGVR Q4: App. L, App. T.
  • RGVR (detector baselines): Sec. 4.5, App. W.
  1. Certificate in the reported metric. New Proposition (Appendix O): with forecast laws conditional on the input, the clamping certificate of Theorem 1 bounds the worst-regime RMSE of a point forecaster, under a fit bound uniform over inputs.
  2. Binding-budget experiment (new Appendix T). 30 consecutive regimes per series, fewer retained adapters than regimes. Covering-based retention (k-center; the one-pass procedure of Theorem 6) is compared with recency, first-k and random at equal memory, on held-out windows.
  3. Non-oracle routing for the frozen variants (descriptor router; recent-error router using only observed targets), Appendix R, Table 17.
  4. Benchmark with validation-based early stopping for every method, three seeds, MSE and MAE, plus equal-memory replay, DER++, LwF and a rank-kdriftk_{\rm drift} adapter (Appendices P-Q, Tables 14-16); FSNet and OneNet (App. X, Table 24).
  5. Rank principle on the implemented backbone: rank sweep with early stopping, and the measured rank of the fine-tuned forecast displacement in the space the adapter acts on (Appendix S, Table 18).
  6. Electricity and Traffic at larger scale: reruns of the paper's setting, and early stopping with three seeds on all Electricity windows and on 8000 Traffic windows per split (Appendix U, Table 21).
  7. Detector baselines (new Appendix W, Table 23): our two drift statistics vs. raw-window MMD, a CUSUM-type mean shift, per-channel KS and covariance change, on synthetic streams with planted changes and on the real cut.
  8. Presentation:
    • the abstract keeps the submitted text; only the zero-forgetting sentence changes (it now says it needs routing by the true regime, and the dataset count is removed so the abstract has no numbers);
    • the notation table (Table 1) is extended, and non-standard terms (MoE, MMD, lead-lag, LwF) are explained;
    • the Section 4 roadmap tags each subsection with the contribution it tests, and each results subsection ends with a one-sentence takeaway; new results enter the main text as short pointers;
    • in the Introduction, Limitations and Conclusion, the only changes in content are to sentences that the new results made false (zero forgetting under true-regime routing; evictions, early stopping and seeds now tested; detector baselines; the W2/MSE gap, now closed by App. O); the other edits there are the wording edits below;
    • Sec. 5 (iii) adds the surrogate's 8.4% effect on ETTh2 (Table 9);
    • Assumptions 3-5 stated in Section 3;
    • the A/B boundary described exactly as implemented;
    • a paragraph on task-free forecasting;
    • Fig. 1 redrawn, key values written on all data figures, Table 7 fixed;
    • cited: MoE, LoRA (Sec. 1), LwF (Sec. 4.2), LwF, DER++ defined (App. P), Dynamic TMoE for the backbone and marginal MMD (Sec. 3.6, 4.1); appendix captions lead with their main number;
    • appendix proofs rewritten for readability: proof ideas first, standard results cited, routine algebra grouped, three shared lemmas (App. A); a new Appendix V maps each proof to its Lean 4 lemmas and Lean hypotheses (Table 22); no statement changes beyond the next item;
    • Theorems 1 and 5 and Proposition 3 now name hypotheses their proofs already used (every visited regime is eligible to be a center; the assumptions of Theorem 1; a pair at distance at least dmin⁡d_{\min}), Definition 1 states its L2(μ)L^2(\mu) condition as an assumption, and small proof gaps are closed (a proof for Proposition 4; the case k<c2k\lt c_2 in Corollary 1; monotonicity of rkr_k; "only if" corrected to "if" in Corollary 3);
    • corrections found while rewording (no result changes): App. M.9 retitled (it compares one expert in and out of its regime, not clamping vs. conditioning); prose in App. M.3 and N.1 aligned with the tables;
    • wording: plain readings of key terms and results, each appendix opened with its purpose, long sentences and paragraphs split, and repeated scope caveats trimmed (Sec. 1-6 and appendices); contributions labelled C1-C4, with pointers to their Section 4 evidence.
AuthorsOpenReview ↗

General Response to All Reviewers

We thank the reviewers. Four points are shared.

1. What the paper claims. The paper gives certificates for frozen-expert memory, not a claim that frozen experts win.

  • The contributions are the certificates (Theorems 1-3, 5-6), the design rules that follow (Design Principle 4), and an account of where the rules hold (p. 2, ll. 073-090).
  • The abstract (l. 026) says frozen experts do "not uniformly out-learn methods that tolerate some" forgetting.
  • Ww3M and RGVR ask whether the certificates predict error when memory binds; DbUs asks for more than one transition. The new experiments answer this (items 3c and 4).

2. W2 certificates and MSE (Ww3M W2, RGVR). Yes, the gap is real, and a new proposition closes it under a fit bound uniform over inputs (Appendix O).

  • The paper already flagged the gap (Appendix A, p. 12, ll. 643-644; Section 5 (iii)).

  • Define forecast laws conditionally on the input, μc(⋅∣x)\mu_c(\cdot\mid x). For a point forecast, W2(δy^(x),μc(⋅∣x))2=E[∥y^(x)−Y∥2∣x]W_2(\delta_{\hat y(x)},\mu_c(\cdot\mid x))^2=\mathbb E[\Vert\hat y(x)-Y\Vert^2\mid x]. With d⋆(c,c′)=sup⁡xW2(μc(⋅∣x),μc′(⋅∣x))d_\star(c,c')=\sup_x W_2(\mu_c(\cdot\mid x),\mu_{c'}(\cdot\mid x)) and the fit condition stated per input, the triangle inequality gives

    RMSEc(expert i)≤Cϵi+d⋆(ci,c).\mathrm{RMSE}_c(\text{expert } i)\le C\epsilon_i+d_\star(c_i,c).

  • So the clamping certificate of Theorem 1, measured in d⋆d_\star, bounds worst-regime RMSE. The permutation counterexample no longer applies, because permuting forecasts changes the conditional laws. The fit term now includes irreducible noise.

3. Evaluation protocol. The split is chronological, zero forgetting needs the regime label (routers without it are measured), and the binding-budget experiment is new.

  • (a) Split. The split is chronological, not random (p. 7, l. 362; App. M.1). Section 4.3 already uses 30 regimes per series (Table 7). One correction from us: the A/B boundary is each benchmark's standard train/test boundary (ETTh2: months 1-12 vs. 17-20; others: first 70% vs. last 20%), not "70/30" (70/30 is the within-regime train/test split of revised App. M.1 and the detection cut of App. N). The revision states it exactly.
  • (b) Regime identity. Yes, zero forgetting needs the regime of each window.
    • The paper states this setting (p. 8, l. 429), and the claim now says so where it is made.
    • New (Appendix R): two routers that never see the label. One uses window statistics; the other uses the error on already-observed targets. The adapters' forgetting then stays between −0.022 and 0.047 (three-seed mean, all 16 router-dataset-adapter combinations); the dense and MLP heads forget more (up to 0.54).
    • On ILI, where shared-weight methods forget most (fine-tune 0.239, replay 0.174), routed forgetting is 0.035-0.047; LwF reaches 0.015.
  • (c) Binding budget. New experiment, Appendix T.
    • Each series has 30 regimes, one adapter each, and k<30k\lt 30 are kept at equal memory. Selection uses calibration windows; scores use held-out windows.
    • We measure the covering radius r(S)r(S) in an error-calibrated metric d^\hat d, an estimate of d⋆d_\star. It tracks the mean held-out excess error (Spearman 0.77-0.90 on ETTm1, ETTh2, Exchange, Traffic; 0.32-0.78 for the worst excess).
    • On ETTh2, covering lowers the worst-regime excess relative to recency, first-k and random (k = 4: 0.36 vs. 1.67, 1.16, 0.88). On ETTm1, recency is as good for k ≤ 2 and slightly better at larger k.
    • Theorem 6's one-pass rule nearly matches offline k-center from k ≈ 2-3.
    • The paper's channel-mean descriptor tracks error less well (Spearman −0.04 to 0.43), so the certificate should use an error-aligned metric.

4. Over-training and baselines. Yes, we reran every method with early stopping (Appendix Q, three seeds).

  • The paper reports the over-training (Section 4.4; Table 11; Section 5 (iv)).
  • The rank-16 adapter's MSE_B drops from 0.513 to 0.475 (ETTh2), 0.084 to 0.072 (Exchange) and 4.095 to 3.370 (ILI).
  • With zero forgetting, the adapters then have the lowest mean MSE_B among methods that train on B on ETTh2 (0.474-0.487 vs. LwF 0.491, replay 0.508) and Exchange (0.071-0.072 vs. LwF 0.073). On ETTh1 they are within 0.010 of the best; on ILI, LwF is best of these (2.945 vs. 3.370).
  • With a buffer the size of one adapter, replay is worse than the adapter on ETTh2, ILI and Exchange and ties on ETTh1. DER++ is better on ILI and ETTh1, with ILI forgetting 0.076.
  • New (Appendix X, Table 24): FSNet and OneNet, two online forecasting methods, run from the official code, each with its own trainer, on the same splits, seeds and horizon. Held-out, both trail the rank adapter on ETTh2, ETTh1 and Exchange (ETTh2: 0.647-0.928 vs. 0.475); OneNet is best on ILI (2.566). Their original protocol updates on targets that overlap later test windows; with delayed feedback the ordering holds.
  • On ETTh2 and Exchange the regime-A model alone is already the best B forecaster, consistent with the weak drift signal at the 70% detection cut in Table 13.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.