acceptodds
Under review as a conference paper at ICLR 2027

Not All History Matters Equally: Tokenization through Instance-Adaptive Aggregation and Time-Decay Preservation

Abstract

Embedding quality constrains the achievable performance of time series forecasting. Patch-based tokenization preserves local temporal structure,but assigns a uniform token to every patch regardless of its predictive value, leaving the downstream model to filter irrelevant information and resolve redundant patterns. We propose RePatch, a forecasting tokenizer built on the principle that not all history matters equally. Through instance-adaptive aggregation, it learns input-dependent weights to suppress uninformative patches and combine related patterns across non-contiguous regions into a compact set of tokens. The forecasting objective jointly trains the tokenizer and downstream model, directing the limited token budget toward information useful for prediction. The learned aggregation also exhibits a preference for recent history.To reinforce this behavior, we introduce a time-decay prior that merges recent patches with the learned tokens. An input-conditioned allocatorcontrols their mixing weights, with stronger preservation for patches closer to the forecast and progressively weaker preservation for more distant candidates. This fusion retains recent context alongside global patterns without increasing the number of tokens. Search-based comparisons on five benchmarks show lower average MAE than the compared methods. Aggregation comparisons, recent-candidate ablations, and parameter sweeps further examine the design and its sensitivity to tokenization choices.

Primary area:
learning on time series and dynamical systems
Keywords:
Time-series forecasting, Adaptive tokenization, Efficient forecasting

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_jrtXOpenReview ↗
Rating
2(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper proposes PatchBasis, a tokenizer for patch-based long-term forecasting that compresses M input patches into K tokens. Each channel's history is split into M patches, which are linearly embedded, given sinusoidal positional encodings and passed through a residual convolution. A shared two-layer MLP (BasisLearner) scores every patch for each of the K output slots. A softmax over patches turns these scores into an input-dependent aggregation map, so each token is a convex combination of patch embeddings plus a learned slot embedding. A "Time-Decay Prior" then mixes slot k with the k-th most recent patch using the logistic weight. The K tokens are fed to a channel-independent Transformer trained with an MAE loss and RevIN. The authors claim three contributions, i.e. a view of tokenization as instance-adaptive selective aggregation of non-contiguous history, an architecture that combines this aggregation with preservation of recent patches, and an empirical study covering accuracy, ablations, efficiency and sensitivity.

Critical strengths

The paper reports its results transparently. It includes results that go against its own method. The plug-in hurts two of the four backbones, the input-dependent variant gives no gain, and inference is slower. It also discloses protocol exceptions that many papers would leave out. This matters because it makes the work easy to audit, and the authors' own diagnostic test in App. F is the right tool for answering my main concern.

The empirical study is broad and yields a useful finding. The method is compared against several tokenizers across three backbone settings and nine datasets. The analysis shows what a learned pooling tokenizer actually converges to, namely recent patches plus daily patterns. This matters because it shows the module is a workable tokenizer and gives the community a clear picture of what such tokenizers learn, even though it does not yet support the specific mechanism the paper claims.

Critical weaknesses

The paper's main claim is not supported, and the mechanism is not new. The paper attributes its gains to aggregation weights that adapt to each input. However, it never tests this against fixed weights in the same pipeline. What it shows about the learned weights is mostly a preference for recent patches plus daily repetition. The aggregation step is also an existing, uncited technique. This matters because it is the paper's main contribution, and without it the novelty is too limited for ICLR

The evidence does not reliably support the reported gains. There are three problem, i.e. (1) the released code implements a different version of the method from the one the paper describes. (2) strong baselines in Table 2 perform far below their published results. (3)main results come from a single run, with margins smaller than the variation shown in the paper's own repeated runs.

Questions for authors

The paper describes a fixed recency budget, but the released code predicts it from each input. Which version produced the main results? What values of K, rho and tau_a were used for each dataset?

In Table 2, how were the shared training settings chosen, and were they tuned for PatchBasis? Several strong baselines score well below their published results (for example, iTransformer on Electricity). Did the baselines use the same loss and checkpoint rule as PatchBasis?

Do the three-seed runs in Table 10 use the same configurations as the main results in Table 11? They are worse by up to 0.015 MSE, which is enough to reverse the comparisons on ETTm1 and ETTh2.

Can you compare against the same pipeline with fixed (input-independent) aggregation weights? Can you also apply your own/mean/swap test to the aggregation weights? If input-specific weights clearly help, the central claim would be much better supported.

How does PatchBasis compare with simply using the most recent K patches, with no aggregation?

3 comments
AuthorsOpenReview ↗

Response to Reviewer jrtX (1/3)

Thank you for your careful review and constructive comments.


[W1] The paper's main claim is not supported, and the mechanism is not new. […] it never tests this against fixed weights in the same pipeline. […] The aggregation step is also an existing, uncited technique. […]

Response: The manuscript already compares against input-independent aggregation through TimeBase. Section 3.1 explicitly identifies its shared weights, and manuscript Table 4 compares TimeBase and BasisLearner under the same downstream backbone within each setting. It repeats this across multiple backbone settings. We therefore disagree that no fixed-weight comparator was tested. The tokenizer representations still differ, as discussed in Q4.

Our contribution concerns how to construct a useful forecasting token basis, why it should adapt to the input, and how recent information can restore details lost through aggregation. The existence of pooling operators does not by itself resolve these questions. Related Work already distinguishes our design from other aggregation schemes.

TokenLearner is the closest operator-level counterpart in our comparison. It targets visual representations and reduces tokens within a ViT stack. PatchBasis builds its compact historical representation before the forecasting backbone and explicitly supplements it with recent patches. Similar operators therefore need to be assessed in the context of their architecture, objective, and evidence.

The introduction defines the forecasting objective: “We explore tokenization from the perspective of instance-adaptive selective aggregation. It reorganizes forecast-relevant information across non-contiguous historical patches into a compact token basis.” Section 3.2 explains how the token basis changes with the input: “Recomputing A from each input lets the same token slot draw on different historical regions across instances.” The Recent branch follows from the observed aggregation weights. Section 3.3 states: “BasisLearner assigns high weights to recent patches in some tokens without an explicit prior (Figure 2).” “Recent patches provide a starting point for forecasting. Preserving them also supplements features lost during aggregation.” The manuscript explicitly combines these components within one budget: “BasisLearner and the Time-Decay Prior jointly retain historical patterns and recent details within a fixed token budget.”

The shared-pipeline comparison, own/mean/swap intervention, and recent-only control in Q4 and Q5 go beyond visual inspection of recency and daily patterns. They test the roles of input matching and historical aggregation directly.

We have added a direct TokenLearner discussion and citation to Section 2, clarifying these forecasting-specific differences.


[W2] […] (1) the released code implements a different version of the method from the one the paper describes. (2) strong baselines in Table 2 perform far below their published results. (3)main results come from a single run, with margins smaller than the variation shown in the paper's own repeated runs.

Response: The main experiments use the fixed-budget model, and we have corrected the repository default and rerun the main comparisons with three seeds. Q1 identifies the implementation and hyperparameters. Q2 explains the baseline settings and Electricity learning-rate correction. Q3 explains the earlier configuration mismatch. The updated main results are no longer based on a single run.

AuthorsOpenReview ↗

Response to Reviewer jrtX (2/3)


[Q1] […] Which version produced the main results? What values of K, rho and tau_a were used for each dataset?

Response: The main results use a fixed recency budget. Dynamic budgeting was our initial design. It did not consistently outperform a fixed budget, so we simplified the model and reran the experiments. The released repository contained the fixed-budget implementation, but it was not the default. This was an upload oversight, and we apologize for the confusion.

The repository is now updated. The fixed-budget implementation is at line 687 of layers/PatchBasisLayers.py in the current version. Dynamic budgeting remains a viable alternative: it removes the need to select a fixed-budget hyperparameter. The two variants have similar overall performance, with dataset-dependent trade-offs.

The temperature τₐ is 1 in every experiment and was not tuned. We have removed this parameter from the manuscript. We previously released the shared settings for manuscript Table 12 but not all model-specific values. The table below provides K and ρ for the listed tasks; we will also release them with the configurations.

Task-specific token counts and recency parameters. K is unchanged across the four horizons within each listed dataset; τₐ = 1 throughout.

DatasetKρ at H = 96ρ at H = 192ρ at H = 336ρ at H = 720
ETTh11610.00010.0001.0002.500
ETTh283.5003.5001.0001.000
ETTm185.0004.0001.0001.000
ETTm243.0003.0003.0003.000
Weather83.5002.5001.5001.500
Electricity20.7500.7500.2500.500
Traffic20.5000.5000.2500.250
Solar-Energy20.5000.5000.1500.150
Exchange163.0006.0003.0003.000

The baselines retain their published best model-specific settings. We select ρ as a model-specific hyperparameter using validation MSE. We do not view replacing a learned budget with a selected fixed budget as an attempt to favor PatchBasis. The dynamic alternative avoids this hyperparameter and gives comparable overall performance. We used the fixed variant in the main text for simplicity.


[Q2] In Table 2, how were the shared training settings chosen, and were they tuned for PatchBasis? […] Did the baselines use the same loss and checkpoint rule as PatchBasis?

Response: The shared configurations were selected using PatchBasis’s average validation MSE, which can favor PatchBasis. The baselines nevertheless use the same training and evaluation procedure, including MAE loss and validation-MAE checkpoint selection. We distinguish this shared-configuration selection bias from unequal treatment during training.

We checked and reran baselines with large discrepancies from their published results and released their checkpoints in the anonymous repository. The Electricity issues for TimePerceiver, SEER, and iTransformer were associated with the learning rate of 0.010. We changed the learning rate to 0.001 for all models on Electricity. This correction is documented in the revised experimental settings.


[Q3] Do the three-seed runs in Table 10 use the same configurations as the main results in Table 11? […]

Response: The two tables use different configurations. Manuscript Table 10 is a recent-candidate ablation. Manuscript Table 11 reports results after an extensive hyperparameter search. Table 10 was not a repeated-seed evaluation of Table 11’s best configurations. The revised manuscript states: “Recent-candidate ablation. Figure 4 and Tables 10 and 3 use L = 512 and H = 192 on five datasets.” Appendix A.3 lists the input-length search range: “including input lengths L ∈ {96, 192, 320, 512}.”

For example, the relevant ETTm1 main-table configuration uses L = 320, whereas the ablation uses L = 512. The ablation also isolates allocator initialization from the random-number stream of shared modules. This keeps shared-module initialization comparable across Recent, Uniform, and no Recent, but differs from the main experiments. The discrepancy cannot be attributed entirely to seed variation. Table 10 alone does not establish the stability of the main-table configurations. We have now separately rerun the main comparisons with seeds 2024, 2025, and 2026.

AuthorsOpenReview ↗

Response to Reviewer jrtX (3/3)


[Q4] Can you compare against the same pipeline with fixed (input-independent) aggregation weights? […] apply your own/mean/swap test to the aggregation weights? […]

Response: We provide both a shared-weight comparator and direct interventions on the learned aggregation matrix. Section 3.1 already describes TimeBase as input-independent: “The aggregation weights remain shared across inputs:”

Manuscript Table 4 includes this comparator. Its coverage of APE, TOTEM, and other native tokenizers requires method-specific representations. We therefore also report the earlier shared lightweight pipeline below.

The lightweight comparison uses TimeBase-MLP-SameRank-PM and BasisLearner (Instance-TL + Position). It uses six datasets, L = 720, H = 336, and three seeds. K, depth, GELU, dropout, and the forecasting backbone are matched. The maximum relative difference in total parameter count is 1.153%. Each method selects its learning rate by validation MSE at patch size 24, then reuses it for the other patch sizes.

Shared lightweight pipeline; L = 720, H = 336, patch size 24. Three-seed mean test MAE.

DatasetTimeBase MLPBasisLearner
ETTh10.4280.421
ETTh20.4180.410
ETTm10.4110.386
ETTm20.3630.332
Weather0.3400.299
Electricity0.2800.259

BasisLearner achieves lower MAE on all six datasets at patch size 24 and on all 16 evaluated dataset–patch combinations. Its scoring structure and weight constraints still differ from TimeBase MLP. We therefore do not describe this as changing only input dependence in the complete PatchBasis pipeline.

For the matrix intervention, we use the trained BasisLearner without Recent and do not retrain it. Own uses the current window’s weights. Mean averages weights over training windows while preserving channels and output slots. Swap takes weights from another test window while leaving the current input unchanged.

Aggregation-matrix interventions without retraining. Test MSE / MAE.

Weight sourceETTh1ETTm1Weather
Own0.370 / 0.3840.293 / 0.3320.158 / 0.194
Mean0.380 / 0.3950.328 / 0.3550.165 / 0.207
Swap0.392 / 0.4030.348 / 0.3670.167 / 0.210

Mean and Swap worsen both metrics on all three datasets. The trained model therefore uses the match between the input and its aggregation weights. This tests A(x), not the recency-budget diagnostics in Appendix F. It supports input matching, but does not rule out a separately retrained static model achieving similar performance.


[Q5] How does PatchBasis compare with simply using the most recent K patches, with no aggregation?

Response: The full model outperforms using only the most recent K patches in all three tested settings. Recent-only retains the latest K patches after the same preprocessing and LocalContext, then uses the same Transformer and prediction head without aggregation. K, the remaining architecture, and the tuning budget are matched within each dataset.

Recent-only and branch ablations; L = 192, H = 96. Three-seed mean test MSE / MAE.

MethodETTh1ETTm1Weather
Recent-only0.378 / 0.3900.303 / 0.3360.162 / 0.198
BasisLearner without Recent0.370 / 0.3840.293 / 0.3320.158 / 0.194
Full model with Recent0.366 / 0.3820.291 / 0.3310.155 / 0.190

At L = 192 and H = 96, the full model with Recent model reduces mean MSE by 3.110%, 3.790%, and 4.510% relative to recent-only on ETTh1, ETTm1, and Weather. BasisLearner without Recent also improves both metrics on all three datasets. Historical aggregation therefore contributes beyond simply keeping recent patches.


We hope these responses address your concerns, and we would be happy to provide further clarification during the discussion period.

Reviewer_hxbpOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

PatchBasis addresses the uniform token allocation of conventional patching, in which every patch receives one token regardless of its predictive value. A BasisLearner computes input-dependent softmax aggregation weights that combine forecast-relevant patterns across non-contiguous historical regions into a compact token basis, while a Time-Decay Prior merges recent patches with weights decreasing with distance from the forecast origin. Experiments on nine long-term benchmarks show the lowest average MAE on all five datasets and the lowest average MSE on four.

Critical strengths
  • Simple design and clear motivation. The equations describe the method very clearly. The method is easy to understand and analyze.
  • Positive forecasting results. Overall, the proposed method achieves good results on the commonly used time series datasets tested.
  • Reported limitations. The authors provide a detailed analysis of the method's limitations.
Critical weaknesses
  • Novelty is somewhat limited. It looks like it's just applying softmax weighting to the patches. Eqs. (1)–(2) closely resemble TokenLearner's MLP version, but this is not discussed. Further clarification is needed on where the innovation of each part lies.
  • The analysis of patching methods for previous time series forecasting models is insufficient; more analysis of prior methods should be included.
  • Although the method is effective on some datasets, the improvement is not significant.
  • The experimental results lack significance analysis.
  • The ablation study is not thorough enough; it does not analyze the removal of each module individually.
Questions for authors

See Weaknesses.

5 comments
AuthorsOpenReview ↗

Response to Reviewer hxbp (1/4)

Thank you for your careful review and constructive comments.


[W1] / [Q1] Novelty is somewhat limited. […] Eqs. (1)–(2) closely resemble TokenLearner's MLP version, but this is not discussed. […]

Response: Our contribution is the forecasting-specific token basis and its recent-information supplement. The MLP aggregation operator is related to TokenLearner, but that relation does not make the complete designs equivalent.

TokenLearner addresses the computational cost of stacked ViT layers by reducing tokens within the network. Its gains over ViT are not uniform across settings. PatchBasis instead constructs compact tokens before the forecasting backbone. It selects and combines useful historical information to improve forecasting with fewer tokens than conventional patch-based models.

The introduction defines the forecasting objective: “We explore tokenization from the perspective of instance-adaptive selective aggregation. It reorganizes forecast-relevant information across non-contiguous historical patches into a compact token basis.” Section 3.2 explains how the token basis changes with the input: “Recomputing A from each input lets the same token slot draw on different historical regions across instances.” The Recent branch follows from the observed aggregation weights. Section 3.3 states: “BasisLearner assigns high weights to recent patches in some tokens without an explicit prior (Figure 2).” “Recent patches provide a starting point for forecasting. Preserving them also supplements features lost during aggregation.” The manuscript explicitly combines these components within one budget: “BasisLearner and the Time-Decay Prior jointly retain historical patterns and recent details within a fixed token budget.”

These statements identify two linked questions: how input adaptation improves a forecasting token basis, and how recent information can compensate for aggregation loss at low cost. The recent information is directly available and does not require a separately trained extractor. We view this combination as the contribution, rather than judging the whole method from the pooling operator alone.

We have added the TokenLearner citation and a direct discussion of the shared operator and the forecasting-specific differences to Section 2.


[W2] / [Q2] The analysis of patching methods for previous time series forecasting models is insufficient […]

Response: Sections 2 and 3.1 already distinguish patch-boundary adaptation, shared aggregation, input-adaptive aggregation, and patch selection. Section 2 states: “While TimeMosaic (Ding et al., 2026) and EntroPE (Abeywickrama et al., 2025) adapt local patch boundaries, PatchBasis adapts aggregation across the history.” “SRSNet (Wu et al., 2026) selects and reorders informative patches, whereas BasisLearner combines multiple patches within each token.”

Section 3.1 contrasts TimeBase’s mapping shared across inputs with BasisLearner’s input-dependent aggregation matrix. The controlled comparison is reproduced in the shared lightweight pipeline table in our response to W5 / Q5. The issue is how to organize information before compression into the forecasting backbone; we do not argue that downstream models with fixed patches cannot learn importance.

AuthorsOpenReview ↗

Response to Reviewer hxbp (2/4)


[W3] / [Q3] […] the improvement is not significant.

Response: The updated results show competitive average performance, with gains that vary by dataset. We do not claim a universal winner. This is consistent with the motivation discussed in Breaking Silos: Adaptive Model Fusion Unlocks Better Time Series Forecasting.

Selected baseline comparisons from updated manuscript Table 11. MSE / MAE averaged over H = 96, 192, 336, and 720.

DatasetPatchBasisPatchTSTDUETTimeFilter
ETTm10.342 / 0.3670.355 / 0.3720.355 / 0.3740.367 / 0.378
ETTm20.250 / 0.3040.254 / 0.3100.252 / 0.3070.253 / 0.306
ETTh10.403 / 0.4170.422 / 0.4270.405 / 0.4190.421 / 0.426
ETTh20.352 / 0.3860.368 / 0.3970.362 / 0.3920.365 / 0.395
Weather0.220 / 0.2480.225 / 0.2530.241 / 0.2750.221 / 0.249

Selected baseline comparisons from updated manuscript Table 12. MSE / MAE averaged over H = 96, 192, 336, and 720.

DatasetPatchBasisPatchTSTDUET-CITimeFilter
ETTm10.380 / 0.3830.385 / 0.3850.404 / 0.3900.405 / 0.394
ETTm20.275 / 0.3160.282 / 0.3170.285 / 0.3240.273 / 0.317
ETTh10.430 / 0.4230.444 / 0.4360.443 / 0.4320.473 / 0.442
ETTh20.366 / 0.3890.371 / 0.3940.374 / 0.3970.373 / 0.397
Weather0.245 / 0.2630.255 / 0.2670.275 / 0.2830.242 / 0.264
Electricity0.185 / 0.2710.187 / 0.2720.222 / 0.3020.167 / 0.256
Traffic0.485 / 0.2860.487 / 0.2840.642 / 0.3680.454 / 0.274
Solar0.237 / 0.2460.297 / 0.3140.347 / 0.3170.245 / 0.272
Exchange0.371 / 0.4090.368 / 0.4090.357 / 0.3990.418 / 0.430

Against PatchTST, PatchBasis lowers horizon-averaged MSE and MAE on all five datasets in the search-based comparison. Under the uniform protocol, it lowers MSE on eight of nine datasets and, at the three-decimal precision shown in the table, MAE on seven datasets, with a tie on Exchange and higher MAE on Traffic. It also matches or improves on strong baselines such as DUET and TimeFilter in several settings, while losing in others. For example, TimeFilter performs better on Electricity under the uniform protocol. These results support competitiveness. W4 / Q4 below reports a statistically significant cross-dataset advantage over PatchTST; this does not imply superiority on every task or over every baseline.

AuthorsOpenReview ↗

Response to Reviewer hxbp (3/4)


[W4] / [Q4] The experimental results lack significance analysis.

Response: PatchBasis significantly outperforms PatchTST in the cross-dataset comparison for both MSE and MAE after Holm correction (adjusted p = 0.039063 for each metric). We apply a two-sided Wilcoxon signed-rank test to the nine paired dataset-level results in manuscript Table 12. Each result averages the four prediction horizons, using the three-seed mean errors. We use unrounded values and apply Holm correction across the two metric tests at a significance level of 0.05. The two mean-and-standard-deviation tables below additionally report variability across random seeds.

PatchBasis results for manuscript Table 11; mean ± sample standard deviation over seeds 2024, 2025, and 2026.

DatasetMetricH = 96H = 192H = 336H = 720
ETTh1MSE0.359 ± 0.0020.397 ± 0.0020.426 ± 0.0020.431 ± 0.009
ETTh1MAE0.384 ± 0.0030.408 ± 0.0010.425 ± 0.0020.449 ± 0.006
ETTh2MSE0.279 ± 0.0010.350 ± 0.0000.371 ± 0.0010.409 ± 0.008
ETTh2MAE0.330 ± 0.0000.378 ± 0.0000.398 ± 0.0010.436 ± 0.004
ETTm1MSE0.272 ± 0.0010.316 ± 0.0010.360 ± 0.0000.420 ± 0.001
ETTm1MAE0.323 ± 0.0010.352 ± 0.0010.378 ± 0.0010.414 ± 0.000
ETTm2MSE0.157 ± 0.0000.216 ± 0.0020.266 ± 0.0040.362 ± 0.001
ETTm2MAE0.241 ± 0.0010.281 ± 0.0020.316 ± 0.0020.377 ± 0.001
WeatherMSE0.145 ± 0.0020.187 ± 0.0010.238 ± 0.0030.310 ± 0.002
WeatherMAE0.183 ± 0.0020.224 ± 0.0010.265 ± 0.0020.319 ± 0.002

PatchBasis results for manuscript Table 12; mean ± sample standard deviation over seeds 2024, 2025, and 2026.

DatasetMetricH = 96H = 192H = 336H = 720
ETTh1MSE0.376 ± 0.0040.426 ± 0.0030.458 ± 0.0080.461 ± 0.011
ETTh1MAE0.386 ± 0.0020.416 ± 0.0010.435 ± 0.0030.457 ± 0.005
ETTh2MSE0.282 ± 0.0010.363 ± 0.0010.407 ± 0.0010.413 ± 0.000
ETTh2MAE0.329 ± 0.0000.380 ± 0.0000.415 ± 0.0000.433 ± 0.000
ETTm1MSE0.313 ± 0.0010.367 ± 0.0020.386 ± 0.0020.453 ± 0.002
ETTm1MAE0.342 ± 0.0020.371 ± 0.0010.390 ± 0.0020.428 ± 0.001
ETTm2MSE0.172 ± 0.0000.237 ± 0.0000.293 ± 0.0000.398 ± 0.000
ETTm2MAE0.250 ± 0.0010.293 ± 0.0000.331 ± 0.0000.392 ± 0.000
WeatherMSE0.163 ± 0.0020.208 ± 0.0000.265 ± 0.0020.344 ± 0.001
WeatherMAE0.198 ± 0.0020.236 ± 0.0000.282 ± 0.0010.335 ± 0.000
ECLMSE0.157 ± 0.0020.172 ± 0.0020.188 ± 0.0010.223 ± 0.001
ECLMAE0.248 ± 0.0000.263 ± 0.0000.271 ± 0.0000.301 ± 0.000
TrafficMSE0.461 ± 0.0010.467 ± 0.0000.486 ± 0.0010.526 ± 0.004
TrafficMAE0.274 ± 0.0000.281 ± 0.0000.286 ± 0.0000.303 ± 0.001
Solar-EnergyMSE0.206 ± 0.0050.238 ± 0.0090.249 ± 0.0050.254 ± 0.010
Solar-EnergyMAE0.217 ± 0.0040.250 ± 0.0050.253 ± 0.0060.264 ± 0.008
ExchangeMSE0.083 ± 0.0020.187 ± 0.0000.348 ± 0.0020.867 ± 0.002
ExchangeMAE0.200 ± 0.0030.306 ± 0.0000.426 ± 0.0010.703 ± 0.001

All values are rounded to three decimals. A displayed standard deviation of 0.000 means that it rounds below 0.001; it does not imply zero variation.

Two-sided Wilcoxon signed-rank tests comparing PatchBasis and PatchTST in manuscript Table 12. Nine paired dataset-level averages; Holm correction across MSE and MAE.

MetricRaw p-valueHolm-adjusted p-value
MSE0.0195310.039063
MAE0.0273440.039063

Both Holm-adjusted p-values are below 0.05. This supports an overall cross-dataset advantage over PatchTST, rather than significance on every individual dataset or prediction horizon.

The three-seed standard deviations describe training variability. The Wilcoxon tests above assess paired differences across datasets. These are complementary forms of evidence.

Reviewer_xrL9OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes PatchBasis for time series forecasting. It learns weights for each input to combine historical patches into fewer tokens and adds recent information through a time-decay rule. Experiments on nine datasets compare forecasting accuracy, examine the two components, and report computational costs.

Critical strengths
  1. The motivation is clear. Different historical regions can have different forecasting value, so adapting their aggregation to each input is reasonable. The results show competitive accuracy across several benchmarks, with larger improvements on Electricity.

  2. The paper includes useful ablations and sensitivity studies. It also reports limitations directly: the plugin does not improve all backbones, and fewer tokens and parameters do not necessarily lead to faster inference.

Critical weaknesses
  1. The technical innovation is unclear. Different historical observations have different forecasting value. This is already a common motivation for attention and temporal weighting. The question is what new method follows from this observation. BasisLearner uses an MLP and softmax to compute aggregation weights, while the recent branch uses a fixed time-decay gate. The paper needs to explain what these designs enable beyond existing attention and patch aggregation methods. Table 3 provides useful comparisons, but the representations and search spaces differ, making the benefit of the proposed aggregation difficult to isolate.

  2. The numerical results are promising, but stronger evidence is needed to support the contribution. Several gains over the strongest baselines are small, and the main comparisons are not repeated across random seeds. The separate three-seed study only compares against PatchTST at one prediction horizon. Also, Table 1 inherits configurations originally selected using test performance. Please provide repeated comparisons with the strongest baselines using validation-only selection. The recent-branch ablation should also show whether the specific time-decay rule is better than simply retaining recent patches.

Questions for authors
  1. What is the main technical difference between BasisLearner and attention pooling? Can you compare them using the same patch representations, token count, backbone, and tuning budget?

  2. Is the improvement from retaining recent information or from the proposed time-decay rule? How does it compare with equal weighting or a learned constant gate?

  3. Are the gains over the strongest baselines consistent across random seeds when all configurations are selected using validation data only?

4 comments
AuthorsOpenReview ↗

Response to Reviewer xrL9 (1/3)

Thank you for your careful review and constructive comments.


[W1] The technical innovation is unclear. […] The paper needs to explain what these designs enable beyond existing attention and patch aggregation methods. […] the representations and search spaces differ, making the benefit of the proposed aggregation difficult to isolate.

Response: Our contribution is to construct an effective instance-adaptive token basis for forecasting and restore local information lost during aggregation. BasisLearner is the implementation used to study this question.

The manuscript already distinguishes aggregation designs by how they generate weights and use the resulting tokens. Related Work states: “Unlike TimeBase’s instance-shared weights, BasisLearner generates aggregation weights from each input.” Section 3.1 contrasts the shared mapping T = AP with the input-dependent mapping T = A(P)P. The method section explains its effect: “Recomputing A from each input lets the same token slot draw on different historical regions across instances.” The manuscript also distinguishes how compact tokens enter the predictor: “These latent tokens update the original patch sequence before decoding, whereas PatchBasis passes compact tokens directly to the forecasting backbone.” For patch selection and replacement, Related Work states: “SRSNet selects and reorders informative patches, whereas BasisLearner combines multiple patches within each token” “SEER preserves the token count during replacement, whereas PatchBasis aggregates the history into a fixed budget of K tokens.”

The Recent branch addresses information loss within that fixed budget: “Beyond global aggregation, PatchBasis incorporates recent patches through a time-decay prior to supplement features lost during aggregation.”

PMA and Perceiver generate pooling weights through query–key attention with learnable queries. BasisLearner directly predicts each patch’s contribution to each output slot with a nonlinear MLP. PatchBasis also blends recent patches into the same K output tokens. This supplies readily available recent information without learning a separate extractor. PMA targets set aggregation; Perceiver targets efficient processing of large inputs. Our focus is a token basis suited to forecasting. Operator similarity alone does not resolve this contribution.

We have added the attention-related comparison to Related Work. The controlled attention-pooling results appear in Q1 below. We have also revised manuscript Table 4 by removing the Full patch configuration with patch/stride = 48/24 and aligning shared search ranges. Full patch, TimeBase, and BasisLearner now use 8/8, 16/16, and 24/24. TimeBase and BasisLearner both search K ∈ {6, 12}. Tokenizer-specific settings remain necessary and are disclosed. Native tokenizers can also differ in their representations. The shared lightweight comparison below reduces these differences and matches parameter counts.

The lightweight comparison uses TimeBase-MLP-SameRank-PM and BasisLearner (Instance-TL + Position). It uses six datasets, L = 720, H = 336, and three seeds. K, depth, GELU, dropout, and the forecasting backbone are matched. The maximum relative difference in total parameter count is 1.153%. Each method selects its learning rate by validation MSE at patch size 24, then reuses it for the other patch sizes.

Shared lightweight pipeline; L = 720, H = 336, patch size 24. Three-seed mean test MAE.

DatasetTimeBase MLPBasisLearner
ETTh10.4280.421
ETTh20.4180.410
ETTm10.4110.386
ETTm20.3630.332
Weather0.3400.299
Electricity0.2800.259

BasisLearner achieves lower mean MAE on all six datasets. This comparison supports input-adaptive representations in a shared forecasting pipeline. Its scoring structure and weight constraints still differ from TimeBase MLP, so it is not a single-factor replacement of input dependence in the complete PatchBasis model.

AuthorsOpenReview ↗

Response to Reviewer xrL9 (2/3)


[W2] The numerical results are promising […] Please provide repeated comparisons with the strongest baselines using validation-only selection. The recent-branch ablation should also show whether the specific time-decay rule is better than simply retaining recent patches.

Response: The updated main comparisons use validation-MSE configuration selection and three seeds; the results show competitiveness, not a universal winner. The two comparison tables below compare PatchBasis with PatchTST, DUET or DUET-CI, and TimeFilter using the updated results.

Selected baseline comparisons from updated manuscript Table 11. MSE / MAE averaged over H = 96, 192, 336, and 720.

DatasetPatchBasisPatchTSTDUETTimeFilter
ETTm10.342 / 0.3670.355 / 0.3720.355 / 0.3740.367 / 0.378
ETTm20.250 / 0.3040.254 / 0.3100.252 / 0.3070.253 / 0.306
ETTh10.403 / 0.4170.422 / 0.4270.405 / 0.4190.421 / 0.426
ETTh20.352 / 0.3860.368 / 0.3970.362 / 0.3920.365 / 0.395
Weather0.220 / 0.2480.225 / 0.2530.241 / 0.2750.221 / 0.249

Selected baseline comparisons from updated manuscript Table 12. MSE / MAE averaged over H = 96, 192, 336, and 720.

DatasetPatchBasisPatchTSTDUET-CITimeFilter
ETTm10.380 / 0.3830.385 / 0.3850.404 / 0.3900.405 / 0.394
ETTm20.275 / 0.3160.282 / 0.3170.285 / 0.3240.273 / 0.317
ETTh10.430 / 0.4230.444 / 0.4360.443 / 0.4320.473 / 0.442
ETTh20.366 / 0.3890.371 / 0.3940.374 / 0.3970.373 / 0.397
Weather0.245 / 0.2630.255 / 0.2670.275 / 0.2830.242 / 0.264
Electricity0.185 / 0.2710.187 / 0.2720.222 / 0.3020.167 / 0.256
Traffic0.485 / 0.2860.487 / 0.2840.642 / 0.3680.454 / 0.274
Solar0.237 / 0.2460.297 / 0.3140.347 / 0.3170.245 / 0.272
Exchange0.371 / 0.4090.368 / 0.4090.357 / 0.3990.418 / 0.430

In the search-based comparison, PatchBasis has lower horizon-averaged MSE and MAE than PatchTST on all five datasets. In the uniform comparison, it has lower MSE on eight of nine datasets and, at the three-decimal precision shown in the table, lower MAE on seven datasets, a tie on Exchange, and higher MAE on Traffic. Results against DUET and TimeFilter depend on the dataset. The gate and recent-only controls in Q2 separate recent-information retention from the decay mechanism. Q3 gives the rerun protocol. These mean comparisons do not establish statistical significance.


[Q1] What is the main technical difference between BasisLearner and attention pooling? […] the same patch representations, token count, backbone, and tuning budget?

Response: The key technical difference is the scoring function: BasisLearner predicts pooling scores with an MLP; the query–key attention-pooling comparator uses dot products. Both normalize over patches and form weighted sums. Our broader aim is to test whether an instance-adaptive token basis improves forecasting over retaining the original patches.

Section 3.2 already states: “We apply a shared two-layer MLP to each contextual patch, producing K scores per patch, one for each output token.”

The controlled comparison disables Recent. BasisLearner and attention pooling share the patch-representation architecture, shared-module initialization, K, Transformer backbone, and Flatten head. Both use MAE training, at most 50 epochs, patience 5, and the same three-learning-rate search budget. Learning rates are selected by mean validation MSE over three seeds. Full Patch uses the same patch settings and retains all patches.

Controlled tokenizer comparison; L = 192, H = 96. Three-seed mean test MSE / MAE.

DatasetFull PatchBasisLearnerAttention pooling
ETTh10.372 / 0.3910.361 / 0.3840.363 / 0.384
ETTm10.296 / 0.3340.293 / 0.3320.300 / 0.338
Weather0.159 / 0.1930.156 / 0.1880.163 / 0.202

Patch/stride and pooled token counts are 24/24 and K = 4 for ETTh1, 24/12 and K = 6 for ETTm1, and 12/12 and K = 8 for Weather.

BasisLearner has lower unrounded mean MSE and MAE than both comparators in all three settings. This supports the value of the learned token basis and the MLP scoring choice in these experiments. The ETTh1 MAE difference from attention pooling becomes a tie after rounding to three decimals.

AuthorsOpenReview ↗

Response to Reviewer xrL9 (3/3)


[Q2] Is the improvement from retaining recent information or from the proposed time-decay rule? […] equal weighting or a learned constant gate?

Response: Retaining recent information explains part of the gains; the best gate depends on the dataset. The submitted Figure 4 and Table 10 already remove Recent and compare recent versus uniformly spaced candidates. That tests candidate positions. The comparison below tests no Recent, equal weighting, a learned constant gate, and decay while holding the remaining architecture, training, and tuning budget fixed.

Gate comparison; L = 192, H = 96. Three-seed mean test MSE / MAE.

GateETTh1ETTm1Weather
No Recent0.370 / 0.3840.293 / 0.3320.158 / 0.194
Equal weighting0.355 / 0.3800.289 / 0.3300.158 / 0.195
Learned constant0.366 / 0.3810.289 / 0.3300.158 / 0.195
Decay0.366 / 0.3820.291 / 0.3310.155 / 0.190

All three gates lower unrounded mean MSE relative to no Recent on all three datasets. Equal weighting is best on ETTh1. The learned constant gate is best on ETTm1 before rounding; it ties with equal weighting at three decimals. decay is best on Weather. Equal weighting and the constant gate slightly worsen Weather MAE relative to no Recent. We therefore do not claim that decay is always best.

Recent-only and branch ablations; L = 192, H = 96. Three-seed mean test MSE / MAE.

MethodETTh1ETTm1Weather
Recent-only0.378 / 0.3900.303 / 0.3360.162 / 0.198
BasisLearner without Recent0.370 / 0.3840.293 / 0.3320.158 / 0.194
Full model with Recent0.366 / 0.3820.291 / 0.3310.155 / 0.190

The full model also outperforms retaining only the latest K patches. Its mean MSE reductions are 3.110%, 3.790%, and 4.510% on ETTh1, ETTm1, and Weather. These controls show that the gains involve both historical aggregation and recent-information retention.


[Q3] Are the gains over the strongest baselines consistent across random seeds when all configurations are selected using validation data only?

Response: We have rerun manuscript Tables 1 and 2 with validation-MSE configuration selection and seeds 2024, 2025, and 2026. The updated tables report means over these runs. We also reran strong baselines whose results differed substantially from their published results. Their seed-2025 checkpoints have been updated for inspection.

SEER did not converge well on Electricity with the original learning rate of 0.010. We changed it to 0.001. The revised experimental settings specify the learning rate of 0.001 for Electricity. The updated comparisons are reproduced in the two tables in our response to W2. They show average performance across seeds; they do not imply that every gain holds for every seed.


We hope these responses address your concerns, and we would be happy to provide further clarification during the discussion period.

AuthorsOpenReview ↗

Follow-up on Our Rebuttal to Reviewer xrL9

Dear Reviewer xrL9,

Thank you again for your thoughtful review. We have provided detailed responses to your concerns, including controlled comparisons with attention pooling, recent-branch gating ablations, and three-seed evaluations with validation-based configuration selection.

When you have a chance, we would appreciate your feedback on whether these additional results and clarifications address your concerns. We would be happy to answer any remaining questions during the discussion period.

Thank you for your time and consideration.

Best regards,
The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.