acceptodds
Under review as a conference paper at ICLR 2027

Adaptive Patching Is Harder Than It Looks for Time-Series Forecasting

Abstract

Adaptive patching has been proposed for time-series forecasting: allocate finer patches where the sequence looks locally informative. We compare three adaptive patching methods with tuned uniform counterparts that use one fixed patch size throughout the input, while preserving each method's backbone family and shared training protocol. Across the evaluated long-horizon benchmarks, we observe mostly small accuracy differences and no consistent advantage for adaptive patching; uniform variants generally train faster and avoid adaptive routing at inference. Motivated by these observations, we develop a theoretical framework for budgeted patch allocation through a common resolution-density representation of uniform, nonuniform, and overlapping layouts, independently of the patching rule. Within a separable shared-loss model, we derive an exact criterion for when nonuniform allocation improves on uniform allocation at the same budget. We characterize modeled gains locally through a quadratic surrogate and bound them globally under strong convexity and a bounded loss derivative. The analysis distinguishes where resolution is placed from how much it is redistributed: even an allocation that correctly ranks regions by their weights can redistribute too much resolution and increase modeled loss. Without coupling across time, positive regional weights preserve the minimizers of the shared loss. When the shared loss has a common minimizing rate, the remaining absolute headroom for improvement tends to zero as the uniform rate approaches it. These results characterize the roles of regional weighting, placement, and redistribution strength under the stated model. The empirical findings support evaluating adaptive patching against a tuned uniform baseline and weighing any accuracy gain against the additional training and inference cost of adaptive routing.

Primary area:
learning on time series and dynamical systems
Keywords:
time-series forecasting, adaptive patching, dynamic tokenization, time-series Transformers, rate-distortion theory

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CBuCOpenReview ↗
Rating
2
Confidence
2(−1)
Informativeness
2
Presentation
2
Soundness
3(+1)
Read review
Paper summary

This paper compares the effects of adaptive blocking and optimized uniform blocking when applied to TSF. The author designed comparative experiments on eight benchmarks using three adaptive blocking methods. The prediction accuracy of both is close, while uniform blocking typically reduces computational costs. The paper further proposes a framework for budget allocation, derives the conditions under which non-uniform allocation is superior to uniform allocation, and characterizes the respective roles of region weighting, allocation alignment, and resolution discretization.

Critical strengths
  1. This paper challenges the theory of the superiority of adaptive blocking through comparative experiments involving two blocking paradigms, providing valuable insights into understanding the trade-off between accuracy and efficiency.

  2. The proposed resolution-density framework offers a unified perspective for block allocation and establishes theoretical analysis for the effectiveness of non-uniform allocation.

Critical weaknesses
  1. The theoretical analysis is not fully substantiated by experiments. It assumes that the objective function is separable and employs a common resolution loss function and regional weights. However, these variables have not been estimated or validated on actual predictive models.

  2. The font size in Figure 1 is too small, and the experimental analysis merely repeats percentage figures without in-depth exploration. Chapter 4 is difficult to understand.

  3. The paper only evaluates three methods, which limits the universality of the conclusions to different adaptive blocking methods.

Questions for authors

See weakness.

7 comments
AuthorsOpenReview ↗

Part 1/5: W1 (1/3), the assumptions of Section 4 and what depends on them

Before the individual answers, we invite you to read our general comment, "A thought experiment for Section 4", which explains the theory through an analogy.

Thank you for the review. In short: the separable shared-loss form is the model the paper states and studies, and this part says what it assumes and what it leaves out. Its two terms are of different kinds: the weights belong to the series and are exact on the series we built, and the loss is the error at each uniform patch length, which we measure on trained forecasters (Part 2). Its predictions are then checked on trained forecasters, where placement has no effect and where it does (Parts 2 and 3).

The first sentence of W1 gives the assumptions of Section 4 as the paper states them: the introduction says "As one instantiation, we analyze separable objectives in which regional weights KtK_t scale a shared loss response D(rt)D(r_t) to resolution", and the abstract places the results "within a separable shared-loss model".

Model perspective. Start from equal patches and, keeping their number, make them slightly finer on some steps and coarser on others. To first order, the change in error is then a sum of one term per step, for any forecaster whose error varies smoothly with resolution; KtK_t measures how strongly the error responds at step tt, up to a common factor (Sec. 4.1). So at first order separability is not an assumption. What the model assumes comes after this first approximation: refining one region does not change what refining another is worth, and the error follows the same convex curve DD of resolution in every region, scaled by that region's weight. Because the curve is convex, unequal patches cost something in themselves (the Jensen penalty), and a placement wins only if its fine patches are on steps that matter enough to outweigh this cost. That is what the exact criterion (Thm 4.4) says: putting the fine patches on the steps that matter is necessary but not sufficient. An analogous trade-off holds for any error that is convex in the resolutions, separable or not: the first-order gain from placement must exceed a nonnegative remainder that vanishes for equal patches. The model writes down the premise of adaptive patching itself, that some regions deserve more resolution than others, with KtK_t saying how much each region gains from it. Each method picks these regions by its own signal: EntroPE starts a new patch where its entropy model is uncertain about the next value, and TimeMosaic, in the words of its abstract, adjusts patch granularity "according to local information density".

Why one shared loss. A shared loss says that regions differ in how much resolution matters, not in which resolution is best. Section 4 studies this case on purpose. Appendix A.1 separates it from the other mechanisms by which unequal patches could help, such as a different loss curve in each region ("different time steps have different ℓt(r)\ell_t(r)"), and shows that with one shared loss and no budget the weights never favour unequal patches: the fixed budget is what makes them matter. The shared loss is also what lets the bound of Thm 4.10 be evaluated from an ordinary uniform baseline, as the next paragraph says.

What depends on the weights. Section 4 takes the weights as given: its results hold for any non-negative weights, and only Prop 4.6 needs them to differ. The result we use to interpret Section 3, Thm 4.10, needs no estimate of them: in the shared-loss model, if one resolution is best for every region, the gain of any placement is at most the uniform model's error at the same number of patches minus its error at the best uniform patch length. Sec. 4.6 notes that this bound "requires only a common global minimum and nonnegative weights". To evaluate it we need only the uniform model's validation error at each patch length of its tuning grid, which tuning the uniform baseline already gives (Appendix C.5, for placements whose patch lengths stay inside the grid). The criterion of Thm 4.4 also holds for any weights, and so does Prop 4.5: a placement that is statistically independent of the weights cannot beat uniform patching. When finer patches lower the loss (Sec. 4.3 to 4.5), how much the weights differ enters the size of the possible gain: the maximum of the local quadratic surrogate (Thm 4.7) and the upper bound of Thm 4.8 both grow with the variance of the weights (Part 5 states each result in plain words).

The flat curve outside our study. PatchTST reports that its "MSE scores don't vary significantly with different choices of P" (patch lengths 4 to 40, Appendix A.4.1), and TimeXer that "the average prediction performance does not vary dramatically with different patch lengths" (Appendix B.2). In the model, an error that hardly changes with the uniform patch length leaves little room for any placement (Thm 4.10). Section 3 finds small differences, in both directions.

AuthorsOpenReview ↗

Part 2/5: W1 (2/3), the two terms of the model: the weights of the series and the loss of the forecaster

We read the second sentence of W1 as asking for the values of the two terms of the model, the weights KtK_t and the shared loss, and for a check of the model on trained forecasters. Please correct us if we have misread it. The two terms are of different kinds. The weights belong to the series: they say how much the forecast depends on each input step. The shared loss belongs to the forecaster: it says how its error changes with the patch length.

The weights. On benchmark data they are unknown, so we designed a series on which they are exact. Each input window has 96 steps in three regions of 32 steps:

  • white noise that is independent of the future, with the largest variance of the three regions;
  • oscillations that continue into the future;
  • a level with a slope, on which the future also depends.

The series is linear and Gaussian, so the weights are exact: the oscillations carry 78 % of the forecast's weight, the level 22 % and the noise 0 %.

The shared loss. We train EntroPE, HDMixer and TimeMosaic on this series with unchanged code and the candidate grids of Section 3 (six settings: three noise levels, two horizons; 10 seeds). In the model, the error of uniform patching is Kˉ\bar K times the shared loss at that patch length, so we measure the test error at each uniform patch length. At any uniform patch length it is at most 0.4 / 1.0 / 1.7 % above the error at the best one (EntroPE / HDMixer / TimeMosaic). The reason is in the backbones: the values inside a patch remain available to the forecast.

  • EntroPE: the forecast head receives one state for every input step, whatever the patch length.
  • TimeMosaic: each patch becomes a token with more features than the patch has values, and the encoder also gets extra tokens that see the whole input.
  • HDMixer (uniform model): a patch becomes a token of 16 numbers, which can hold every value up to 16-step patches.

On the benchmarks the error also changes little with the patch length: the difference between the validation error of the uniform model at the adaptive model's number of patches and its error at the best uniform patch length is a median 0.25 / 1.08 / 0.38 %. By Thm 4.10 this difference bounds the gain of any placement, for any weights.

What the two predict. The weights are far from equal, so a placement that follows them can gain, but by Thm 4.4 only as much as the loss changes with the patch length. For the published forecasters the loss is flat, so the model predicts that placement has no effect, in either direction. That is what we measure. EntroPE's selector puts its short patches on the noise, and its error is that of uniform patching. And when we impose on published EntroPE placements that follow the weights, oppose them or ignore them, its error stays within 0.4 % of uniform patching in every case. To see a placement gain or lose, a patch has to lose information (Part 3).

In the revised manuscript (new text in blue): main text, new Section 5; appendix, Table 3 (the three regions and their exact weights) and Table 7 (the bound on the benchmarks, per method).

AuthorsOpenReview ↗

Part 3/5: W1 (3/3), the model checked where a patch loses information

Placement can matter only if a patch loses information, and in the published models it does not: the values inside a patch remain available to the forecast (Part 2). We make a patch lose information in two ways.

Patch averages. We replace every input patch by its average before it enters the forecaster. The longer the patch, the more detail is averaged away, so it now matters which region gets the short patches. The forecaster is the same network in every comparison, an MLP-mixer backbone (the one of HDMixer, without any patch selector). We impose the patching ourselves and train the network from scratch for each placement, so the placement is the only thing that changes. The baseline is uniform patching with 16-step, 8-step or 4-step patches. Each placement uses the same number of patches as its baseline and only changes their lengths. We fix three placements:

  • aligned: patches half as long as the uniform ones on the oscillations, and twice as long on the noise and on the level;
  • reversed: the same lengths, with the short patches on the noise;
  • random: the same lengths in shuffled order.

They use the same patch lengths, so they have the same Jensen penalty and differ only in the alignment gain.

Thm 4.4 predicts the gain of each placement before it is trained, from the exact weights and this forecaster's error at each uniform patch length; nothing is fitted to the placements. The predicted gain is the alignment gain (the benefit of putting the short patches on the important steps) minus the Jensen penalty (the cost of unequal patch lengths at the same number of patches). For the aligned placement, in % of the error of the uniform baseline (mean over the six settings, positive = better than uniform):

Uniform baselineAligned: oscillations / noise and levelAlignment gainJensen penaltyPredicted gainMeasured gain
16-step patches8-step / 32-step10.81.7+9.1+10.9
8-step patches4-step / 16-step13.77.5+6.2+4.1
4-step patches2-step / 8-step8.69.6−1.0−2.5

The aligned placement beats 16-step and 8-step uniform patches, as predicted. In the last row the short patches are still on the right region, yet the criterion predicts a loss against 4-step uniform patches, because the Jensen penalty now exceeds the alignment gain, and the loss is measured in each of the six settings (the case of Thm 4.7). In all 18 cases (3 baselines × 6 settings) the predicted sign is the measured one, and the two gains differ by 1.7 points in median.

The reversed placement has a negative alignment gain and loses against all three baselines, as predicted: −9.8 / −17.7 / −16.0 % predicted, −9.8 / −17.7 / −16.1 % measured. The random placement has an alignment gain near zero, and the criterion correctly predicts that it loses in every setting: −2.6 / −8.0 / −9.7 % predicted, −2.6 / −3.9 / −3.9 % measured. We also imposed the boundaries of EntroPE's entropy model, which put the short patches on the noise: the predicted gain is −15.5 % and the measured one −14.6 %. With the backbones of EntroPE or TimeMosaic, or a linear map, in place of the MLP mixer, the order against 8-step uniform patches is the same in every setting: aligned gains, random loses, reversed loses most.

Closing a path of the published model. The second way leaves the input untouched. EntroPE's forecast head receives one state per input step, a path that brings the individual values back after patching. We closed it, so the input reaches the forecast only through the patches. On ETTh1 and ETTm1 (horizon 96, 5 seeds), the error then varies by 45 % and 9.0 % across uniform patch lengths 4 to 32, against 1.9 % and 1.0 % as published. EntroPE's selector, within the variance across seeds of uniform patching before (+1.2 %, −0.9 %), is then worse in every seed (−8.3 %, −3.1 %).

In the revised manuscript (new text in blue): main text, new Figure 6 (error against uniform patch length; the plane of Thm 4.4 with one dot per placement; predicted against measured gain); appendix, Tables 6 (both terms, predicted and measured gain of every placement), 5 (the other forecasters) and 8 (EntroPE with the step path open and closed).

Reviewer_wRcNOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper uses smaller-grained patches for local parts of the sequence that are information-dense, abrupt and complex in patterns, and large patches for simple regions, improving prediction performance under a fixed token budget.

Critical strengths
  1. Beyond reporting end-to-end wall-clock times, the authors isolate the exact routing overhead by caching generated tokens and timing the backbone in isolation. This is what I consider the highlight of the paper.
  2. This paper provides substantial theory to justify the proposed conclusions and uses lean formalization with axiomatic auditing to eliminate ambiguity regarding the validity of the proofs. This helps me confirm the validity of the theory in this paper.
Critical weaknesses
  1. The theoretical framework in Section 4 is built on a fixed nominal resolution budget. However, the uniform baseline can freely choose pU∈{4,8,16,24,32}p_U\in\{4,8,16,24,32\}, which leads to substantial differences in the token count NN, nominal resolution budget rˉ\bar{r}, and even model parameter size. This is where I consider the experiments to be insufficient.
  2. There is a lack of empirical connection between Section 3 and Section 4. It does not measure the empirical resolution dispersion σr\sigma_r generated by EntroPE, HDMixer or TimeMosaic, nor does it conduct synthetic or semi-synthetic experiments. This is also where I consider the experiments to be insufficient.
Questions for authors
  1. Can you report a budget-matched comparison from your existing tuning runs where the adaptive configuration is compared against the uniform patch length pUp_U that most closely matches its effective token count NN?
  2. Can you design a controlled synthetic forecasting experiment to plot where actual adaptive selectors fall in the Alignment Gain and Jensen Penalty plane of Figure 5?
  3. How do your alignment and over-redistribution bounds generalize when a selector imperfectly estimates rt∗r_t^* under a budget constraint E[rt]=rˉ\mathbb{E}[r_t] = \bar{r}?
7 comments
AuthorsOpenReview ↗

Part 1/4: W1 and Q1, budget-matched comparison (token count N and budget r̄), and parameter size

Before the individual answers, we invite you to read our general comment, "A thought experiment for Section 4", which explains the theory through an analogy.

Thank you for these comments. We ran the budget-matched comparison of Q1, and at a matched budget the conclusion of Section 3 is unchanged. New text in the revised paper is marked in blue.

In Table 1 the adaptive model is compared with the uniform patch length that has the best validation error, whatever its number of patches. Here we compare it with the uniform patch length that has the closest number of patches. For each of the 96 settings (3 methods × 8 datasets × 4 horizons) we measure the rate N/TN/T of the adaptive model (patches per input step, the rˉ\bar r of Section 4) and take the uniform patch length of the grid whose rate is closest. This uniform model is trained and tested as in Table 1, with the same five seeds. After matching, the rates of the two models differ by a median 8.8 / 2.4 / 9.7 % (EntroPE / HDMixer / TimeMosaic). In the table, "selected" is the comparison of Table 1 and "matched" is the budget-matched one. G=100(A−U)/AG = 100(A-U)/A as in Table 1, with AA and UU the test MSE of the adaptive and of the uniform model; positive favours uniform.

Methodmean GG, selected → matcheduniform better (of 32)within ±2 % (of 32)signed-rank pp, matched
EntroPE+0.20 → −0.1118 → 1626 → 280.95
HDMixer+1.81 → +1.0923 → 1823 → 260.55
TimeMosaic−0.10 → −0.0912 → 1527 → 261.00

Over all settings, 80 of 96 are within ±2 % (76 before), and no setting is significant after correction (smallest BH q=0.35q = 0.35). The matching changes most where Table 1 has its largest gap: for HDMixer on Electricity the advantage of the uniform model falls from +10.26 % to +5.54 %, because the selected uniform model used 8- or 16-step patches where the adaptive model's rate corresponds to 4- or 8-step patches. At a matched budget the two arms are therefore even closer than in Table 1. On cost, uniform inference stays 1.96× / 4.28× / 1.08× faster at the matched rate, measured as in Table 1 (1.94× / 5.63× / 1.08× there), consistent with the routing overhead isolated in Figure 4.

In the revised manuscript (new text in blue): appendix, Section D.1, Tables 15 (budget-matched comparison per method) and 16 (per setting).

AuthorsOpenReview ↗

W1, parameter size

Matching the rate also removes most of the mismatch in parameter size, the third one you list.

The patch length changes the parameter count of the uniform model only for HDMixer, whose linear head reads one dmodeld_{\rm model}-wide vector per patch. For EntroPE and TimeMosaic the uniform model has the same parameters at every patch length. Averaged over the datasets, the adaptive model has 1.09 / 2.03 / 1.00 times as many parameters as the selected uniform model (EntroPE / HDMixer / TimeMosaic), and 1.09 / 1.23 / 1.00 times as many as the rate-matched one. The comparison above is therefore also close to parameter-matched. What remains for HDMixer comes mostly from the network that predicts its patch offsets, which the uniform model does not have.

In the revised manuscript (new text in blue): appendix, Table 15 (parameter ratio at the selected and at the matched patch length).

AuthorsOpenReview ↗

Part 2/4: W2 and Q2 (1/2), a synthetic series with known weights, and where the three selectors fall

Thank you for W2 and Q2, which shaped a new section of the paper. Once the actual selectors are placed in the plane of alignment gain against Jensen penalty (the two terms of Thm 4.4), the plane explains the small differences of Section 3: all three selectors sit at its origin. The error of the three published forecasters hardly changes with the patch length, so no placement can gain much; in a forecaster where a patch loses information, placement matters, and Thm 4.4 predicts the sign of the gain of each placement we fix (Part 3). Placing a selector in the plane needs the weights KtK_t, which are unknown on benchmark data, so we designed a series on which they are exact. Each input window has 96 steps in three regions of 32 steps:

  • white noise that is independent of the future, with the largest variance of the three regions;
  • oscillations that continue into the future;
  • a level with a slope, on which the future also depends.

The series is linear and Gaussian, so the weights are exact: the oscillations carry 78 % of the forecast's weight, the level 22 % and the noise 0 %. The noise region is the most complex part of the input and has no weight in the forecast, so a selector that follows the complexity of the input and a selector that follows what the forecast needs must choose different regions. The two informative regions are independent, so the separable objective of Section 4 holds by construction. We use six settings (three noise levels, two horizons) and 10 seeds, and train the three methods with unchanged code and the candidate grids of Section 3.

For every trained adaptive model we read the rate rtr_t at each input step on the test set (Def. 4.1) and compute σr\sigma_r, Corr⁡(K,r)\operatorname{Corr}(K, r) and the two terms of Thm 4.4, with the exact KtK_t and no fitted parameter. Mean over all candidates and settings, range in parentheses:

EntroPEHDMixerTimeMosaic
σr/rˉ\sigma_r / \bar r0.74 (0.08 to 1.21)0.21 (0.19 to 0.24)0.12 (0.02 to 0.28)
Corr⁡(K,r)\operatorname{Corr}(K, r)−0.41−0.01−0.03
largest alignment term (absolute), % of uniform loss0.120.030.01
largest Jensen penalty, % of uniform loss0.220.080.00

EntroPE's selector has the largest dispersion and puts its fine patches on the noise. The rates of HDMixer and TimeMosaic do not follow the weights. All three sit at the origin of the plane: both terms stay below 0.25 % of the loss. The reason is that the error of the published forecasters hardly changes with the uniform patch length (at any uniform patch length the test error is at most 0.4 / 1.0 / 1.7 % above the error at the best one), so D(r)D(r) is flat and both terms are near zero whatever the rates are. On the benchmarks the error also changes little with the patch length: the difference between the validation error of the uniform model at the adaptive model's number of patches and its error at the best uniform patch length is a median 0.25 / 1.08 / 0.38 %.

In the revised manuscript (new text in blue): main text, new Section 5 (the synthetic series and the placement tests); main text, new Figure 6 (a: error against uniform patch length; b: the plane of Thm 4.4 with one dot per placement; c: predicted against measured gain); appendix, Table 4 (σr\sigma_r, Corr⁡(K,r)\operatorname{Corr}(K, r) and the two terms of Thm 4.4 for each method); appendix, Table 7 (the same difference on the benchmarks, per method).

Reviewer_Ve99OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary
  • The central question of the paper is well-posed (When does adaptive patching fail) and explains a negative result.
  • The theoretical results are reasonable: adaptively patching the time series in low heteroscedasticity/uncorrelated error case shouldn't be helpful.
Critical strengths
  • The central question of the paper is well-posed (When does adaptive patching fail) and explains a negative result.
  • The theoretical results are reasonable: adaptive patching shouldn’t help when regional loss weights are nearly homogeneous or when the allocation is uncorrelated.
Critical weaknesses
  • The work could improve in clarity. Some sections exhibit a heavy use of jargons, assume the reader's familiarity with certain terms and notation before introducing them. For example line 195: "Let DdhD_dh and UdhU_dh denote mean test MSE over seeds" mean test MSE of what? It had to be assumed that DD corresponds to the adaptive patching error and UU to the uniform one. There is also overloading/notation clash in various places.
  • The theoretical work is presented as a framework for budgeted adaptive patching. However, I am struggling to to not see it more as strictly an explanation for why adaptive patching does not offer any consistent benefits. The theory explains why adaptive patching often does not provide any benefit, and indeed the experiments agree, most experiments exhibit little or no relative gain from adaptive patching.
Questions for authors
  1. Is the authors' intent to provide theroetical explanations for why adaptive patching hardly provides benefits, or rather to provide insights into how to enhance adaptive patching, or both?
  2. It seems that TimeMosaic is the only studied method that consistently shows gains over uniform patching (albeit very small ones). Do the authors have an explanation about why this could be? It is also interesting that this is so in the lowest overhead method.
  3. I am a little confused about Section 4.3 "For a decreasing loss". Do the authors mean decreasing over the prediction horizon? If so, is this a common/reasonable case to study? Given that residuals should compound for more distant timesteps, this seems to be a rare case. Do the authors observe it often in their experiments?
  4. How does the choice of backbone interact with the effectiveness/ineffectiveness of patching?
6 comments
AuthorsOpenReview ↗

Part 1/5: W2 and Q1 (1/2), what the paper is for

Part of your questions go to the purpose of Section 4. Before the individual answers, we invite you to read our general comment, "A thought experiment for Section 4", which explains the theory through an analogy.

Thank you for these comments. Thinking about them led us to new experiments that clarified the scope of the paper. In short, we used Thm 4.4 to characterise a limitation that the current adaptive methods share, and to show what a method needs in order to gain.

The paper is meant as an explanation of when adaptive patching can beat a tuned uniform patch length, and why it often does not. The explanation is Thm 4.4, an "if and only if": a placement beats uniform patching exactly when its alignment gain (short patches where the forecast depends on the input) exceeds its Jensen penalty (the cost of making the patch lengths unequal). Read the other way, the theorem gives the design insight, in three steps:

  • First, check that the error changes with the uniform patch length. If it does not, no placement can gain much (Thm 4.10).
  • Then put the short patches where the forecast depends on the input, which need not be where the input looks complex (Thm 4.4, Prop 4.5).
  • Finally, do not spread the patch lengths too far (Thm 4.7, 4.8).

Section 3 concerns the small gains of current methods. To show both sides, the revised version adds a controlled experiment in which we know how the forecast depends on the input; on benchmark data this is unknown. Each input window has 96 steps in three regions of 32 steps: white noise (the largest variance, independent of the future), oscillations that continue into the future, and a level with a slope. The series is linear and Gaussian, so the importance of each region for the forecast can be computed exactly: 78 % for the oscillations, 22 % for the level, 0 % for the noise. The most complex region has no weight in the forecast, so a selector that follows the complexity of the input and one that follows what the forecast needs must choose different regions. We use six settings (three noise levels, two horizons) and 10 seeds.

Thm 4.4 suggests that what any placement can gain is decided by the model’s error at each uniform patch length, before a selector is trained. If that error does not change with the patch length, both terms of the theorem are near zero, and no placement can gain, however well the selector chooses. Seen through the theorem, the small differences of Section 3 are expected if the published models have a flat error, and this is what we measure. On the synthetic series, the test error of the three published models at any uniform patch length is at most 0.4 / 1.0 / 1.7 % above their error at the best uniform patch length (EntroPE / HDMixer / TimeMosaic; maximum over patch lengths and settings). The theorem therefore predicts that placement can gain little in them, as Section 3 finds. Their error is flat because the values inside a patch remain available to the forecast (shown in Part 4). Placement can matter only in a model where patching loses detail, which we test next.

AuthorsOpenReview ↗

Part 2/5: W2 and Q1 (2/2), where placement matters and what a designer takes from it

To test where placement does matter, we replace every patch of the input by its (replicated) patch average before the forecaster reads it, so the detail lost grows with patch length. The forecaster is the same network in every comparison, an MLP-mixer backbone (the one of HDMixer, without any patch selector), trained from scratch for every placement. The baseline is uniform patching with 16-step, 8-step or 4-step patches. Each placement uses the same number of patches as its baseline and only changes their lengths. We fix three placements:

  • aligned: patches half as long as the uniform ones on the oscillations, and twice as long on the noise and on the level;
  • reversed: the same lengths, with the short patches on the noise;
  • random: the same lengths in shuffled order.

They use the same patch lengths, so they pay the same Jensen penalty and differ only in where the short patches are.

Thm 4.4 predicts the gain of each placement from the model's uniform forecasting error (its error at each uniform patch length) and the importance of each input step for the forecast, which is known here, before any model is trained with that placement. The predicted gain is the alignment gain minus the Jensen penalty. In this experiment, the results of the aligned placement, in % of the error of the uniform baseline (mean over the six settings, positive = better than uniform), are:

Uniform baselineAligned: oscillations / noise and levelAlignment gainJensen penaltyPredicted gainMeasured gain
16-step patches8-step / 32-step10.81.7+9.1+10.9
8-step patches4-step / 16-step13.77.5+6.2+4.1
4-step patches2-step / 8-step8.69.6−1.0−2.5

The aligned placement beats 16-step and 8-step uniform patches, as the theorem predicts. The last row is particularly informative. The short patches fall on the right region, yet the criterion predicts a loss against 4-step uniform patches, because the Jensen penalty exceeds the alignment gain; in the results the aligned placement is worse than 4-step uniform patches in all settings. The reversed placement has a negative alignment gain and loses against all three baselines (measured −9.8 / −17.7 / −16.1 %, predicted −9.8 / −17.7 / −16.0 %). The random placement, with an alignment gain near zero, loses 2.6 to 3.9 %. The predicted sign is the measured one for every placement, baseline and setting. A placement that puts short patches where the forecast depends on the input can therefore beat uniform patching (+10.9 % here), and Thm 4.4 gives the sign of its gain from the importance of each input step and the uniform errors.

The revised conclusion states the design insight as one measurement and two conditions on the selector.

  • First, measure the error at each uniform patch length (Thm 4.10), which tuning a uniform baseline already does. By Section 4 the gain of any placement over uniform patching is bounded by the difference between the uniform model's error at the same number of patches and at the best uniform patch length. On the eight benchmarks this bound is a median 0.25 / 1.08 / 0.38 % of the validation error (EntroPE / HDMixer / TimeMosaic), so placement has little to gain in the published models.
  • The short patches must be where the forecast depends on the input (Thm 4.4, Prop 4.5): the correlation between the importance of each input step for the forecast and the resolution a selector assigns, Corr⁡(K,r)\operatorname{Corr}(K, r), must be positive. On the synthetic series it is −0.41 for EntroPE, whose entropy rule puts 59 % of its patches on the noise, and −0.01 and −0.03 for HDMixer and TimeMosaic. With averaged patches, the forecaster's error with EntroPE's boundaries is on average 14.6 % higher than with uniform patches of the same number.
  • The patch lengths must not vary too much (Thm 4.7, Thm 4.8): the Jensen penalty, the cost of unequal patch lengths, must stay below the alignment gain. The aligned placement in the table's last row gives the right direction (Corr⁡(K,r)=+0.96\operatorname{Corr}(K, r) = +0.96), but the placement still loses 2.5 %.

A selector has to meet both conditions, which is the sense of "harder than it looks" in the title.

In the revised manuscript (new text in blue): new Section 5 and Figure 6 (the synthetic series and the placement tests); conclusion, Section 6; appendix, Tables 6 (the two terms, predicted and measured gain of every placement), 7 (this bound on the benchmarks) and 4 (Corr⁡(K,r)\operatorname{Corr}(K, r) for each method).

AuthorsOpenReview ↗

Part 3/5: Q2 (TimeMosaic) and Q3 (the "decreasing loss" of Sec. 4.3)

Q2 (TimeMosaic)

The difference is small and not statistically significant. TimeMosaic's adaptive model gives a lower MSE in the majority of settings, but the mean difference is only 0.10 % and the median 0.16 % (95 % interval −0.66 to +0.03 %, signed-rank p-value 0.250). Against the uniform model with the closest number of patches, it is ahead in 17 of 32 settings and 4 of 8 datasets (mean difference 0.09 %, p-value 1.000).

Your remark on the overhead points to the reason. TimeMosaic's selector is bound by design: it restricts each 32-step input region to patch lengths of 8, 16 or 32. It is a two-layer MLP with ~2,300 parameters in a model of several million. If it chose the same length everywhere, the adaptive model would be exactly the uniform model of that length. The adaptive model therefore stays close to a uniform one, in cost (1.2 % of training time and 2.9 % of inference time, Figure 4) and in accuracy.

We also measured where its patches go on the synthetic series of Part 1, where we know which region the forecast needs (validation-selected adaptive model, six settings, 10 seeds):

RegionImportance for the forecastShare of TimeMosaic's patches
Noise0 %33.7 %
Oscillations78 %32.8 %
Level22 %33.5 %

Each region gets about a third of the patches, which is what uniform patching gives, although the forecast depends on the oscillations and not on the noise. The reason is in TimeMosaic's design. It turns every patch, whatever its length, into a vector large enough to keep all the values of the patch, so a region read as one 32-step patch gives the forecaster as much as the same region read as four 8-step patches. The router is trained to lower the forecast error, and when the long patches lose nothing, it has little to learn about where to put the short ones. On the same series, a forecaster whose long patches do lose detail depends strongly on where the short patches are (Part 2).

In the revised manuscript (new text in blue): Appendix D.1, Tables 15 and 16 (budget-matched comparison per method and per setting).

Comments on the paper

3 comments
AuthorsOpenReview ↗

A thought experiment for Section 4

Several of your questions are about what Section 4 says, so we invite you to a short thought experiment. The replies refer back to it.

A friend must guess how a 96-page story continues without reading it. You may give them 12 index cards, one sentence each, every card summing up a run of consecutive pages. The story has three chapters of 32 pages: a side story about a minor character who never appears again, written in the richest language of the book; the conversation in which the plan is made; and a chapter saying where everybody ends up. The guess depends mostly on the conversation, somewhat on the last chapter, and not at all on the side story.

Even cards. Four cards per chapter, one every 8 pages. This is uniform patching, and the 12 cards are the fixed budget of Section 4.

Moving cards to where the guess needs them. Take two cards from the side story and give them to the conversation. The friend learns more about the plan and loses nothing that matters, so the guess improves (Prop 4.6). Every such move has a price, the detail lost by making the cards unequal (the Jensen penalty), and a gain, the detail placed where the guess needs it (the alignment gain). Uneven cards beat even cards exactly when the gain exceeds the price (Thm 4.4).

Moving too many. Give ten cards to the conversation and one to each other chapter. The friend knows the plan almost word for word but has one sentence for where everybody ends up, which the guess also needs. The guess is worse than with even cards, although the cards went to the right chapter (Thm 4.7, 4.8).

Moving cards to the richest writing. Someone who chooses by how rich the writing looks makes a choice unrelated to the guess, and such a choice cannot beat even cards (Prop 4.5). Here it is worse still, because the richest writing is the side story. A selector that follows the complexity of the input does this.

When nothing is lost. If the friend keeps the book next to the cards, no arrangement changes what they know, and even cards are as good as any (Thm 4.10). This can be checked with even cards alone: if the guess is about as good with cards of 4, 8 or 16 pages, no arrangement can do much better.

The three published models are in this last situation: the values inside a long patch stay available to the forecast, as if the friend kept the book, so adaptive and uniform models are almost equal, as Section 3 finds. The new Section 5 builds a series with these three chapters and takes the book away, so that the other situations appear and can be measured.

In the notation of Section 4, a page is an input step tt, a card is a patch, how much the guess depends on a page is its weight KtK_t, and the detail lost on a page covered by a card of 1/r1/r pages is D(r)D(r).

AuthorsOpenReview ↗

Invitation to Continue the Discussion

Thank you again to all reviewers for your time and feedback. We have posted our point-by-point responses and updated the manuscript with additional experiments and clarifications. We would welcome your thoughts on these updates and would be happy to discuss any remaining questions

AuthorsOpenReview ↗

Summary of the discussion and the revision

Dear Reviewers, ACs, SACs and PCs,

As the discussion period comes to an end, we thank the reviewers, ACs, SACs and PCs for their time and effort, and for a constructive discussion of our submission.

We are encouraged that the reviewers found that the central question is well posed and that the paper explains a negative result (Ve99), the theoretical results reasonable (Ve99) and the resolution-density framework a unified perspective on patch allocation (CBuC). We also appreciate the recognition of the isolation of the exact routing overhead and of the formal check of the proofs (wRcN), as well as the valuable insights the comparison provides into the trade-off between accuracy and efficiency (CBuC).

During the rebuttal, we added experiments to show why the differences of Section 3 are small and when patch placement does matter. We built a synthetic series on which the importance of every input step for the forecast is known exactly, and measured the two terms of Thm 4.4 for the three published selectors: all three sit at the origin of its plane, where placement has almost no effect (Ve99 W2 and Q1; wRcN W2 and Q2; CBuC W1 and W2). On the same series, we showed that when the network sees a compressed version of the patches, so that placement matters, Thm 4.4 predicts the sign of the measured gain (Ve99 W2, Q1 and Q4; wRcN W2 and Q2; CBuC W1). On real data, we modified EntroPE so that its error depends on the patch length, showing that the placement of the patches then affects the forecast (Ve99 Q4; wRcN W2 and Q2; CBuC W1), and we swapped the selectors of EntroPE and TimeMosaic between their backbones, showing that on these backbones the selector that provides the patches does not change the outcome (Ve99 Q4). To check that the conclusion of Section 3 holds at a matched number of patches and beyond the three methods of the paper, we added a budget-matched comparison with the uniform model, with the parameter ratio at the matched patch length (wRcN W1 and Q1), and a fourth adaptive method, Pathformer (CBuC W3). We extended the theory to selectors that estimate the target allocation with error (wRcN Q3). For TimeMosaic, we showed that its small advantage is not significant and explained why it stays close to its uniform model, in accuracy and in cost (Ve99 Q2). We revised the conclusion to state what a designer takes from the theory: one check on the uniform patch-length sweep and two conditions on the selector (Ve99 W2 and Q1; CBuC W2). To make the paper easier to read, we explained the decreasing loss of Section 4.3 (Ve99 Q3) and revised the notation, the presentation of Section 4 and Figure 1 (Ve99 W1; CBuC W2).

During the discussion period, Reviewer wRcN stated that their major concerns were removed and kept their weak accept rating. Reviewer CBuC found the empirical study thorough, wrote that the token-matched comparisons strengthened their confidence in its findings and that the theoretical analysis offers a useful explanation under the shared-loss assumption, noted the improved presentation, and raised their soundness rating by one point to reflect the stronger empirical evidence. We promptly answered their remaining comments on the significance of the work and its comparison with DLinear.

We respectfully ask that the submission be evaluated in light of the new experiments, the new Section 5 and the clarified theory.

Sincerely, Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.