- Rating
- 2
- Confidence
- 2(−1)
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 3(+1)
Read review
Paper summary
This paper compares the effects of adaptive blocking and optimized uniform blocking when applied to TSF. The author designed comparative experiments on eight benchmarks using three adaptive blocking methods. The prediction accuracy of both is close, while uniform blocking typically reduces computational costs. The paper further proposes a framework for budget allocation, derives the conditions under which non-uniform allocation is superior to uniform allocation, and characterizes the respective roles of region weighting, allocation alignment, and resolution discretization.
Critical strengths
-
This paper challenges the theory of the superiority of adaptive blocking through comparative experiments involving two blocking paradigms, providing valuable insights into understanding the trade-off between accuracy and efficiency.
-
The proposed resolution-density framework offers a unified perspective for block allocation and establishes theoretical analysis for the effectiveness of non-uniform allocation.
Critical weaknesses
-
The theoretical analysis is not fully substantiated by experiments. It assumes that the objective function is separable and employs a common resolution loss function and regional weights. However, these variables have not been estimated or validated on actual predictive models.
-
The font size in Figure 1 is too small, and the experimental analysis merely repeats percentage figures without in-depth exploration. Chapter 4 is difficult to understand.
-
The paper only evaluates three methods, which limits the universality of the conclusions to different adaptive blocking methods.
Questions for authors
See weakness.
7 comments
Part 2/5: W1 (2/3), the two terms of the model: the weights of the series and the loss of the forecaster
We read the second sentence of W1 as asking for the values of the two terms of the model, the weights and the shared loss, and for a check of the model on trained forecasters. Please correct us if we have misread it. The two terms are of different kinds. The weights belong to the series: they say how much the forecast depends on each input step. The shared loss belongs to the forecaster: it says how its error changes with the patch length.
The weights. On benchmark data they are unknown, so we designed a series on which they are exact. Each input window has 96 steps in three regions of 32 steps:
- white noise that is independent of the future, with the largest variance of the three regions;
- oscillations that continue into the future;
- a level with a slope, on which the future also depends.
The series is linear and Gaussian, so the weights are exact: the oscillations carry 78 % of the forecast's weight, the level 22 % and the noise 0 %.
The shared loss. We train EntroPE, HDMixer and TimeMosaic on this series with unchanged code and the candidate grids of Section 3 (six settings: three noise levels, two horizons; 10 seeds). In the model, the error of uniform patching is times the shared loss at that patch length, so we measure the test error at each uniform patch length. At any uniform patch length it is at most 0.4 / 1.0 / 1.7 % above the error at the best one (EntroPE / HDMixer / TimeMosaic). The reason is in the backbones: the values inside a patch remain available to the forecast.
- EntroPE: the forecast head receives one state for every input step, whatever the patch length.
- TimeMosaic: each patch becomes a token with more features than the patch has values, and the encoder also gets extra tokens that see the whole input.
- HDMixer (uniform model): a patch becomes a token of 16 numbers, which can hold every value up to 16-step patches.
On the benchmarks the error also changes little with the patch length: the difference between the validation error of the uniform model at the adaptive model's number of patches and its error at the best uniform patch length is a median 0.25 / 1.08 / 0.38 %. By Thm 4.10 this difference bounds the gain of any placement, for any weights.
What the two predict. The weights are far from equal, so a placement that follows them can gain, but by Thm 4.4 only as much as the loss changes with the patch length. For the published forecasters the loss is flat, so the model predicts that placement has no effect, in either direction. That is what we measure. EntroPE's selector puts its short patches on the noise, and its error is that of uniform patching. And when we impose on published EntroPE placements that follow the weights, oppose them or ignore them, its error stays within 0.4 % of uniform patching in every case. To see a placement gain or lose, a patch has to lose information (Part 3).
In the revised manuscript (new text in blue): main text, new Section 5; appendix, Table 3 (the three regions and their exact weights) and Table 7 (the bound on the benchmarks, per method).
Part 3/5: W1 (3/3), the model checked where a patch loses information
Placement can matter only if a patch loses information, and in the published models it does not: the values inside a patch remain available to the forecast (Part 2). We make a patch lose information in two ways.
Patch averages. We replace every input patch by its average before it enters the forecaster. The longer the patch, the more detail is averaged away, so it now matters which region gets the short patches. The forecaster is the same network in every comparison, an MLP-mixer backbone (the one of HDMixer, without any patch selector). We impose the patching ourselves and train the network from scratch for each placement, so the placement is the only thing that changes. The baseline is uniform patching with 16-step, 8-step or 4-step patches. Each placement uses the same number of patches as its baseline and only changes their lengths. We fix three placements:
- aligned: patches half as long as the uniform ones on the oscillations, and twice as long on the noise and on the level;
- reversed: the same lengths, with the short patches on the noise;
- random: the same lengths in shuffled order.
They use the same patch lengths, so they have the same Jensen penalty and differ only in the alignment gain.
Thm 4.4 predicts the gain of each placement before it is trained, from the exact weights and this forecaster's error at each uniform patch length; nothing is fitted to the placements. The predicted gain is the alignment gain (the benefit of putting the short patches on the important steps) minus the Jensen penalty (the cost of unequal patch lengths at the same number of patches). For the aligned placement, in % of the error of the uniform baseline (mean over the six settings, positive = better than uniform):
| Uniform baseline | Aligned: oscillations / noise and level | Alignment gain | Jensen penalty | Predicted gain | Measured gain |
|---|---|---|---|---|---|
| 16-step patches | 8-step / 32-step | 10.8 | 1.7 | +9.1 | +10.9 |
| 8-step patches | 4-step / 16-step | 13.7 | 7.5 | +6.2 | +4.1 |
| 4-step patches | 2-step / 8-step | 8.6 | 9.6 | −1.0 | −2.5 |
The aligned placement beats 16-step and 8-step uniform patches, as predicted. In the last row the short patches are still on the right region, yet the criterion predicts a loss against 4-step uniform patches, because the Jensen penalty now exceeds the alignment gain, and the loss is measured in each of the six settings (the case of Thm 4.7). In all 18 cases (3 baselines × 6 settings) the predicted sign is the measured one, and the two gains differ by 1.7 points in median.
The reversed placement has a negative alignment gain and loses against all three baselines, as predicted: −9.8 / −17.7 / −16.0 % predicted, −9.8 / −17.7 / −16.1 % measured. The random placement has an alignment gain near zero, and the criterion correctly predicts that it loses in every setting: −2.6 / −8.0 / −9.7 % predicted, −2.6 / −3.9 / −3.9 % measured. We also imposed the boundaries of EntroPE's entropy model, which put the short patches on the noise: the predicted gain is −15.5 % and the measured one −14.6 %. With the backbones of EntroPE or TimeMosaic, or a linear map, in place of the MLP mixer, the order against 8-step uniform patches is the same in every setting: aligned gains, random loses, reversed loses most.
Closing a path of the published model. The second way leaves the input untouched. EntroPE's forecast head receives one state per input step, a path that brings the individual values back after patching. We closed it, so the input reaches the forecast only through the patches. On ETTh1 and ETTm1 (horizon 96, 5 seeds), the error then varies by 45 % and 9.0 % across uniform patch lengths 4 to 32, against 1.9 % and 1.0 % as published. EntroPE's selector, within the variance across seeds of uniform patching before (+1.2 %, −0.9 %), is then worse in every seed (−8.3 %, −3.1 %).
In the revised manuscript (new text in blue): main text, new Figure 6 (error against uniform patch length; the plane of Thm 4.4 with one dot per placement; predicted against measured gain); appendix, Tables 6 (both terms, predicted and measured gain of every placement), 5 (the other forecasters) and 8 (EntroPE with the step path open and closed).
Part 1/5: W1 (1/3), the assumptions of Section 4 and what depends on them
Before the individual answers, we invite you to read our general comment, "A thought experiment for Section 4", which explains the theory through an analogy.
Thank you for the review. In short: the separable shared-loss form is the model the paper states and studies, and this part says what it assumes and what it leaves out. Its two terms are of different kinds: the weights belong to the series and are exact on the series we built, and the loss is the error at each uniform patch length, which we measure on trained forecasters (Part 2). Its predictions are then checked on trained forecasters, where placement has no effect and where it does (Parts 2 and 3).
The first sentence of W1 gives the assumptions of Section 4 as the paper states them: the introduction says "As one instantiation, we analyze separable objectives in which regional weights Kt scale a shared loss response D(rt) to resolution", and the abstract places the results "within a separable shared-loss model".
Model perspective. Start from equal patches and, keeping their number, make them slightly finer on some steps and coarser on others. To first order, the change in error is then a sum of one term per step, for any forecaster whose error varies smoothly with resolution; Kt measures how strongly the error responds at step t, up to a common factor (Sec. 4.1). So at first order separability is not an assumption. What the model assumes comes after this first approximation: refining one region does not change what refining another is worth, and the error follows the same convex curve D of resolution in every region, scaled by that region's weight. Because the curve is convex, unequal patches cost something in themselves (the Jensen penalty), and a placement wins only if its fine patches are on steps that matter enough to outweigh this cost. That is what the exact criterion (Thm 4.4) says: putting the fine patches on the steps that matter is necessary but not sufficient. An analogous trade-off holds for any error that is convex in the resolutions, separable or not: the first-order gain from placement must exceed a nonnegative remainder that vanishes for equal patches. The model writes down the premise of adaptive patching itself, that some regions deserve more resolution than others, with Kt saying how much each region gains from it. Each method picks these regions by its own signal: EntroPE starts a new patch where its entropy model is uncertain about the next value, and TimeMosaic, in the words of its abstract, adjusts patch granularity "according to local information density".
Why one shared loss. A shared loss says that regions differ in how much resolution matters, not in which resolution is best. Section 4 studies this case on purpose. Appendix A.1 separates it from the other mechanisms by which unequal patches could help, such as a different loss curve in each region ("different time steps have different ℓt(r)"), and shows that with one shared loss and no budget the weights never favour unequal patches: the fixed budget is what makes them matter. The shared loss is also what lets the bound of Thm 4.10 be evaluated from an ordinary uniform baseline, as the next paragraph says.
What depends on the weights. Section 4 takes the weights as given: its results hold for any non-negative weights, and only Prop 4.6 needs them to differ. The result we use to interpret Section 3, Thm 4.10, needs no estimate of them: in the shared-loss model, if one resolution is best for every region, the gain of any placement is at most the uniform model's error at the same number of patches minus its error at the best uniform patch length. Sec. 4.6 notes that this bound "requires only a common global minimum and nonnegative weights". To evaluate it we need only the uniform model's validation error at each patch length of its tuning grid, which tuning the uniform baseline already gives (Appendix C.5, for placements whose patch lengths stay inside the grid). The criterion of Thm 4.4 also holds for any weights, and so does Prop 4.5: a placement that is statistically independent of the weights cannot beat uniform patching. When finer patches lower the loss (Sec. 4.3 to 4.5), how much the weights differ enters the size of the possible gain: the maximum of the local quadratic surrogate (Thm 4.7) and the upper bound of Thm 4.8 both grow with the variance of the weights (Part 5 states each result in plain words).
The flat curve outside our study. PatchTST reports that its "MSE scores don't vary significantly with different choices of P" (patch lengths 4 to 40, Appendix A.4.1), and TimeXer that "the average prediction performance does not vary dramatically with different patch lengths" (Appendix B.2). In the model, an error that hardly changes with the uniform patch length leaves little room for any placement (Thm 4.10). Section 3 finds small differences, in both directions.