acceptodds
Under review as a conference paper at ICLR 2027

Doing More with Less: Dense Tokenization for Recurrent Monthly Forecasting

Abstract

Many forecasting tasks rely on monthly observations, either because finer observations do not exist or because figures are reconciled only at the end of each period. We ask whether changing how these observations are represented can improve a model's accuracy. We propose flat monthly tokenization, which repeats each monthly average across several tokens that carry known calendar features. This representation enables multiple recurrent updates per observation while preserving the information in the monthly history and calendar. On monthly tasks built from two public daily datasets, a small GRU with 90 tokens per month reduces CRPS relative to the monthly baseline by 20 to 49% on retail and 28 to 36% on electricity. On retail, it also achieves lower CRPS than N-HiTS and the same network trained on daily observations. Ablations indicate contributions from the input representation and repeated recurrent updates, with an additional contribution from varying calendar features on electricity. Decomposing the score into median and nonmedian components shows that increasing density from 30 to 90 tokens improves both components on electricity, whereas on retail the improvement is confined to the nonmedian component. These results identify token density as a useful modeling choice for monthly forecasting. On the two evaluated datasets, dense monthly tokenization recovers or exceeds the accuracy of the same network trained on daily observations, using only monthly observations and known calendar features.

Primary area:
learning on time series and dynamical systems
Keywords:
Monthly time-series forecasting, dense tokenization, recurrent neural networks, probabilistic forecasting, temporal aggregation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_CqgwOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies forecasting when only monthly aggregates are available, and asks whether changing how each observation is represented can improve accuracy without adding information. It proposes flat monthly tokenization: each monthly average is repeated across k tokens (k = 90 in the main experiments), and each token carries known calendar features (day of week, month of year), a mask and scaling statistics. A two-layer GRU with about 174k parameters reads this sequence and outputs seven monotone quantiles. A short proposition shows that the dense representation contains no information beyond the monthly history and the calendar, so any gain must come from how the model computes, not from new data.

The authors build monthly tasks from two public daily datasets: Online Retail (2071 products) and UCI electricity (336 clients). This lets them compare against the same network trained on the daily series. Compared with a one-token-per-month GRU, the 90-token model lowers scaled CRPS by 20-49% on retail and 28-36% on electricity at horizons 1-3. It matches the daily-input reference on electricity and beats it on retail. The paper compares against six models trained from scratch (including N-HiTS, PatchTST and an LSTM) and six zero-shot foundation models.

Ablations are used to attribute the gain to three sources: more recurrent updates per observation, calendar features that change across tokens, and the flat input's invariance to how values are distributed within a month. A gate analysis argues that the dense GRU makes many small state corrections rather than integrating slowly. Splitting the score into median and non-median parts shows where the probabilistic improvement occurs.

Critical strengths
  1. Clear question with a principled framing. The paper separates observation frequency (how many measurements exist) from token density (how many sequence positions represent each measurement). Proposition 1 says clearly that the dense representation adds no information. Any improvement therefore has to come from how the model learns and computes. This framing is useful and could apply beyond this paper.
  2. Simple, cheap method. Flat tokenization needs no extra parameters, no external data and no model that guesses daily values. The 90-token GRU trains in 2-4 minutes on a laptop GPU. That makes it easy to adopt in practice, especially when only aggregated data exist.
  3. Informative experimental design. The authors build monthly tasks from daily datasets, so the same network can be trained on the daily series. This lets them measure directly how much accuracy aggregation costs and how much the representation recovers. The comparison is rarely available and is a good idea.
  4. Careful statistics and transparent reporting. Every comparison is paired across models, and intervals come from bootstraps that resample series, with a two-way version that also resamples seeds. The selection rule was fixed before the density experiments. Screening uses 3 seeds and confirmation uses 8. The paper also reports results that work against it: validation and test disagree, the foundation models win on median error, the best GRU is close to the naive forecast, and some ablations were run after the test evaluation. The core contrast between the 90-token and monthly models stays significant on both datasets even when seeds are resampled too.
  5. Thorough analysis of where the gain comes from. The ablations separate input map, density, varying versus fixed calendar, weekday and month of year channels, dropout, and how many recurrent steps run per month. Together with the gate measurements and the median/non-median split of the score, they go well beyond reporting one leaderboard number. The finding that the dense GRU makes many small state corrections, rather than integrating slowly, is a nice mechanistic result.
  6. Reproducibility. Code, the scored results behind every table, and regeneration scripts are provided. Datasets are public, training is deterministic given the seed, and compute is fully reported.
Critical weaknesses
  1. Claims go beyond the evidence. Token density is presented as a general modeling choice, but the gain appears only for recurrent models. PatchTST, iTransformer and DLinear at 90 tokens do worse than the GRU (Table 4). The claims should be limited to recurrent forecasters.
  2. Weak baseline. The 20-49% gain is measured against a monthly GRU that does poorly on retail. Its CRPS at horizons 1-3 is 0.884 / 1.369 / 1.888, versus 0.804 / 0.974 / 1.178 for the monthly LSTM and 0.716 / 0.795 / 0.924 for naive. Going to 90 tokens improves the LSTM by only 2-5%. The 90-token GRU is about level with naive at horizon 1 and worse at horizons 2-3.
  3. Test-set selection. GRU (best, ours) in Tables 4 and 5 is the best configuration on the test set. On electricity it is the one cycle model, not the validation selected one. On retail it is the 4-layer model, trained after the test evaluation. Comparisons with baselines should use only models selected on validation.
  4. Small evaluation. There are two constructed datasets with 2-3 test origins each, and validation and test rankings reverse (Fig. 7, App. B). The motivating cases (GDP, inflation) are never tested. Truly monthly benchmarks (M4, Tourism, FRED-MD) with rolling origins are needed.
  5. Limited novelty, missing related work and baselines. On retail, identical tokens cost at most 2% (Table 12), so the retail gain is largely Repeat-RNN. The paper omits mixed-frequency state-space and nowcasting models, MIDAS, and ODE-RNNs. These run a recurrence on a finer grid than the observations. Missing baselines: those models, Repeat-RNN reported as a baseline, and alternatives with matched compute (the 90-token model uses about 80x the forward FLOPs).
  6. Post-test analysis and caveats left out of the main text. Most ablations supporting the three-source explanation (6.4) were run after the test evaluation. Neither the abstract nor the main text mentions the validation/test reversal, the near-naive retail accuracy, or the foundation models' lower median error.
Questions for authors
  1. Does the density effect appear in any model that isn't recurrent, such as a Transformer or a state-space model with dense tokens rather than patches? If not, would you limit the claims to recurrent forecasters?
  2. Why does the monthly GRU do so much worse than the monthly LSTM and the naive forecast on retail? How large is the 90-token gain over the strongest monthly baseline you have, including naive?
  3. Can you report Tables 4 and 5 using only the configurations selected on validation? Which conclusions in §5.4 and §7 still hold?
  4. Can you add results on truly monthly or quarterly benchmarks (e.g., M4 monthly, Tourism, FRED-MD) with rolling-origin evaluation? Does the method work at other aggregation ratios, such as quarterly to monthly?
  5. On validation, accuracy does not improve steadily with density, and the monthly model is best at retail horizon 1. What explains the reversal on test, and how sensitive are the results to the choice of test origins?
  6. How does flat tokenization differ from mixed-frequency state-space models that run a recurrence on a finer grid than the observations? Can you compare against such a model, and against Repeat-RNN as a standalone baseline?
  7. Does the 90-token model beat a monthly model given the same compute, through extra depth, internal micro-steps or an ODE-RNN?
  8. Do the ablations run after the test evaluation reproduce on the validation set, so that the three-source explanation does not rest on test-time analysis alone?
2 comments

Thank you for the careful review. Your questions led to further monthly baselines, a validation result for every ablation, and three native monthly benchmarks. On test, the 90-token GRU outperforms the monthly GRU on both datasets and all six trained baselines on retail, and the native sets mark where it stops helping.

  • Claims limited to recurrent models. We have added this limitation throughout, including the abstract and conclusion.
  • Weak monthly baseline. The retail result holds against stronger monthly recurrent models. The 90-token model has 10 to 34% lower retail test CRPS than the monthly GRU trained with the schedule of the 90-token model, 10 to 35% lower than a monthly GRU given weekday shares, and lower CRPS than the monthly LSTM (Table 6). Against the naive forecast, it is level at horizon 1 and 4% higher at horizons 2 and 3, which Section 5.4 now states. The monthly GRU does poorly on retail test because the months before the origin misled it there. Restricted to the origin month, it has 22 to 49% lower test CRPS on retail and 20 to 45% lower on electricity. On retail validation, the same restriction raises its CRPS by 8 to 21%, so validation would keep the full history. The 90-token model stays close to the better of the two without that choice (Section 6.2).
  • Test-set selection, and whether the ablations reproduce on validation. With the validation-selected model, the retail conclusions of Section 5.4 hold at every horizon, and at the original electricity test origins, only N-HiTS and the LSTM are significantly lower at horizon 3. Table 13 sets each ablation beside its validation result. On electricity the gains reproduce, as the 90-token model has 7 to 14% lower validation CRPS than the monthly GRU and 9 to 20% lower than a model with one token per month, both two-way significant. On retail, the gain over the 30-token model (6 to 26%) and the LSTM's gain from density (14 to 23%) reproduce with two-way significance, and the weekday effect (6 to 12%) reproduces in direction. The retail gains over the monthly GRU and the one-token model do not reproduce, with 3 to 11% and 4 to 13% higher validation CRPS, neither two-way significant. Section 6.2 explains this reversal, as the earlier months help the monthly GRU on the validation span and mislead it on test. Six retail comparisons reverse with two-way significance, among them the gain over the same GRU trained on the daily series (Appendix D). On electricity, the gain over the 30-token model, 8 to 21% on test, is at most 1.3% on validation.
  • Truly monthly benchmarks. We added FRED-MD, M3 monthly, and Tourism monthly, each truncated to one common cutoff, with a context of 24 months and three rolling test origins in place of the standard holdouts (Appendix P). On all three the earlier months help the monthly GRU, so we registered, before running the 90-token model, that it would have higher CRPS. On validation it does, by 3 to 19% in all nine combinations of benchmark and horizon. On test it is 12 to 15% higher on Tourism and 1.6 to 3.7% higher on FRED-MD, where no two-way interval excludes zero, and on M3 it is level at horizons 1 and 2 and 2.0% lower at horizon 3. It has a lower test CRPS than the best classical model, chosen on validation, in 6 of the 9 combinations. These benchmarks mark the boundary of the method. We did not test aggregation ratios other than daily to monthly (Appendix Q).
  • Related work, Repeat-RNN and matched compute. Section 2 now covers MIDAS, mixed-frequency state-space models, nowcasting, and ODE-RNNs. MIDAS and mixed-frequency state-space models forecast from an indicator observed at a higher frequency, and our setting has no such indicator, only the calendar. Repeat-RNN, which gives the monthly GRU internal micro-steps, is now a standalone baseline. Without its first-repeat flag, it matches the 90-token model on test, with no two-way significant difference, so the gain comes from the repeated updates. Added width does not reproduce it, as the 90-token model has 17 to 45% lower test CRPS than a wider monthly GRU matched in forward FLOPs. A third or fourth layer raises the monthly GRU's electricity validation CRPS by 10 to 36% at horizons 1 and 2 (Appendix K). We did not run an ODE-RNN or a state-space model of a latent daily process (Appendix Q).
  • Caveats in the main text. Section 5.4 now states the naive and foundation-model results, and Section 6.2 the retail reversal on validation. The 90-token model has lower CRPS than the monthly GRU at all 4 retail rolling origins and at 9 to 11 of the 13 electricity origins (Table 23), and Figure 5 shows where it loses.

Dear Authors, thank you for the detailed response! LImiting the claims to recurrent models, adding Repeat-RNN and a wider GRU, adding validation results for each ablation, registering predictions before running the native benchmarks - all these make the paper clearer and more honest. However, the new results confirm my main concerns. On retail dataset, the 90-token model is worse than monthly GRU and one-token model. Several comparisons reverse with two-way significance (including the gain over the daily input model). On FRED-MD, M3 and Tourism, it is worse than monthly GRU on validation in every cell and on test for two of the three sets. Repeat-RNN without its flag matches the 90-token model. So the gain seems to come mainly from plain repetition and shorter memory, not from the dense calendar tokens. It helps when the earlier months mislead the model (which is hard to know in advance), and it hurts on real monthly data where history is informative. Retail accuracy is also still on par or behind the naive forecast and zero-shot foundation models. I think the paper is now a careful analysis of when repetition helps a recurrent forecaster but its contribution is narrower than the paper claims, so I will keep my score.

Reviewer_YxDDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies whether monthly forecasting can benefit from representing each observation with multiple tokens. It repeats each monthly average on a flat calendar grid and feeds the resulting value, mask, calendar, and scale channels to a recurrent quantile forecaster. Repetition increases recurrent computation without supplying additional measured observations. Proposition 1 formalizes the corresponding information limit. On monthly tasks constructed from Online Retail II and UCI electricity, the selected two-layer 90-token GRU reduces the scaled seven-quantile score, called CRPS in the paper, by approximately 20–49% and 28–36%, respectively, against the monthly GRU at horizons 1–3. The paper compares daily-input references and other forecasters, and studies input maps, optimization schedules, calendar features, dropout, depth, and recurrent gates. Its strongest contribution is an empirical demonstration that repeated processing can improve a small recurrent forecaster on these tasks.

Critical strengths
  1. A simple intervention with substantial observed gains. The method is straightforward and useful for settings where only monthly measurements are available. The principal 90-token versus monthly comparison shows large improvements on both datasets. Appendix L reports that these gains at horizons 1–3 remain significant when both seeds and entities are resampled, so the central observation is not supported solely by entity-only intervals. The LSTM results provide some evidence beyond one recurrent cell (Table 6 and Appendix D).
  2. Careful separation of additional computation from additional observations. The information-limit proposition and value-invariance discussion correctly avoid claiming recovery of unobserved daily information. The appendix contains substantial controls, including the one-token dense input map, schedule-matched comparisons, identical-token calendars, and dropout removal. The authors also disclose negative findings and important limitations, including stronger monthly foundation-model forecasts, weaker long-horizon results, and seed sensitivity. These details make the empirical finding informative even without a state-of-the-art claim.
Critical weaknesses
  1. Test-selected comparisons obscure the performance of a fixed model. Tables 4–5 use the lowest test value per row across Table 6 configurations, rather than one model selected via val-set. Electricity CRPS values 0.664/0.866/0.782 come from the one-cycle ablation; the selected WSD model scores 0.736/0.910/0.863. Retail comparisons also mix configurations (pp. 16–17). This does not invalidate Table 3’s fixed-model improvements, but weakens the practical comparisons. Please report Tables 4–5 for the configuration frozen before testing, including comparisons with N-HiTS, the classical baseline, and foundation models.
  2. Exploratory analyses are not clearly separated from confirmatory evaluation. Appendices D and J place further schedule/input ablations and depth studies after initial testing (pp. 22 and 26). Appendix C retains WSD despite failing the stated selection criterion (p. 21). Later validation checks do not turn these analyses into an untouched confirmatory evaluation. Please identify the configuration fixed before the first test evaluation, explain the departure from Equation 11, and distinguish subsequent exploratory variants from the original evaluation.
  3. Few test origins limit evidence of temporal robustness. Each dataset has one training cutoff, with only two retail and three electricity test origins (Table 2). Entity resampling cannot establish robustness across calendar periods. Please report per-origin gains for the frozen model and two-way seed-and-entity intervals for the daily-input and 90-versus-30 comparisons. Appendix L already covers the main contrast against the monthly model; the remaining uncertainty concerns the additional comparisons used to interpret the gains.
  4. Calendar-resolved tokenization is not isolated from repeated computation. Repeat-RNN [1] already studies repeated recurrent updates. Under matched WSD training, 90 versus 30 tokens improves retail CRPS by only 0.7–3.4%, falling to at most 1.1% without dropout. Identical mid-month calendars cost only 0.2–1.9%; electricity’s calendar effect reverses at horizons 5–6 (Table 12). Existing middle-day controls also change the month’s representation. Weekday composition is standard [2] and motivated by Equation 10. Please compare with a monthly model using weekday counts or proportions and a fixed-repeat model receiving the same calendar summary, with matched optimizers and update budgets. Existing results or a compact validation-only comparison would help establish whether calendar order adds value.
  5. Endpoint measurements do not establish the proposed memory mechanism. Small gate products constrain retention through the direct gate path, but small month-end displacement does not imply that intermediate candidates remain near the current state: large movements can cancel. Table 13 measures neither these movements nor history sensitivity. Please report within-month state increments and direct sensitivity to earlier months, or qualify the candidate-map memory explanation if these measurements are unavailable. The repeated-computation finding remains credible without this stronger mechanistic interpretation.
Questions for authors

See Critical Weaknesses above.

1 comment

Thank you for the detailed review and the effort behind it. Your questions on selection and evidence changed how the revision reports results. It now uses one validation-selected model in the baseline comparisons, dates every selection decision, adds two-way intervals for the remaining comparisons and measures the recurrent state directly. At the same recurrent width and depth, the 90-token model lowers test CRPS against the monthly GRU by 20 to 49% on retail and 28 to 36% on electricity.

  • Test-selected column. It has been removed. Tables 4, 5 and 9 and Figures 4 and 7 now report one model, the 2-layer 90-token GRU selected on validation. The one-token comparison model of Table 12 now uses the learning rate chosen on validation, and the 90-token model has 4 to 25% lower retail test CRPS than it.
  • Confirmatory and exploratory work, and WSD. Appendix R dates each decision. The 2-layer 90-token GRU and its WSD schedule were selected on validation before its test evaluation. The monthly, 30-token and daily-input models had been evaluated on test before the first 90-token run. After the first of these evaluations we clipped the forecast range, and after the 90-token test evaluation we kept the default model over a variant that skips padding tokens. The WSD schedule was chosen on the 30-token model. It fails Equation 11 at one pair, electricity horizon 4, where it has 1.8% [+0.1, +4.4] higher CRPS than the one-cycle schedule, and we kept it for its larger mean validation improvement (Section 4.5, Appendix C). The main result does not depend on this choice, as with both models on the one-cycle schedule the 90-token model lowers test CRPS against the monthly GRU by 15 to 47% on retail and 32 to 42% on electricity. The controls and the second cutoff were registered before they ran, and the native benchmarks before the 90-token model ran on them. The rolling origins, the sensitivity measurements and the history-value comparison are labeled exploratory.
  • Per-origin results and two-way intervals. Table 20 gives two-way intervals for the daily-input and 90-versus-30 comparisons. Against the same GRU trained on the daily series, under either schedule, the 90-token model is two-way significantly lower at every retail horizon, by 8 to 13%, and on electricity no two-way interval excludes zero at horizons 1 to 3. Against the 30-token model the gain is two-way significant at retail horizon 3 and electricity horizons 2 and 3, and not at retail horizons 1 and 2 or electricity horizon 1. The contrast with the monthly GRU is two-way significant at every horizon on both datasets, and every 90-token training run has lower test CRPS than every monthly run in five of the six dataset-horizon pairs (Appendix M). Table 23 gives per-origin results at 17 rolling origins. The 90-token model has lower CRPS than the monthly GRU at all 4 retail origins and at 9 to 11 of the 13 electricity origins, and pooled the gain is two-way significant at every retail horizon and at electricity horizon 3 (Table 25).
  • Weekday-count baseline, and a repeated input with a calendar summary. We ran both. The controls do not establish that calendar order adds value, and the paper no longer claims it (Section 6.5). The 90-token model has 10 to 35% lower retail test CRPS than a monthly GRU given each month's seven weekday shares, and 34 to 40% lower on electricity. Two results go against our registered predictions. On retail validation the weekday-share models have 6 to 10% lower CRPS than the 90-token model at horizons 2 and 3, and adding the shares to Repeat-RNN raises its electricity CRPS by 1 to 5%. The repeated models share one schedule and learning rate, and the monthly models keep the learning rate selected for them on validation.
  • Mechanism beyond endpoints. We now report both measurements (Appendix F). Within a month the second-layer state of the 90-token model travels 1.6 to 6.6 times its norm in small steps and ends 0.21 to 0.56 of its norm from where it started. A perturbation of the state at the start of a month decays to 7 to 8% after one month, against 80 to 89% for the monthly GRU. With the level and spread of the context held constant, the months before the origin carry 3% of the 90-token model's horizon-1 sensitivity, against 74 to 75% for the monthly GRU. With the level and spread recomputed from the perturbed history the shares are 36 to 56% and 62 to 66%, so the earlier months reach the 90-token forecast mainly through these two statistics. This measured account replaces the candidate-map explanation. Repetition also helps when no earlier month is read, as with both models reading only the origin month 90 tokens have 5 to 23% lower electricity test CRPS and 7 to 21% lower retail validation CRPS than one token (Section 6.2).
Reviewer_vh8COpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies if repeating monthly observations can improve forecasting. It repeats each monthly average across multiple tokens, adds calendar features, and uses a recurrent network to predict future monthly values. Experiments on retail and electricity data compare token densities, forecasting models, and several design choices.

Critical strengths
  1. The problem is meaningful when only monthly observations are available. The method is simple to implement and shows substantial improvements over the monthly recurrent baseline on both datasets.

  2. The experimental analysis is detailed. The paper examines input representation, calendar features, training schedules, repeated updates, and computational costs. It also reports multiple seeds and checks whether the main improvement remains when both training seeds and entities are resampled.

Critical weaknesses
  1. The technical innovation is limited. This is becuase the paper already discusses Repeat-RNN and other methods that apply several recurrent updates to the same observation. Here, the main change is to repeat monthly values and attach calendar features. What new capability does this provide beyond repeated-input recurrence with calendar information? Proposition 1 establishes that a deterministic transformation adds no information, but it does not explain when repetition should improve forecasting or how to select the token density. The paper needs a clearer contribution beyond applying this existing idea to monthly forecasting.

  2. The numerical results do not yet establish how broadly the method works. This is because the two datasets contain only 24 and 48 months, with two retail test origins and three electricity test origins. Many entities provide useful cross-sectional evidence, but they do not establish performance across different time periods. The larger gains are also mainly against the monthly GRU. Appendix A reports that the best GRU is close to the classical retail baseline at longer horizons, and N-HiTS performs better at longer electricity horizons. Moreover, Table 18 shows substantially higher computation than the monthly GRU despite similar parameter counts. Additional monthly datasets, more evaluation periods, and comparisons at similar computational cost are needed.

Questions for authors
  1. What is the main technical difference from Repeat-RNN with the same calendar features? Does including its new-token flag change the results?

  2. Do the gains remain on longer monthly datasets and across more forecast origins, particularly against strong classical and N-HiTS baselines?

  3. At a similar computational budget, does repeating observations outperform giving a monthly model more computation through other designs?

1 comment

Thank you for the careful review and the time it took. Your questions on novelty, breadth and matched compute shaped much of the revision, which now runs Repeat-RNN as a control, adds rolling test origins, a second training cutoff and three native monthly benchmarks, and compares the 90-token model with a wider monthly GRU matched in forward FLOPs. At the same recurrent width and depth, the 90-token model lowers test CRPS against the monthly GRU by 20 to 49% on retail and 28 to 36% on electricity.

  • Novelty relative to Repeat-RNN, and the flag. We build on Repeat-RNN and now credit it for repeated updates per input (Sections 1 and 2). Fojo et al. tested up to 12 repeats on parity and addition and left open why repetition helps. Our contribution is the study of this mechanism in monthly forecasting, with three results. A model reading only monthly observations recovers or exceeds the test accuracy of the same network trained on the daily series. Repetition gives it a short effective memory, as 7 to 8% of a state perturbation remains after one month, against 80 to 89% for the monthly GRU. The gain appears where the months before the forecast origin mislead the monthly GRU, a condition we tested with a registered prediction on native monthly benchmarks. The technical difference from Repeat-RNN is in the inputs, as every token carries calendar features and no flag marks the first repeat. Run with 90 repeats and no flag, Repeat-RNN matches our model on test, with CRPS between 1.9% lower and 6.0% higher and no two-way significant difference (Appendix N), so the gain comes from repetition under either input design. The flag does not help consistently. On Repeat-RNN it raises electricity test CRPS by 4 to 8% and changes retail test CRPS by less than 2%. Added to a 90-token model whose tokens all carry the same calendar, it raises CRPS by 2 to 8% on test and by up to 6.5% on validation.
  • When repetition helps, and the choice of k. Repetition helps where the months before the origin mislead the monthly GRU and mostly hurts where they help it (Section 6.2). Proposition 1 only bounds the information available, so this condition is empirical. It holds in 73 of 81 combinations of dataset, origin and horizon, and in the one prospective test, on three native monthly benchmarks, the outcome we registered holds in all 9 validation combinations. We choose k on validation. There 90 tokens were retained over 30, and neither 60 nor 150 met the selection criterion against 90 (Appendix C). Under the second electricity cutoff the 90-token model has 2.9 and 11.1% higher test CRPS than the 30-token model at horizons 2 and 3, and validation there also favors 30 tokens (Table 24).
  • Breadth. Over 17 rolling test origins the 90-token model has lower CRPS than the monthly GRU at all 4 retail origins and at 9 to 11 of the 13 electricity origins. Pooled, the gain is 25 to 51% on retail and 13 to 17% on electricity. Against N-HiTS it is 40 to 41% lower on retail and 13 to 16% higher on electricity (Table 25). With every model retrained under a second electricity cutoff, the gain over the monthly GRU is 21, 4 and 8% at horizons 1 to 3. On three native monthly benchmarks the monthly GRU has lower validation CRPS, as we registered, and on test the 90-token model has lower CRPS than the best classical model, chosen on validation, in 6 of the 9 combinations of benchmark and horizon (Appendices O and P).
  • Gains mainly over the monthly GRU. On retail test the 90-token model also outperforms the LSTM, sLSTM, PatchTST, iTransformer, DLinear and N-HiTS at every horizon (Table 4). The same GRU trained on the daily series has 12 to 15% higher retail test CRPS, two-way significant at every horizon (Tables 8 and 20). On retail validation the order reverses, and the 90-token model has 6 to 15% higher CRPS than the daily-series GRU trained with the same schedule (Table 13). Under the second electricity cutoff it has 14 to 36% lower CRPS than the daily-series GRU at every horizon (Table 24).
  • Matched compute. The 90-token model has 17 to 45% lower test CRPS than a monthly GRU widened to 15M parameters to match it in forward FLOPs (Appendix N). Width does not shorten the memory. One month after a perturbation of its state the wide GRU keeps 94.5% of it on retail and 99.8% on electricity, against 3.0% and 7.8% for Repeat-RNN without its flag (Appendix F). The gain and the short memory therefore go with repetition and not with added width. Of the models that add computation per observation we ran Repeat-RNN and this wider GRU, and Appendix Q lists the others.

Comments on the paper

1 comment
Area_Chair_nuPrOpenReview ↗

Reminder: Reviewer–Author Discussion

Thank you to the reviewer who have already engaged in the discussion and provided valuable feedback. As the discussion deadline is approaching, I would kindly encourage the remaining reviewers to review the authors' responses and share their follow-up comments, particularly on whether their concerns have been adequately addressed.

Authors are also encouraged to respond promptly to any further questions.

Thank you all for your time and contributions to a constructive discussion.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.