- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
The paper introduces ProST, which progressively refines predictions of the same future in a joint temporal–spatial forecast space. It decomposes each correction into a weighted coarse-scale update and zero-aggregate detail, executed by a shared scale-conditioned forecaster. A history-conditioned selector chooses a complete refinement sequence before forecasting begins. The paper establishes a cross-scale aggregation identity and evaluates the method on three traffic datasets and one air-quality dataset. Fixed-sequence ProST achieves the lowest mean MAE in seven of eight settings. Training across sequences improves average and worst-sequence errors, while frozen-weight interventions support complementary roles for temporal coarse-scale updates and detail. Adaptive selection provides accuracy–computation trade-offs, but does not consistently outperform input-independent random selection.
Critical strengths
-
ProST explicitly separates coarse-scale updates from zero-aggregate detail and proves that aggregating a refined forecast yields the preceding forecast plus the weighted coarse-scale update. This makes the effects of the correction components precisely describable. Both refinement steps have positive mean gains on all four datasets, and removing either the temporal coarse-scale update or detail increases terminal MAE in every evaluated seed. This is a principal technical argument for acceptance: the mechanism has both an algebraic definition and functional evidence within the trained model, although the direct experimental evidence should be restricted to the temporal decomposition.
-
Fixed-sequence ProST achieves the lowest mean MAE in seven of eight settings, including a reported 7.73% improvement over STORM on KnowAir at . Moreover, when evaluated on identical test samples and the same 22 sequences, training across sequences reduces both average and worst-sequence errors on all four datasets without changing per-sequence FLOPs. These results provide the strongest empirical support for acceptance: a shared forecaster can support multiple execution sequences and improve its performance across them through training. This conclusion should nevertheless be interpreted alongside the reported seed variability.
Critical weaknesses
-
The joint temporal–spatial mechanism is not fully tested.
The method uses joint temporal–spatial decomposition, but the interventions and numerical checks only test temporal decomposition. This leaves the joint mechanism insufficiently validated. Add spatial-only and joint interventions, and check the joint aggregation identity in the actual implementation. -
Cross-scale consistency does not guarantee better predictions.
Proposition 4.4 proves an aggregation identity, but does not guarantee lower prediction error. The paper also shows that larger local gains can lead to worse final predictions. Clearly state the theorem’s scope and report how coarse-scale error, local gains, and final error change together. -
Frozen-weight interventions do not establish the benefit of the design after training.
Removing components from a trained model shows that the model depends on them. It does not show that alternative models would perform worse after retraining. No direct evidence found in the manuscript for retrained comparisons. Retrain models without the coarse update, detail, or damping, and include a standard additive correction model using the same backbone, path, and training budget. -
The benefit of training across paths is mixed with extra training.
Training across paths adds 100 epochs, but there is no control that continues fixed-path training for the same number of epochs. The improvements also vary across seeds. No direct evidence found in the manuscript for an equal-budget control. Add continued fixed-path training with the same optimizer and training budget, and report results for each seed. -
Adaptive selection does not consistently beat simple selection.
Its gains over random selection are very small on PEMS04, and it performs worse on KnowAir. The comparison at matched cost gives positive gains in only two of three seeds. Compare against fixed and random strategies at the same average cost, using more seeds and paired confidence intervals. -
The reported cost does not cover the full system.
FLOPs exclude feature computation and selector execution. End-to-end latency, memory use, and throughput are not reported, so actual inference speedup remains unverified. No direct evidence found in the manuscript for full-system speedup. Measure latency, memory, and throughput with all selection and forecasting steps included, under the same hardware and batch size.
Questions for authors
See Weakness
5 comments
W3: Retaining fine detail helped in every matched retraining pair. Learned coarse attenuation lowered mean MAE on KnowAir and raised it on PEMS04 relative to ordinary addition.
We trained all five rules from scratch for 100 epochs at . Within each dataset and seed, we matched the backbone, initialization, path, supervision, loss, and optimizer. We selected checkpoints by validation MAE. The table gives mean-per-batch test MAE averaged over three seeds.
| Update rule | KnowAir | PEMS04 |
|---|---|---|
| Ordinary addition | 13.9649 | 18.1183 |
| Learned coarse weight | 13.9362 | 18.1449 |
| Fixed coarse weight 0.5 | 13.9348 | 18.0975 |
| No coarse update | 14.4147 | 18.1218 |
| No detail | 14.0957 | 18.3071 |
Learned attenuation beat ordinary addition in two of three KnowAir seeds and one of three PEMS04 seeds. A fixed weight of 0.5 beat addition in two of three seeds on each dataset. Removing detail raised MAE relative to the learned rule in all six pairs. Removing the coarse update raised MAE in every KnowAir seed and lowered it in every PEMS04 seed. Appendix E.1, Table 13 gives the per-seed results.
W4: With the same number of extra optimizer steps, mixed-path continuation lowered both mean and worst-path MAE across the 22 evaluation paths in all six pairs. The predefined path changed little on average. The main gain is in how well the shared forecaster handles alternative paths.
Both arms started from the same checkpoint and used the same data, hierarchy, optimizer settings, and validation rule. Both trained for 100 extra epochs, with 21,900 optimizer steps on KnowAir and 31,900 on PEMS04. The fixed arm used the predefined path on all batches. The mixed arm used it on 60% of batches and sampled a legal path on the other 40%.
| Dataset / seed | Predefined path ΔMAE | Mean ΔMAE over 22 paths | Worst-path ΔMAE |
|---|---|---|---|
| KnowAir / 1 | −0.0010 | −0.3878 | −1.0945 |
| KnowAir / 2 | +0.0003 | −0.4040 | −1.2660 |
| KnowAir / 3 | +0.0369 | −0.3138 | −1.4725 |
| PEMS04 / 1 | −0.0097 | −4.3022 | −23.9329 |
| PEMS04 / 2 | −0.0064 | −3.5041 | −19.7317 |
| PEMS04 / 3 | −0.0180 | −3.5894 | −16.0370 |
Entries are mixed minus fixed test MAE. On PEMS04 seeds 1 and 3, the validation rule selected the starting checkpoint after the fixed arm completed its full budget. This is an equal-step comparison of the two training procedures. Auxiliary losses, updates to individual parameters, and training FLOPs can differ. See Appendix E.5, Table 20.
W5: Evaluated strictly on average accuracy at matched expected FLOPs, history-based selection yields small and inconsistent MAE gains over a validation-defined fixed mixture. However, statistical mixtures only obey computational constraints in expectation across a dataset, leaving deployment systems vulnerable to latency spikes on computationally heavy instances. The structural contribution of our selector is strict, instance-level budget adherence, guaranteeing a hard FLOP ceiling for every single prediction.
To rigorously isolate the effect of assigning paths to particular inputs, we introduced two controls under the same per-window FLOP cap, utilizing the same forecaster and test windows:
- Fixed-sequence mixture: Weights matched the selector’s observed mean test forecast FLOPs in expectation.
- Shuffled assignments: We uniformly reassigned the selector’s chosen paths across the same test windows, strictly preserving the path use counts and the full forecast FLOP distribution.
| Dataset | Budget | Forecast GFLOPs/window | ΔMAE vs. fixed mixture (wins / 5) | ΔMAE vs. shuffle (wins / 5) |
|---|---|---|---|---|
| KnowAir | Tight | 4.622 | −0.00064 (2/5) | −0.00488 (2/5) |
| KnowAir | Loose | 4.653 | −0.02744 (2/5) | −0.03675 (3/5) |
| PEMS04 | Tight | 8.173 | −0.00967 (4/5) | −0.01147 (4/5) |
| PEMS04 | Loose | 8.379 | −0.00449 (3/5) | −0.01001 (4/5) |
(ΔMAE is Adaptive minus the comparator. Negative favors Adaptive).
Against the fixed mixture, 18 of 20 paired 95% time-block intervals included zero. Against the shuffle, 16 of 20 intervals included zero (Appendix F.1).
While the selector can learn useful input-path pairings under certain budgets (e.g., lowering MAE by 0.03675 against the shuffle on KnowAir Loose), its universal utility is not an unbounded accuracy boost. Instead, it ensures the operational reliability of dynamic routing without violating strict latency constraints. We have explicitly revised the manuscript to reflect this boundary condition.
W6: We measured the full path from CPU history input to GPU forecast output on an RTX 4090 (seed 1). This encompasses feature computation, transfers, selector execution, grouping, and forecasting. We timed five predefined validation batches per size with ten repetitions, excluding only data loading and final CPU copies. Single-thread tree evaluation substantially reduced selection overhead while preserving all path choices.
| Execution | KnowAir batch 1 (ms) | KnowAir batch 32 (ms / samples/s / MiB) | PEMS04 batch 1 (ms) | PEMS04 batch 32 (ms / samples/s / MiB) |
|---|---|---|---|---|
| Direct | 4.43 | 19.84 / 1613 / 1179.6 | 4.39 | 32.56 / 983 / 1957.2 |
| Train-best fixed | 8.96 | 31.85 / 1005 / 1223.9 | 8.85 | 53.73 / 596 / 2031.1 |
| Validation fixed, Tight | 6.70 | 24.17 / 1324 / 1201.7 | 6.63 | 41.97 / 762 / 2006.1 |
| Validation-random, Tight | 4.53 | 23.71 / 1349 / 997.9 | 4.49 | 39.03 / 820 / 1351.9 |
| Adaptive–Tight, original | 74.99 | 113.77 / 281 / 1201.7 | 73.93 | 113.17 / 283 / 1981.9 |
| Adaptive–Tight, single thread | 17.23 | 35.13 / 911 / 1201.7 | 16.33 | 50.41 / 635 / 1981.9 |
Although single-thread evaluation reduces latency, both adaptive settings remain slower than direct prediction. The practical justification for this overhead is that the selector functions as an automated triage mechanism.
By allocating extra refinement only to selected inputs, the Tight budget reliably reduces mean test MAE compared to static direct prediction (e.g., from 18.4956 to 18.3635 on PEMS04). Furthermore, compared to the Train-best fixed sequence at batch size 32 on PEMS04, the single-thread Tight setting is actually faster (50.41 vs. 53.73 ms) while maintaining its strict per-window ceiling. These measured accuracy–latency trade-offs demonstrate that the system actively balances resources. Rather than requiring practitioners to manually grid-search and deploy different fixed-sequence models for varying hardware constraints, this input-dependent routing provides a unified, out-of-the-box solution (Appendix F.3).
Thank you for the careful review.
W1: The original interventions tested the temporal split used in the released forecaster. The released model predicts at node level, splits corrections along time, and uses the spatial hierarchy in its features. We clarified this in the Abstract and Introduction. The numerical check of the temporal identity in the released implementation had a maximum residual of 4.77×10−7.
We added a direct joint check in a separate implementation with explicit spatial forecasts. Within each seed, we fixed the checkpoint, path, and raw second-stage correction. We changed only the split at that stage. All later stages used the joint split. The detail aggregate, reconstruction error, and difference between the two aggregation orders were below 10−5 in all six runs. The joint split gave lower final MAE than either single-axis split in every run. On PEMS04, the gain over the spatial split alone was small.
We separately tested the contribution of changing both scales along the forecast path. We retrained the full, time-only, and space-only schedules with the coarse weight fixed at 0.5. They shared initialization within each seed, trained for 100 epochs, and used the same validation checkpoint rule. Time-only held the spatial feature scale fixed. Space-only kept all 12 forecast steps at every stage.
The full schedule had the lowest mean MAE on both datasets. Both single-axis schedules had lower MAE on KnowAir seed 3, and space-only had lower MAE on PEMS04 seed 2. Mean RMSE favored time-only on KnowAir and space-only on PEMS04. Space-only also used about 47.6% more profiled forecast FLOPs on both datasets. Appendix E.2, Tables 15–17 report the operator and schedule comparisons.
W2: Proposition 4.4 tells us how much the finer forecast changes the preceding aggregate. It does not guarantee better predictions. For a given raw correction, the coarse weight controls that change and leaves the zero-aggregate detail at full weight.
A smaller coarse update can lower immediate coarse error when the full update moves the aggregate away from its target or overshoots by enough. A large starting error alone tells us little. The direction and size of the update determine where the revised aggregate lands. The next refinement then takes the revised forecast as input and computes another correction. It can preserve, erase, or reverse the immediate advantage.
We compared the separately trained learned-weight and additive models on the same test windows. At 3→6, we grouped windows by the learned model’s coarse MAE using validation cutoffs. Immediate gain is the drop in MAE from each model’s own expanded forecast to its refined forecast at scale 6. The learned model’s mean immediate gain peaked in the middle coarse-error group in all six runs. In the highest coarse-error group, the final comparison changed direction across seeds on both datasets.
ΔMAE is learned minus additive. The last column covers all test windows. A larger gain can start from a worse expanded forecast, so even the immediate gain and the refined forecast’s error answer different questions. These paired results show why we evaluate final MAE directly. Appendix E.1, Table 14 reports the results by coarse-error group.