- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper proposes ReViR, a visual retriever for time series forecasting. It first retrieves historical windows, then reranks them by the overlap of triangle regions constructed from representative points. The retrieved continuations are aggregated and combined with a lightweight forecaster. Experiments cover seven real-world forecasting benchmarks.
Critical strengths
- Comparing time series through geometric overlap is an interesting approach to historical retrieval. The similarity score can be inspected directly through the triangle regions, without relying on a pretrained vision encoder.
- Sampling density and raster resolution are separately configurable, giving users control over the representation size and computational cost.
Critical weaknesses
-
The evaluation needs improvement. First, it is unclear whether the forecasting gains come from retrieval. The paper presents only one retrieval example and does not quantitatively evaluate retrieval quality. Second, the non-retrieval baselines miss several of the latest and relevant methods, such as MixLinear, TimePro, etc. Third, the numbers used in the results are different from the numbers in the original papers for many methods. The authors need to show more details if they reran the experiments. To summarize, it is difficult to support the claim of SOTA with these gaps.
-
The paper evaluates the visual retriever by removing the visual module in the ablation study. This shows that the module is useful, but does not prove whether the gains come from the complex design of triangle-region overlap. The authors may compare it with simpler alternatives, such as using the same representative points with a distance-based similarity measure, to determine whether the gains come from representative point selection or the triangle shape designed.
Questions for authors
See the questions in the Weaknesses.
More:
- Many concepts remain unexplained, such as how the Raw Future MSE is calculated. It only appears in Fig. 1.
- Why does performance decrease despite an increase in the number of retrieved candidates?
- The hundredfold storage saving needs a clearer explanation, ensuring it's due to the new design and not other implementation skills.
- Reporting more experiment results with different seeds would be better, especially when the experiments are not heavy.
5 comments
W1: The non-retrieval baselines miss several of the latest and relevant methods, such as MixLinear, TimePro, etc.
We have added TimePro and MixLinear, and the table below reports their results alongside ReViR’s. Since the original TimePro paper uses a lookback length of 96, we reran its official implementation with a lookback length of 720 to match ReViR's evaluation setting, keeping all other configurations unchanged. The MixLinear paper reports MSE only, so we reran its official scripts as provided to obtain both MSE and MAE.
As shown in the table, ReViR outperformed TimePro. MixLinear performed slightly better than ReViR on ETTh1 and ETTh2, while ReViR performed better on the remaining datasets. As stated in the MixLinear paper, the authors performed hyperparameter searches for each dataset and setting. Therefore, its superior prediction performance on the ETTh1 dataset can be regarded as an implementation skill.
| Model | Horizon | ETTh1 MSE | ETTh1 MAE | ETTh2 MSE | ETTh2 MAE | ETTm1 MSE | ETTm1 MAE | ETTm2 MSE | ETTm2 MAE | Exchange MSE | Exchange MAE | Weather MSE | Weather MAE | Electricity MSE | Electricity MAE |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| ReViR | 96 | 0.362 | 0.385 | 0.270 | 0.330 | 0.290 | 0.338 | 0.160 | 0.246 | 0.083 | 0.200 | 0.162 | 0.205 | 0.130 | 0.226 |
| ReViR | 192 | 0.403 | 0.411 | 0.336 | 0.376 | 0.324 | 0.357 | 0.216 | 0.286 | 0.169 | 0.291 | 0.202 | 0.245 | 0.142 | 0.238 |
| ReViR | 336 | 0.434 | 0.429 | 0.371 | 0.406 | 0.353 | 0.376 | 0.267 | 0.321 | 0.307 | 0.400 | 0.247 | 0.282 | 0.151 | 0.249 |
| ReViR | 720 | 0.442 | 0.457 | 0.411 | 0.446 | 0.404 | 0.406 | 0.348 | 0.375 | 0.752 | 0.648 | 0.315 | 0.333 | 0.167 | 0.264 |
| ReViR | Average | 0.410 | 0.420 | 0.347 | 0.390 | 0.343 | 0.369 | 0.248 | 0.307 | 0.328 | 0.385 | 0.232 | 0.266 | 0.148 | 0.244 |
| MixLinear | 96 | 0.351 | 0.383 | 0.283 | 0.340 | 0.333 | 0.375 | 0.170 | 0.261 | 0.085 | 0.204 | 0.172 | 0.228 | 0.139 | 0.233 |
| MixLinear | 192 | 0.395 | 0.423 | 0.337 | 0.376 | 0.353 | 0.382 | 0.223 | 0.298 | 0.174 | 0.295 | 0.214 | 0.262 | 0.154 | 0.248 |
| MixLinear | 336 | 0.411 | 0.434 | 0.356 | 0.395 | 0.391 | 0.407 | 0.275 | 0.332 | 0.319 | 0.407 | 0.258 | 0.295 | 0.171 | 0.265 |
| MixLinear | 720 | 0.423 | 0.456 | 0.380 | 0.423 | 0.439 | 0.435 | 0.360 | 0.384 | 1.242 | 0.837 | 0.325 | 0.345 | 0.209 | 0.297 |
| MixLinear | Average | 0.395 | 0.424 | 0.339 | 0.383 | 0.379 | 0.400 | 0.257 | 0.319 | 0.455 | 0.436 | 0.242 | 0.283 | 0.168 | 0.261 |
| TimePro (Rerun) | 96 | 0.423 | 0.436 | 0.328 | 0.376 | 0.311 | 0.361 | 0.186 | 0.276 | 0.100 | 0.229 | 0.191 | 0.252 | 0.187 | 0.295 |
| TimePro (Rerun) | 192 | 0.462 | 0.462 | 0.401 | 0.418 | 0.356 | 0.389 | 0.258 | 0.314 | 0.196 | 0.323 | 0.242 | 0.293 | 0.222 | 0.327 |
| TimePro (Rerun) | 336 | 0.527 | 0.503 | 0.434 | 0.446 | 0.381 | 0.403 | 0.311 | 0.352 | 0.350 | 0.439 | 0.274 | 0.313 | 0.215 | 0.319 |
| TimePro (Rerun) | 720 | 0.505 | 0.504 | 0.506 | 0.500 | 0.443 | 0.436 | 0.367 | 0.395 | 1.718 | 0.965 | 0.335 | 0.356 | 0.283 | 0.373 |
| TimePro (Rerun) | Average | 0.479 | 0.476 | 0.417 | 0.435 | 0.373 | 0.397 | 0.281 | 0.334 | 0.591 | 0.489 | 0.261 | 0.303 | 0.227 | 0.329 |
| TimePro (Original) | 96 | 0.375 | 0.398 | 0.293 | 0.345 | 0.326 | 0.364 | 0.178 | 0.260 | 0.085 | 0.204 | 0.166 | 0.207 | 0.139 | 0.234 |
| TimePro (Original) | 192 | 0.427 | 0.429 | 0.367 | 0.394 | 0.367 | 0.383 | 0.242 | 0.303 | 0.178 | 0.299 | 0.216 | 0.254 | 0.156 | 0.249 |
| TimePro (Original) | 336 | 0.472 | 0.450 | 0.419 | 0.431 | 0.402 | 0.409 | 0.303 | 0.342 | 0.328 | 0.414 | 0.273 | 0.296 | 0.172 | 0.267 |
| TimePro (Original) | 720 | 0.476 | 0.474 | 0.427 | 0.445 | 0.469 | 0.446 | 0.400 | 0.399 | 0.817 | 0.679 | 0.351 | 0.346 | 0.209 | 0.299 |
| TimePro (Original) | Average | 0.438 | 0.438 | 0.377 | 0.403 | 0.391 | 0.400 | 0.281 | 0.326 | 0.352 | 0.399 | 0.251 | 0.276 | 0.169 | 0.262 |
W2: The authors may compare it with simpler alternatives, such as using the same representative points with a distance-based similarity measure, to determine whether the gains come from representative point selection or the triangle shape designed.
We compared Triangle-IoU with two alternatives that use the same sampled anchors. We kept the candidate pool and forecasting setup fixed and varied only the reranking measure.
Piecewise-linear . We connect the anchors of each series at their original time coordinates to obtain two piecewise-linear curves, and . We measure their root-mean-square separation over the history window of length in time coordinates:
To evaluate the integral exactly, we merge and sort the anchor timestamps from both curves as . With , we obtain
The factor weights each interval by its actual duration.
DTW. We extract the anchor values in temporal order as and , counting the window start once if it is also selected by the first sampling bucket. We use the absolute difference between anchor values as the local matching cost:
We set and initialize the remaining entries in the zeroth row and column to . The final distance is:
The table below compares forecasting accuracy across the three measures. Because all three use the same anchors, the comparison tests whether filled triangle-region overlap adds value beyond what can be achieved through pointwise comparison of the interpolated curves or elastic matching of the anchor values.
| Dataset | Horizon | ReViR MSE | ReViR MAE | Piecewise-linear L2 MSE | Piecewise-linear L2 MAE | DTW MSE | DTW MAE |
|---|---|---|---|---|---|---|---|
| ETTh1 | 96 | 0.362 | 0.385 | 0.365 | 0.390 | 0.363 | 0.390 |
| ETTh1 | 336 | 0.434 | 0.429 | 0.441 | 0.452 | 0.438 | 0.451 |
| Exchange | 96 | 0.083 | 0.200 | 0.083 | 0.201 | 0.083 | 0.201 |
| Exchange | 336 | 0.307 | 0.400 | 0.311 | 0.402 | 0.307 | 0.400 |
Q1: Many concepts remain unexplained, such as how the Raw Future MSE is calculated. It only appears in Fig. 1.
We have added definitions of both Raw Future MSE and Aligned Future MSE to the main text.
Let be the history length and the forecast horizon. Let and denote the observed values of the query and the top-ranked candidate returned by the retrieval method, respectively.
Raw Future MSE measures the error between the retrieved candidate’s original continuation and the query’s ground-truth future.
Aligned Future MSE uses the same candidate and measures the smallest future MSE attainable by applying a constant vertical shift from the set . The set includes zero and the shift that aligns the candidate with the query at each historical time step. We select the shift retrospectively using the query’s ground-truth future, so Aligned Future MSE is an oracle diagnostic of how closely the retrieved continuation can match the query’s future after a vertical shift.
,
Q2: Why does performance decrease despite an increase in the number of retrieved candidates?
Increasing incorporates lower-ranked candidates whose less relevant continuations may introduce noise into the weighted aggregation. Its optimal value depends on the recurrence of relevant patterns and retrieval-library coverage: broader aggregation benefits from abundant relevant matches but may introduce mismatched continuations when such matches are scarce. Among the evaluated settings, achieves the best trade-off between forecasting accuracy and retrieval efficiency.
Response to Reviewer UkEM
Dear Reviewer UkEM,
We greatly appreciate your time and the insightful comments on our manuscript. Your suggestions are very helpful for improving this work, and we have carefully revised the paper to address all the issues you raised. Our point-by-point responses are provided below:
W1: It is unclear whether the forecasting gains come from retrieval. The paper presents only one retrieval example and does not quantitatively evaluate retrieval quality.
We have expanded the case studies in the Appendix D to cover datasets. Each case contains two panels. Panel (a) shows the histories and continuations of the top-five retrieved candidates, allowing readers to examine how similar historical patterns relate to subsequent trajectories. Panel (b) overlays the query’s Visual Sampling triangles with those of the top-ranked candidate after reranking, and compares the forecasting continuation with the query’s ground-truth future. These examples illustrate the geometric correspondence captured by Triangle-IoU and the information that retrieved continuations provide for forecasting.
W1: The numbers used in the results are different from the numbers in the original papers for many methods. The authors need to show more details if they reran the experiments. To summarize, it is difficult to support the claim of SOTA with these gaps.
The main results table compares forecasting accuracy under a unified experimental protocol. We reran all baselines because their original evaluations used different settings, including input lengths and hardware environments, and some papers did not report all the metrics needed for our comparison. To ensure a fair comparison across all datasets and metrics, we evaluated every method using 720 historical time steps, forecast horizons of 96, 192, 336, and 720 steps, and identical chronological training, validation, and test splits. All experiments were conducted on an NVIDIA GeForce RTX 4090 GPU with 24 GB of memory, using PyTorch 2.4.0 and CUDA 12.1.
We will release the exact code, commands, and configurations for every baseline run, organized by model, dataset, and forecast horizon. These materials will specify the learning rate, batch size, number of training epochs, random seed, and model-specific settings.