- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This manuscript challenges existing public benchmarks for irregular multivariate time-series (IMTS) forecasting. By constructing and evaluating naive methods within a specific anchor family on these benchmarks, the paper points out that the complex designs of modern neural networks for IMTS may be redundant. Furthermore, the authors conclude that neural modeling is mainly advantageous when the input context is short and less informative.
Critical strengths
- The problem is clear, and the motivation to question the existing irregular time series benchmark is reasonable and meaningful. As corresponding neural models become more and more complex, my own intuitions also indicate that the patterns under these irregular series may be sufficient enough for imputation and therefore easy to model. The authors verified this point in a relatively competitive benchmark.
- The verifiers used in the study are simple and reasonable, and thus are robust and trustworthy in reflecting the conclusion.
Critical weaknesses
Although the motivation is valid and the empirical results support the authors' main claim, the content of the manuscript is rather thin, whether in the methodology or the experimental section. It highlights a critical issue—that the community should be more mindful of the necessity and validity of complex neural modeling for irregular time series—which is valuable. However, the authors merely report experimental results showing that naive methods can achieve competitive or even superior performance on the selected benchmarks. While this is a meaningful empirical finding, it does not meet the bar for a full scientific paper in the ICLR main track. Besides, the main figure and tables need substantial improvements; the AI-generated evidence is strong.
In conclusion, I lean toward a 'reject' rather than a 'strong reject,' as the paper raises a valid challenge to the community, even though its content and depth are far from sufficient for a rigorous scientific publication. Since that specific intermediate option is unavailable in this year's scoring standard, I am assigning a score of 2.
Questions for authors
Could you polish the main figure and include more in-depth analyses? For example, how should future neural networks for irregular time series be designed based on your findings, and what are the theoretical intuitions behind this redundancy? Furthermore, would this conclusion still hold if the patterns in the irregular time series were sufficiently complex?
2 comments
I thank the authors for their additional explanations and modifications. After reviewing the updated PDF, I maintain my position that the figures and tables are not yet polished enough for a top-tier conference, as they retain a distinctly AI-generated style; the authors should at least refine and adjust them manually. Furthermore, the additional evidence indicates that the study's main conclusion is not universal, which may limit its broader impact and influence.
[Request for Acceptance Appraisal] Strengthened Scientific Depth, Mechanistic Evidence, and Boundaries of Neural Modeling Necessity
We appreciate the reviewer’s recognition of the central question and take the concern about scientific depth seriously. The revised paper is not merely about simple models winning: it develops a benchmark audit of which historical signals explain IMTS performance, when they suffice, and when neural models add value, aiming to enforce a basic but challenging guardrail that all future IMTS studies must surpass before claiming the SOTA title.
Q1. Is there sufficient methodological and scientific depth?
Section 3 and new Figure 2 define ANCHOR as a five-level diagnostic framework, not one naïve predictor. A shared library contains persistence, robust summaries, EMA, bounded trends, and phase lookup. Naive-, Expo-, and SparseAnchor isolate structures; ERMAnchor uses non-test empirical selection; AutoAnchor adds structural diagnostics. Per-variable choices are frozen before testing, without sample-wise test routing. The evaluation now spans 18 latest neural baselines, four datasets, and matched splits, scaling, masks, and metrics.
The framework can in principle operate on regular or irregular series, without implying untested regular-time superiority. It controls for predictability already available from history. Our Anchor family serves as a diagnostic tool for future studies in irregularity.
Q2. Are the underlying irregular series genuinely complex?
The new Table 1 documents measurable heterogeneity in the sampling process:
USHCN combines highly variable gaps with sparse asynchronous variable coverage. Sampling irregularity is real, but it does not imply that the target-relevant information is irreducibly complex.
Q3. What explains the apparent redundancy of neural complexity?
The revised analyses move beyond aggregate error:
Theoretical intuition: when low-complexity summaries already capture most target-relevant historical information, learned representations have limited extra signal to exploit. When latent states, cross-variable coupling, or longer dependencies matter, that sufficiency breaks down. Perturbations and ablations support this interpretation, but we do not present it as a formal theorem.
Q4. Would the conclusion survive sufficiently complex patterns?
Not universally, and the revision demonstrates why. In Figure 3, AutoAnchor achieves the lowest MSE in 13 of 19, not all, lookback–horizon settings. On USHCN, its MSE rises from 0.1565 at lookback/horizon (150,3) to 1.1591 at (100,3); extending the horizon at lookback 150 also restores a neural MSE advantage. Under 10% history retention, even after matched recalibration, AutoAnchor reaches 0.4265 ± 0.0831 MSE versus 0.2351–0.2673 for neural baselines. These are explicit counterexamples defining the boundary of our claim.
Appendix Table 12 shows MSE gains over persistence even in high-volatility strata (29.0% P12; 22.7% MIMIC; 65.0% USHCN; 35.4% HumanActivity). Yet volatility is not a complete test of nonlinear or coupled dynamics, so we avoid that stronger claim.
Q5. Figure quality and implications for future architectures.
We polished Figure 1 into a TikZ-style technical schematic; new Figure 2 visualizes ANCHOR's rule library, calibration, and frozen test-time forecasting following same theme as Figure 1 for optimal visual consistency. The numerical evidence comes from the documented experiments, not the illustrations; anonymous source code is provided. Section 5 identifies priorities for neural design: latent states, cross-variable dependencies, regime changes, and limited-history/long-horizon forecasting.
We respectfully request a positive reconsideration: the revised contribution is a mechanistic benchmark audit, with a reusable control framework, quantitative irregularity analysis, interventions, uncertainty, and documented failures. It asks when complexity earns its predictive value, rather than presuming it from leaderboards.