acceptodds
Under review as a conference paper at ICLR 2027

Do Graph-Completion Front Ends Help Mask-Aware Forecasting? A Study of Structured Sensor Outages

Abstract

Does graph completion help when a forecaster already represents missingness? We study RESOLVE, a two-step clamped completion front end, against four controls using shared structured input outages and unchanged future targets. Across Beijing Air, METR-LA, and PEMS-BAY, the primary matrix contains 45 checkpoints and 720 evaluation units. RESOLVE achieves lower mean absolute error in 17 of 24 designated sensor- and mixed-outage comparisons, but only two of six against a mask-aware diffusion-recurrent model. An equal-parameter validation ablation likewise finds small, uneven gains. Native-input estimates meet a 2% point margin against GraphGRU, whereas risk-conditioned intervals satisfy a descriptive coverage–width criterion in only 10 of 45 cells. Separate three-seed weather and feeder-energy studies yield contrasting rankings: weather favors diffusion and feature propagation, while RESOLVE has the lowest mixed-outage energy mean error. The small energy advantage over the diffusion comparator is not stable to paired-seed resampling. Prior test-aggregate exposure and unavailable primary per-origin evidence limit the primary findings to descriptive comparisons; supplementary bootstrap intervals are sensitivity checks. Overall, the evidence supports neither a universal completion advantage nor an isolated causal benefit, and highlights the need for matched architectural controls and evaluation beyond network-wide average error.

Primary area:
learning on time series and dynamical systems
Keywords:
time series forecasting, graph learning, missing data, structured sensor outages, empirical evaluation

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_o2gcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper conducts an empirical analysis of missing time-series completion in spatiotemporal datasets. It investigates whether data completion can improve forecasting performance or whether models that natively handle missingness, such as Mask-DCRNN, can already perform the task effectively.

Critical strengths
  • Principled evaluation methodology: Evaluates methods under different types of missingness.
  • Multiple datasets and contexts: Tests the approaches across several datasets and settings.
  • Honest reporting: Transparently reports the results and limitations of the empirical study.
  • Baseline comparisons: Compares the proposed approach against several baselines.
Critical weaknesses

Presentation and Clarity:

  • Abstract clarity: The abstract introduces many terms that are difficult to understand without reading the paper first, making it unnecessarily difficult to follow.
  • Motivation for graph completion: The introduction describes graph completion as a “natural response.” However, some of the experimented datasets use artificially constructed correlation graphs, making this assumption less obvious. It is also unclear whether the relationships captured by these graphs are necessary, or whether a model could learn them implicitly.
  • Output dimensionality (Section 3): (\hat{Y}) is defined over (P) and (N), without a feature dimension. It is unclear why the model does not forecast each of the (F) input channels, or whether it predicts only a single target variable per node.

Content and Contribution:

  • Ambiguous contribution: The paper's main contribution is somewhat unclear. Is the goal to demonstrate that Mask-DCRNN effectively exploits missingness information, or to introduce RESOLVE? The paper acknowledges that the completion method is established and frames the work as an empirical study, but this does not by itself establish the significance of the findings.
  • Unclear takeaways: even the conclusion that Mask-DCRNN handles missing values effectively is not consistently supported across all datasets. This makes the overall takeaways less clear. The paper could benefit from more discussion of the characteristics and dynamics of each dataset, potentially identifying patterns that explain why different baselines perform better in different contexts.
Questions for authors

What is the justification for the chosen 2% performance threshold?

Reviewer_6GPuOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper empirically evaluates whether a graph-completion front end, specifically RESOLVE, a two-step damped clamped feature-propagation operator, improves mask-aware spatiotemporal forecasting under structured sensor outages. Across Beijing Air, METR-LA, and PEMS-BAY, RESOLVE achieves lower descriptive MAE than the tested controls in 17 of 24 designated sensor- and mixed-outage contrasts, but wins in only 2 of 6 comparisons against MASK-DCRNN, a mask-aware diffusion-recurrent model. An equal-parameter validation ablation yields a negligible mean relative improvement of 0.074% and fails the prespecified continuation criteria. The authors conclude that the reported evidence establishes neither a universal advantage of graph-completion front ends nor an isolated causal benefit attributable to completion, framing the study as an empirical diagnostic rather than a claim of broad superiority.

Critical strengths
  1. The evaluation isolates confounding factors by generating timeline-level failure masks prior to windowing, holding targets and eligibility masks invariant, and pairing shared corruption across models under a controlled structured-outage protocol. Moreover, granular diagnostics decompose error into input-failed and unaffected subsets, exposing substantial performance divergence hidden by aggregate network-wide averages.
  2. The manuscript explicitly notes that the equal-parameter ablation failed pre-registered continuation criteria and discloses that exploratory test exposure guided comparator choices. Documenting these negative outcomes and recording the interval flag discrepancies in the open provides valuable methodological clarity, even if it cannot cure the underlying statistical inference and selection-bias constraints.
Critical weaknesses
  1. The confirmatory strength of the primary evaluation is constrained by prior exploratory test exposure that influenced comparator selection and subsequent research-question formulation. Missing per-origin additive-error shards in the primary transfer further prevent reconstructing the planned 10,000-draw paired moving-block bootstrap and multiplicity-adjusted superiority decisions. Because the 15 evaluation cells per condition share only three fitted checkpoints and five corruption realizations rather than 15 independent training runs, the reported cell-level standard deviations characterize cell dispersion rather than standard errors, leaving the core empirical claims purely descriptive point estimates unsupported by formal statistical inference.
  2. The empirical evaluation omits specialized missing-data forecasting methods like BRITS, GRIN, PriSTI, and HD-TTS, limiting the assessment of RESOLVE relative to existing approaches despite potential differences in task settings. Furthermore, RESOLVE achieves lower MAE than MASK-DCRNN in only 2 of 6 primary settings with substantial dataset-dependent performance variation, while the equal-parameter ablation yields a negligible 0.074% gain and fails pre-registered continuation criteria. These mixed outcomes provide limited evidence for a meaningful benefit attributable specifically to completion, leaving the findings largely restricted to descriptive comparisons rather than a generalizable account of when graph completion helps.
Questions for authors
  1. Which test splits and metrics were inspected during exploratory analysis, which comparator choices were conditioned on them, and was any evaluation performed on an untouched holdout after protocol freezing?
  2. Given that the fitted checkpoints are retained, can the primary per-origin error files be regenerated to reconstruct the prespecified paired moving-block bootstrap and formal superiority decisions? If not, which planned inferential conclusions remain unverifiable?
  3. What technical or protocol constraints prevented the inclusion of specialized missing-data forecasting methods like BRITS, GRIN, PriSTI, or HD-TTS, and what explains the sharp performance reversal between Beijing Air and PEMS-BAY given the negligible 0.074% ablation gain?
Reviewer_KLe8OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper asks whether a graph-completion front end helps forecasters that already encode missingness.

Critical strengths

Failures are generated on the original split timeline before windowing, and failure seeds depend on dataset, condition, and replicate but not on method or training seed.

The paper separates network-wide error, input-failed subset error, and interval coverage paired with width. Table 2 shows failed-node MAE roughly 1.2 to 1.9 times the unaffected value, which a network-wide average alone would hide.

Critical weaknesses

MASK-DCRNN has 50,934 parameters on Beijing and 36,828 on traffic, against 15,654 and 14,988 for the RESOLVE family, and also differs in recurrent operator and mask encoding. The 2/6 headline count cannot be attributed to completion.

The equal-parameter ablation, the only design that isolates completion, wins 2 of 6 settings with a mean relative improvement of 0.074% against a pre-set 1% threshold, and it is validation-only. The paper’s cleanest evidence is null.

The pre-specified 10,000-draw bootstrap, the superiority rule, and the non-inferiority test were all defined but none were executed, because the per-origin shards were not transferred. The primary matrix is point estimates only.

All five methods are in-project definitions or adaptations rather than official reproductions, so the negative conclusion is about this implementation family rather than about completion front ends in the literature.

Questions for authors

The equal-parameter ablation is the only design that isolates completion. Can it be run on the test split across all conditions and reported as the primary comparison, with the model-level comparison demoted to context?

SAFE-ST, which only renormalizes incoming messages by source availability, has lower MAE than RESOLVE in all four PEMS-BAY cells. Under what conditions does the two-step clamped update beat this simpler control?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.