- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
MAFF addresses forecasting from incomplete time series within an impute-and-forecast pipeline. Its central intervention computes instance-normalization statistics from observed entries only and reuses them for denormalization, excluding fillers from the statistics applied to both inputs and forecasts. The framework also injects gated per-channel missingness summaries into backbone-specific feature pathways and provides a learnable MASK token for fully missing channels. Evaluation covers five datasets, four backbones, four synthetic missingness protocols, and four forecasting horizons, using shared masks across methods and three mask seeds with a fixed training seed. Reported aggregate MSE reductions are 2.7% for CMoS, 0.7% for iTransformer, 0.4% for CycleNet, and 0.2% for DLinear. Component ablations provide evidence for observed-only normalization, while clean-data controls and reported mask statistics leave questions about attribution and protocol comparability. The MASK token does not activate in the reported experiments.
Critical strengths
- S1. Clear mechanistic problem formulation and lightweight pipeline integration. The paper identifies a concrete pathway through which fillers affect forecasting: they enter instance-normalization statistics, which are subsequently reused during denormalization. Equation 1 directly excludes fillers from those statistics without introducing a learned imputer or additional parameters for this component. The normalization-only ablation retains the baseline's parameter count and reports a 2.8% aggregate MSE reduction on the ETTh1/CMoS subset with horizons 96 and 336, with positive gains under P2, P3, and P4 (Section 6 and Table 8). This is meaningful evidence that at least part of the improvement does not require additional capacity. The intervention is simple and potentially practical, although excluding fillers does not guarantee recovery of the complete window's true statistics.
- S2. Transparent empirical reporting of negative results, boundary conditions, and per-window behavior. The evaluation spans four architecture families and four missingness protocols with shared masks across methods. The authors report absolute MSE/MAE, component ablations, clean-data controls, and mask-seed variability, while explicitly disclosing small or negative gains, the inactive MASK token, and the dropout-stream difference between paired runs (Sections 5, 6, Appendices A, F, and G). The per-window analysis further shows that 36.2% of test windows regress under P2 in the examined ETTh1/CMoS setting despite a positive aggregate gain. This reporting makes the method's benefits and limitations assessable and provides a useful basis for investigating the unresolved attribution questions.
Critical weaknesses
- W1. The missingness-specific contribution is not sufficiently separated from wrapper capacity and training-path effects. CMoS improves even on complete inputs, by 4.8% on ETTh1 and 4.0% on ETTm1 after pooling horizons. Using the displayed values in Table 1, its gain under missingness is smaller than the corresponding clean-data gain at the same dataset and horizon in 60 of 80 cells, including all 20 P1 cells. This does not establish that the missing-data gains are artifacts, but it makes a direct attribution control important. The paper acknowledges additional wrapper parameters, a normalization-epsilon difference between MAFF and native CMoS (), and different dropout draws caused by wrapper construction. The clean-data result is a useful diagnostic, but it is not an established bound on these effects under missing inputs and cannot simply be subtracted as a causal correction.
The normalization-only ablation is valuable parameter-matched evidence, but it is limited to ETTh1 and two horizons, and the manuscript does not establish whether epsilon and other training-path differences are also matched in that comparison. Moreover, Table 6 measures variability over mask seeds with a fixed training seed, leaving training-seed robustness unresolved. This is particularly consequential for the 0.2 to 0.7% aggregate gains outside CMoS. The interpretation of the feature pathway also needs clarification. Sections 5 and 6 describe the embedding as conveying gap position, whereas the three specified summaries can be identical for gaps of the same length at different temporal locations. Protocol-specific priors or the completed input may still provide positional information, but this differs from explicitly encoding location in the mask summaries. Likewise, the threshold sweep establishes insensitivity to the tested values of 𝜏, not the necessity of gating. These are secondary limitations on the mechanistic explanation rather than independent grounds for rejection.
W2. The reported mask statistics are inconsistent with interpreting the protocols as having a common 30% realized missing rate. Sections 3 and 4 state that each protocol targets 30% missingness. However, Table 16 reports mean longest missing runs under P2 of 53.86 for ETTh1/ETTh2, 53.83 for ETTm1/ETTm2, and 92.33 for Weather, with look-back length L=96. For each window and channel, the longest missing run cannot exceed the total number of missing entries. Under the table's stated averaging convention,
These values therefore imply mean realized missing rates of at least approximately 56.1% on the ETT datasets and 96.2% on Weather. A nominal generator parameter of 0.3 could differ from the realized rate, but the manuscript does not explain this discrepancy. Shared masks still support within-protocol comparisons between the baseline and MAFF. The concern is that missingness amount and structure are confounded when comparing gains across protocols, weakening the claim that the ordering reflects exploitable structure rather than severity. The same clarification is needed to interpret the missingness-strength sweep.
Questions for authors
-
Q1, addressing W1: attribution and statistical reliability. Were normalization epsilon, backbone initialization, minibatch order, and stochastic draws matched in the normalization-only ablation? Please clarify and, where necessary, repeat a representative part of the existing ETTh1/CMoS ablation with these factors controlled and paired results across several training seeds. A capacity-matched wrapper control with constant mask features would help isolate the injection contribution; to test that contribution specifically, it should retain the same observed-only normalization as full MAFF. Reporting normalization-only results under none, with matched epsilon, and absolute MSEs for the clean-data controls would further clarify the source of the offsets. These are targeted controls for the proposed explanation, rather than a request to repeat the entire benchmark grid.
-
Q2, addressing W2: mask-generation clarification and validation. How does the nominal ratio determine block counts, lengths, overlaps, and channel coverage? Please reconcile Table 16 with the 30% setting and report realized missing fractions for each dataset/protocol, including the strength sweep. Why is the P3 gain reported as +8.9% at every tested ratio, and does the nominal ratio change its generated masks? If realized rates differ substantially, does the protocol-ordering conclusion survive a comparison at matched realized rates on a representative existing setting?
-
Q3, addressing the feature-pathway interpretation in W1: positional information and mathematical specification. Please provide the exact equations and dimensions mapping the three summaries through gating, the GELU projection, and residual injection. Are any raw-mask or explicit positional features used beyond the three stated summaries? If not, how should the statements that the embedding conveys gap position be understood? In particular, please distinguish information explicitly represented by the summaries from information inferred through the completed input or a protocol-specific prior.
-
Q4, addressing the component-level evidence in W1: gate necessity. Figure 7 already evaluates threshold sensitivity. On that existing setting, how does MAFF perform with , keeping the summaries, projection, and training setup unchanged? This would clarify whether the gate adds value beyond the learned projection or whether the evidence supports the simpler ungated feature pathway.
3 comments
Q2 Mask generation, realized rates and protocol ordering.
| Protocol | ETT | Weather | Longest run (ETT / Weather) |
|---|---|---|---|
| P1 | 30.1% | 30.0% | 3.5 / 3.5 |
| P2 | 85.4% | 96.9% | 53.5 / 92.5 |
| P3 | 17.0% | 18.8% | 16.3 / 18.1 |
| P4 | 30.2% | 30.2% | 29.0 / 29.0 |
On block counts and coverage: P2 computes its block count as with , but every block covers all channels, so coverage scales with the channel count, about 85% at 7 and 97% at 21, with overlapping starts keeping it slightly below the bound. P3 fixes the intra-group block length at 20% of the window, so the ratio only reaches sampling inside an already zeroed region and the realized fraction stays at 17% to 19%. Hence the same number at every ratio: the ratio never reaches the mask, which is not the model being insensitive to it. The corrected implementation is a separate protocol and P1 to P4 stay bit-identical; afterwards P2 realizes 28.0% at a nominal 30% and no longer scales with channels, and P3 realizes 30.4% and scales with the ratio (9.8%, 30.4%, 50.0%).
On whether the ordering survives at matched rates we tested three backbones by three strengths (ETTh1, three seeds, paired differences, negative means lower MSE):
| Backbone | Protocol | r=0.1 | r=0.3 | r=0.5 |
|---|---|---|---|---|
| CMoS | short blocks | +0.02% | −0.19% | −0.60% |
| CMoS | correlated long blocks | −0.11% | −2.45% | −6.09% |
| CMoS | tail | −0.72% | −5.64% | −12.32% |
| iTransformer | short / corr. / tail | −0.38% / −0.36% / −0.17% | −1.10% / −0.59% / −0.99% | −0.41% / −0.37% / −1.46% |
| CycleNet | short / corr. / tail | +0.34% / +0.23% / −0.26% | +0.33% / −1.41% / −1.53% | −0.06% / −3.38% / −4.07% |
The ordering has two parts. Tail strongest and correlated groups second holds at matched rates, and the backbones share the shape. Block missingness ranked second because its realized fraction was 85%, not because of its structure; at the matched 28% it falls into the noise with mixed signs, so the revision restricts the gain to long contiguous gaps and tail gaps. The same protocols on ECL (321 channels): +1.10%, −0.36%, −2.95%, −6.68%.
Q3 Equations and position. The three summaries (missing rate, longest contiguous missing run and fully-missing indicator) give the same input for equal-length gaps at different positions, so they carry no position; position is carried by the per-position normalized values and by the completed input. Sections 5 and 6 attribute the gain to the model knowing where the gap is without separating information the summaries explicitly represent from information inferred from the input, and the revision corrects that and gives the full equation: , where , , and is a per-channel scalar initialized small; backbones with aligned tokens project the embeddings at the same position. is initialized to the identity on the block and to zero on the block and the bias, so training starts from no injection, and no raw mask or explicit positional feature enters this path beyond the three summaries.
Q4 Necessity of the gate. Across the error changes by less than 2e-3%, so the gate is not a sensitive hyperparameter. Together with the paired control in Q1, the body of the gain of this path comes from the learnable residual, and the gate and the summaries are corrections of the same small order; the revision describes the gate as an initialization detail.
We hope these clarifications address your concerns. We would be glad to provide further details if needed.
Thank you for auditing the mask generator and running the matched controls. The results strengthen the case for observed-only normalization, particularly for tail gaps on CMoS. Given the small incremental gain from the mask summaries and the variation in protocol ordering across backbones, I am keeping my rating at weak rejection. I have no further questions.
We thank the reviewer. W1, W2 and the feature-pathway questions all bear on one quantity: how much of the wrapper's gain belongs to the mechanism we claim. We ran 564 additional runs.
Q1 Attribution. You ask for a wrapper that retains observed-only normalization, holds the mask features constant and matches capacity. We implemented it: the full wrapper, the same parameter count and observed-only normalization, with the three summaries zeroed so the injection path carries no missingness information.
The paired difference is 0.03% (24 pairs, all with the same sign, paired standard deviation 0.010%), so the mask information does contribute; the residual branch of the same path spans CMoS −5.56% to DLinear +0.38% on clean data, two orders of magnitude larger.
The component decomposition on clean data (protocol none) shows the other side, ETTh1, three seeds, two horizons.
These are the none-row absolute values requested in Q1. The normalization-only arm is bit-identical to the baseline on all four backbones, maximum absolute difference 5.96e-8; epsilon moving from 1e-10 to 1e-5 gives 1.03e-7 relative, below resolution. The offset comes entirely from the injection branch and changes direction, CMoS −5.56%, CycleNet −0.90%, iTransformer −0.10%, DLinear +0.38%, which accounts for the 60 of 80 cells.
DLinear is an additional negative control: no instance normalization, nothing for the repair arm to act on, difference zero under all four protocols. This also answers the dropout-stream point: the wrapper does change the random stream, but here both arms run the same definition and forward path and the difference is numerically zero.
On minibatch order and initialization, both arms use identical definitions, the same seed and batch order, differing only in where the statistics come from, at 937 parameters each.
Training seeds: three (2021, 7, 13), mask seed fixed, ETTh1, three backbones, four protocols, two horizons, two arms, 144 runs. For a fixed configuration the training seed moves the error by 0.02% to 2.29%, median 0.12%. Paired difference of the repair arm from the baseline follows (%, negative means lower MSE; each cell gives horizon 96 then horizon 336).
21 of 24 cells share a sign across the three seeds and all seven cells above 1.5% do, with a maximum seed standard deviation of 0.92%. The three with mixed signs are CMoS short blocks at 336 (+0.02%, seeds −0.07, +0.04, +0.07), iTransformer correlated groups at 336 (−0.23%, −0.61, −0.41, +0.33) and iTransformer tail at 336 (−1.43%, −3.72, −2.26, +1.70); there the seed spread exceeds the effect and we do not rely on that cell. Point-wise missingness is consistently positive across the three backbones and the three seeds, from +0.24% to +0.87%.
Q2 to Q4, with the strength sweep, follow in the next comment.