acceptodds
Under review as a conference paper at ICLR 2027

Mask-space distributionally robust optimization for learning under stochastic missingness

Abstract

Training with stochastic feature masking enables classifiers to handle missing data during inference. However, standard masked empirical risk minimization (ERM), even when aligned with the evaluation missing-rate distribution, can still overfit, leading to degraded performance across varying missing rates. We propose mask-space distributionally robust optimization (MaDRO), a transport-cost DRO framework centered at the joint data-mask distribution induced by a nominal masking rule, with transport restricted to mask space and toward additional missingness. MaDRO has a closed-form robust objective function: the base masked loss plus a hinge penalty on the worst deletion-cost-adjusted loss increase caused by one additional feature deletion. For each masked training example, MaDRO either keeps the sampled mask unchanged or deletes one additional observed feature, choosing the deletion only when its loss increase outweighs its deletion cost. Across public tabular and vision benchmarks, controlled semi-synthetic studies, structured missingness shifts, and attribution-fidelity analyses, MaDRO improves nominal-mask performance and reduces overfitting relative to aligned masked ERM.

Primary area:
optimization
Keywords:
Optimization, Distributionally robust learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_dKbEOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies learning under feature missingness without relying on imputation. Building on masked ERM, the paper argues that matching train-test mask distributions alone is insufficient, since prediction sensitivity to additional feature deletions can degrade both robustness under stochastic missingness and the fidelity of Shapley-style feature attributions. Under the assumption that missingness patterns are correlated with target labels, the authors propose MaDRO, a transport cost DRO framework defined around a chosen base masking distribution and a cost-penalized monotone feature deletion process. There are several related works and clarifications that should be discussed further (see Weaknesses and Questions).

Critical strengths
  • The supplementary material is thorough and contains many additional analyses. e.g., comprehensive, covering different training setups, masking mechanisms, and hyperparameter sweeps.
  • The paper tackles a practically relevant problem that is likely to be of interest to the ICLR community (robustness to missing features at test time without relying on feature imputation).
Critical weaknesses
  • Main message is unclear. The central contribution is somewhat difficult to identify. The main results are around budget comparisons but robustness is emphasized through out the paper. It is unclear which of these is the primary motivation and if the results are showing the robustness or even outperforming baselines. Also, the discussion around training budget and the use of only 3,000 updates appears important in the paper, but the rationale and practical significance are not sufficiently explained.

  • Limited baseline comparisons and motivation behind DRO. Main results include comparisons between the variants of the proposed method alongside relatively few external baselines (SAdD, HGB, Dropout) in the main paper. The evaluation would be stronger with comparisons against a broader set of existing approaches, especially methods that explicitly handle missing features or provide feature attributions. In addition, the implementation details of the dropout are not described, making it difficult to assess the fairness.

  • What is the key conceptual advantage gained from the DRO perspective compared to directly optimizing a masked ERM objective? Could a similar idea be incorporated into masked ERM while retaining the same robustness goals? More generally, if the key contribution is the cost-penalized feature deletion mechanism, alternative optimization formulations could achieve similar effects.

  • If computational efficiency is an important part of the paper, it would be useful to discuss connections to efficient Shapley-value approximation methods (e.g., FastSHAP and related approaches). Such methods may provide a natural baseline or point of comparison for the attribution-fidelity motivation.

  • Performance improvements are not consistently convincing. The empirical results do not consistently demonstrate clear improvements over the baselines. For example, Tables 1 and 2 do not show consistently superior performance compared to competing methods. The paper would benefit from a clearer explanation of when and why MaDRO is expected to outperform competing approaches.

    • Addition to above, there is limited discussion of dataset-specific behavior. Although vision tasks were mentioned in the main manuscript multiple times, the paper presents results on non-imaging datasets. Given that these results appear in supplementary materials and that Table 5 shows limited gains in several cases, additional discussion is necessary to explain where MaDRO succeeds, where it does not, and what characteristics of the datasets drive these outcomes.
  • For the robustness claim, having different masking rules and their results/discussion should be reported in the main manuscript. Important vs unimportant feature removal experiments also will beneficial.

(minor) The section on theoretical properties could likely be shortened and moved to the appendix. More space could instead be devoted to above weaknesses.. (minor) Intro section appears to be missing a citation to SAdD.

Questions for authors

Most of my questions are reflected in the weaknesses above. In particular, I would appreciate clarification regarding (i) the necessity of the DRO formulation over masked ERM and their potential variants w/o DRO, (ii) the role of computational efficiency and the training budget in the paper's motivation, and (iii) the interpretation of the empirical results where the proposed method does not substantially outperform existing baselines.

Reviewer_Zj6FOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes Mask-Space Distributionally Robust Optimization (MaDRO) to improve model robustness under stochastic feature missingness. The method formulates a transport-cost DRO framework that restricts perturbations to additional feature deletions and derives a tractable objective combining masked empirical risk with a cost-penalized worst-case single-feature deletion loss. Two variants with uniform and conditional-variance-based deletion costs are introduced. Experiments on tabular, vision, and semi-synthetic datasets demonstrate modest improvements in prediction accuracy under extended training, as well as potential benefits for robustness and Shapley attribution fidelity.

Critical strengths
  1. **Well-motivated method with theoretical basis. To be specific, the paper designs a theoretical DRO model for learning with stochastic missingness, and provides a practical objective function with theoretical guarantee for local feature deletion.

  2. **Overall experimental validation. This paper evaluate various data sets and missingness patterns, compare different training budgets, and test prediction accuracy, robustness, and attribution fidelity.

Critical weaknesses

The improvement induced by the masked ERM is slight and it is sensitive to different data sets. With the same training time, ERM could also beat MaDRO on the Gaussian benchmark. Additionally, MaDRO requires much more extra effort during adversarial feature deletion and hyperparameter search. Such results question the cost-effectiveness of small improvements.

Questions for authors

At what situations can MaDRO strike a balance between prediction accuracy and efficiency, and what are the cases in which MaDRO can still outperform masked ERM with comparable training-time cost?

Reviewer_XneGOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MaDRO for classification with missing features. It uses a distributionally robust optimization framework to control the loss increase caused by deleting an additional feature. A deletion cost determines whether the feature is removed during training. The paper provides theoretical analysis and evaluates prediction accuracy, deletion robustness, and attribution fidelity on tabular, vision, and semi-synthetic datasets.

Critical strengths
  1. The paper addresses a practical problem in prediction with missing inputs.

  2. The experiments include useful controls for training budgets, random deletion, and adversarial deletion.

Critical weaknesses
  1. The motivation does not establish why deletion sensitivity is an appropriate target for improving prediction. Deleting useful information can increase loss even for an optimal predictor. The paper does not establish when penalizing this increase addresses avoidable prediction errors.

  2. The objective can be indifferent to correctable errors on the original masked input. When the cost-adjusted deletion loss dominates, correcting those errors alone does not reduce the objective.

  3. The baseline comparison omits relevant methods for prediction with missing inputs. MIWAE [1] learns to impute missing numerical values from incomplete data, providing an alternative prediction pipeline when paired with a classifier.

[1] Mattei and Frellsen. MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets. ICML, 2019.

  1. The ablations provide limited evidence that feature-specific costs improve prediction beyond uniform costs. The benefit of thresholding varies across datasets and training budgets. These results do not establish a consistent advantage over simpler alternatives.
Questions for authors

Address the concerns raised in the weaknesses above.

Reviewer_c6vuOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper looks at training classifiers under stochastic feature missingness. Standard masked ERM matches the test mask distribution but does not control how much the loss rises when one more feature is deleted. The authors propose MaDRO, a transport-cost DRO objective over mask space where transport only deletes features, and with a delete-one adversary this reduces to the base masked loss plus a hinge on the worst cost-adjusted loss increase. They test it on WDBC, a Gaussian benchmark, two naturally incomplete UCI datasets, and exploratory sweeps on tabular and vision data, with some Shapley attribution diagnostics.

Critical strengths

The formulation is clean. Restricting transport to monotone deletions on top of any base masking rule gives a simple hinge form, and the paper is careful about what the surrogate does and does not guarantee. The thresholding idea is also a real difference from plain adversarial deletion, since the cost decides which examples get the robust branch, and the fixed-mixture control is the right way to check that this is not just a reweighted ERM and AdvDelete blend.

I also appreciate that the experiments are reported honestly. The authors compare at short, equal-time and extended horizons, select on validation data, and say plainly where ERM or HGB wins. That makes the paper useful as a reference on when this kind of regularizer helps, even if the answer is "not by much".

Critical weaknesses

The main claim is improved nominal-mask accuracy over aligned ERM, and the evidence for it is thin. On WDBC the gain is about 0.5 points against a partition SD of roughly 1 point. On the Gaussian benchmark it is about 0.7 to 0.8 points at the extended horizon, but ERM leads at equal training time and ERM itself gains 2.4 points just from training longer. On Credit Approval MaDRO is no better than ERM, and the boosted trees baseline beats every neural method on both natural-missingness datasets. So the result looks like a modest, budget-dependent regularization effect, not a clear win.

The cost side is also not on the same footing. Full-family search for MaDRO takes about 10,000 seconds versus about 500 for ERM, so at equal total compute including tuning, ERM with longer training or a better schedule may well sit on the same frontier. The larger public benchmark sweeps, including all the vision ones, pick the positive-cost configuration by test mixture mean, which means they cannot support a performance claim. Since the vision and larger tabular settings are exactly where the method would need to matter, the paper currently shows it on small, easy datasets with validation selection and on bigger ones only with test selection. The attribution-fidelity story is similarly secondary and the paper itself says that lower deletion sensitivity does not guarantee better attributions.

Questions for authors
  1. Can you rerun the vision and MiniBooNE sweeps with validation-based selection and the same equal-time and full-search budget accounting you use on Gaussian data? If the gain survives there, it changes my view a lot.

  2. If ERM gets the same total compute as the full MaDRO search, for example by tuning the learning rate, masking rate schedule, and training length, does the gap on the Gaussian benchmark remain?

  3. Is there a setting where MaDRO beats both ERM and the tuned boosted trees baseline by more than the partition SD?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.