acceptodds
Withdrawn from ICLR 2027

Co-Evolutionary Probabilistic Recovery for Multi-Dimensional Classification with Missing Labels

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Multi-dimensional classification (MDC) requires assigning labels to each instance across multiple class dimensions simultaneously, but fully labeling all dimensions is often costly or even impractical. Thus, a setting with missing labels more closely approximates the reality of MDC. Despite its practical relevance, learning from incomplete labels has yet to be investigated in MDC. Existing label completion strategies developed for other learning paradigms cannot be directly applied to MDC, as they typically rely on predefined label correlations or independent recovery assumptions, while overlooking the fact that missing labels may simultaneously obscure the dependencies among multiple class dimensions. Moreover, inaccurate completion can further distort the dependency information available to the learner and introduce unreliable supervision for downstream classifier learning. To address these challenges, we propose CSER, a competitive co-evolutionary framework for probabilistic label recovery. CSER first constructs complementary feature-only and dependency-conditioned class-probability representations from observed labels, and then co-evolves two interacting populations of recovery strategies and pseudo-missing challenges. The challenge population searches for difficult recovery targets that expose weaknesses of current strategies, while the recovery population learns confidence-aware, dimension-wise fusion preferences. A robust strategy is finally selected against accumulated historical challenges and used to recover labels for genuinely missing entries, with multiple imputation propagating the recovered supervision to downstream MDC learning. Experiments on 17 benchmark datasets demonstrate that CSER significantly outperforms MDC baselines equipped with label completion strategies across various label-missing scenarios.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Multi-Dimensional Classification, Incomplete-Label Learning, Competitive Co-Evolution, Missing-Label Recovery, Probabilistic Label Inference

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_hCGsOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates multi-dimensional classification (MDC) with missing labels and proposes CSER, a competitive co-evolutionary framework for probabilistic label recovery. The method constructs feature-only and dependency-conditioned probability representations and learns confidence-aware fusion strategies through the co-evolution of recovery strategies and pseudo-missing challenges. The recovered label distributions are then used for multiple imputation and downstream MDC learning. Experiments on 17 benchmark datasets demonstrate consistent improvements over existing MDC methods under different missing-label rates.

Critical strengths
  1. The paper addresses a meaningful and practically relevant problem. Most existing multi-dimensional classification (MDC) methods assume fully observed training labels, which may be difficult to obtain in real-world applications. Investigating MDC with incomplete labels is therefore valuable and extends the applicability of existing MDC research.

  2. The proposed framework is intuitive and well motivated. The authors construct pseudo-missing challenges from observed labels to provide verifiable feedback for recovery-strategy learning. By jointly evolving recovery strategies and challenging missing-label configurations, the framework offers an interesting way to improve the robustness of label recovery.

Critical weaknesses
  1. The recovery strategy involves only one fusion coefficient per class dimension, which could be optimized directly using observed labels and cross-fitted predictions. Comparisons with simpler approaches, such as direct Brier-loss minimization, would help establish the benefits of the proposed framework.

  2. CSER employs multiple imputation and ensemble prediction, while the baselines use their original MDC classifiers after deterministic label completion. Comparisons using the same downstream classifier and direct evaluation of recovery accuracy on hidden labels would provide more convincing evidence.

  3. The contribution of cross-dimensional dependency modeling remains unclear. The individual effectiveness of feature-only and dependency-conditioned probability representations is not sufficiently evaluated. Comparing these representations separately would help establish whether cross-dimensional predictions provide additional benefits and justify their fusion.

  4. The evaluation is limited to randomly generated missing-label patterns. In practice, missingness may depend on class identity or sample difficulty. Since pseudo-missing challenges are constructed only from observed labels, their effectiveness under non-random missingness remains unclear. Experiments with more structured missing-label patterns would strengthen the evaluation.

Questions for authors

Please see the Weaknesses

Reviewer_wB48OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the problem of missing training labels in multi-dimensional classification (MDC) and proposes CSER, a probabilistic label recovery framework based on competitive co-evolution. The method first constructs two types of class-probability representations using instance features and cross-dimensional predictive information. It then employs competitive co-evolution to jointly optimize recovery strategies and pseudo-missing challenges, thereby learning dimension-specific probability fusion weights. Finally, the optimized strategy estimates the probability distributions of missing labels, and multiple imputation is used to train downstream classifiers. Experiments on 17 datasets under three label-missing rates demonstrate favorable performance compared with the evaluated baselines.

Critical strengths
  1. The paper addresses a practically relevant problem by considering incomplete training labels in MDC.
  2. The experiments are relatively comprehensive, covering 17 datasets and multiple label-missing rates.
Critical weaknesses
  1. The proposed method mainly combines existing techniques, including class-probability representations, confidence-aware fusion, competitive co-evolution, and multiple imputation. Although the authors provide specific designs for missing-label recovery, the overall methodological novelty appears somewhat incremental.

  2. According to Eq. (9)–(10) and Appendix A.3, for a fixed recovery-strategy population, the highest-fitness challenge can be obtained by ranking the average prediction errors within each class under the prescribed class-wise quotas. Similarly, for a fixed challenge population, the mean recovery loss can be optimized independently across dimensions. The paper lacks comparisons with these simpler optimization approaches, such as direct ranking and one-dimensional search, leaving the advantages of competitive co-evolution over simpler alternatives insufficiently demonstrated.

  3. Appendix D shows that CSER achieves relatively limited average performance improvements over CSER-E, despite introducing additional mechanisms such as competitive co-evolution, elite archives, and archive-based robust selection. The paper does not sufficiently justify the trade-off between the added algorithmic complexity and the resulting performance gains.

  4. The current ablation studies are insufficient to isolate the individual contributions of key components. In Appendix D, CSER-H simultaneously changes the probability representations, fusion mechanism, and label recovery approach, while CSER-E only uses a fixed non-evolutionary recovery rule for comparison. Furthermore, components such as confidence-aware fusion, archive-based robust selection, and multiple imputation lack individual ablation studies.

  5. The effectiveness of the proposed method is mainly evaluated through downstream MDC classification performance, without directly assessing the quality of missing-label recovery. The authors are encouraged to evaluate label recovery accuracy and probabilistic prediction quality using the ground-truth labels of artificially masked entries.

Small Issues:

  1. There are some issues with the citation formatting. The authors are encouraged to carefully check and revise the citations.
Questions for authors

Please address the above weaknesses.

Reviewer_k7m4OpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies multi-dimensional classification with incomplete training labels and proposes CSER. The method first constructs feature-only and dependency-conditioned probability representations through cross-fitting. It then combines the two representations using entropy-based confidence and jointly optimizes recovery strategies and pseudo-missing challenges through competitive co-evolution. The final strategy is used for multiple imputation and downstream classification. Experiments simulate 30%, 50%, and 70% label missingness on 17 nominal MDC datasets.

Critical strengths

1.The paper try to addresses multi-dimensional classification under incomplete supervision, a problem with some practical relevance. Probability representations, confidence-aware fusion, and multiple imputation are organized into a reasonably complete pipeline.

2.The experiments cover three label-missing rates and include ablation studies, runtime analysis, and an analysis of the evolutionary process. The implementation details are also relatively comprehensive.

Critical weaknesses
  1. Clarification on dataset provenance and representativeness. The paper currently reports only dataset sizes and feature counts. It would be helpful if the authors could further document the sources and acquisition periods of these datasets (as some appear to be traditional benchmarks from the 1980s–1990s). Incorporating or discussing more recent real-world datasets would further strengthen the claim regarding the method's applicability to current incomplete-supervision applications.

  2. Discussion on the relationship between original modalities and pre-processed features. We note that the experiments utilize pre-processed numerical features (e.g., geometric facial descriptors and acoustic statistics) derived from facial videos or speech. The authors might consider elaborating on why the evaluation is restricted to fixed tabular features, and how applying the method to raw, multi-modal data might present different challenges or limitations. This would help readers better understand the scope of applicability.

  3. Justification of the evolutionary strategy and the cost-performance trade-off. The competitive co-evolution strategy is currently used to optimize only q q continuous parameters (with q q ranging from 2 to 14). For such a relatively low-dimensional optimization problem, it would be more convincing to include comparisons with lightweight optimization methods (e.g., grid search, Bayesian optimization). Furthermore, given that the full CSER improves over CSER-E by only approximately 0.4–0.9 percentage points, a discussion on the trade-off between this marginal accuracy gain and the additional computational overhead of the complex evolutionary mechanism would be highly valuable.

  4. Exploring the gap between the pseudo-missing challenge and genuine missingness. The paper states that challenge entries are treated as missing only during fitness evaluation, without modifying the observation mask or triggering retraining. While cross-fitting effectively prevents label leakage, the authors could further discuss the extent to which this design mimics genuine information missingness. Additionally, clarifying the advantages of this challenge mechanism compared to random or fixed masking strategies would better support its design motivation.

  5. Ablation and direct evaluation of the label-recovery module. Since different baselines employ varying imputation methods, classifiers, and training protocols, it is currently difficult to fully isolate the independent contribution of the label-recovery module. The authors are encouraged to include a more controlled ablation study targeting this module. Moreover, directly reporting recovery metrics (such as recovery accuracy, Brier scores, or probability calibration) on the artificially hidden entries would provide more direct and compelling evidence for its effectiveness.

Questions for authors

see the weaknesses

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.