acceptodds
Withdrawn from ICLR 2027

Multiple Correlation Encoder-based Naive Bayes

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Naive Bayes (NB) continues to be one of the top 10 data mining algorithms. However, due to its assumption of attribute conditional independence, NB encounters significant challenges in addressing attribute-class correlations, attribute-attribute correlations, instance-class correlations, instance-instance correlations, and so on. In the last few decades, a large number of improved algorithms have been proposed, but none of them simultaneously addresses all these correlations. To bridge this gap, this paper proposes a novel algorithm called multiple correlation encoder-based naive Bayes (MCENB). In MCENB, we first design a multiple correlation encoder to generate new attributes, where multiple correlations are simultaneously captured and optimized. Specifically, the newly generated attributes are highly correlated with the class, yet uncorrelated with each other. Instances consisting of new attribute values are highly correlated with those in the same class. Subsequently, we augment original attributes by concatenating them with new attributes. Finally, we weight each augmented attribute to alleviate the attribute redundancy and then build NB on the weighted attributes. The experiments across numerous datasets show that MCENB significantly outperforms its benchmark competitors.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
Naive Bayes, Attribute generation, Attribute augmentation, Attribute weighting, Multiple correlation encoder

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_WN6JOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

This paper presents an auto-encoder based naïve Bayes algorithm, which leverages the reconstruction loss, the loss that maximizes the discriminant power and minimizes feature redundancy, and the loss that maximizes the feature similarity of samples in the same class while minimizes the feature similarity across all samples. A standard attribute weighting algorithm is then used to further weigh different attributes. While the authors claimed three contributions, but the core design is a so-called “multiple correlation encoder”, which is essentially an auto-encoder. The proposed MCENB is validated on 24 datasets and demonstrates superior performance.

Critical strengths

While there are critical flaws in the proposed method, at least the problem identified by the authors are valid and worth for investigation, though there are significant issues in the novelty and technical soundness of the proposed method.

Critical weaknesses
  1. The novelty is limited. The authors listed three main contributions, but these contributions are not accurately identified nor thoroughly justified. For example, the first claimed contribution is a well-known factor rather a novel design/finding. The second contribution claimed MCE as a new design, but without any justifications on what is exactly new in MCE. MCE is essentially a customized auto-encoder. The last contribution, MCENB, claimed the utilization of both generated attributes and original attributes as novel design, again lacking sufficient novelty.
  2. There are significant flaws in the proposed method. As shown in Fig. 2, the one-hot label encoding is appended at the end of embedding vector as the input. This may be fine for training but not applicable to inference, where no ground-truth labels are available! Neither the main manuscript nor the supplementary materials mentioned how to obtain the one-hot label encoding during inference. This would be potential label leakage.
  3. There are many unclear or unjustified design choices. a) Why choose MLP as encoder or decoder? Why three layers? Why use these numbers of neurons per layers? b) how to obtain Zprior\mathcal{Z}_{prior} in Eq. (3). c) Sign in Eq. (4) is important but the authors choose to discard it, why? Any ablation? d) In Eq. (5) and (6), the authors intentionally added a negative sign. But why not simply using the reciprocal of the ratio? Any ablation? e) In Eq. (7), there lacks a parameter to control the contribution of the two losses. Any ablation? f) There may be potential tensions among so many loss terms. Any mechanisms to resolves these tensions?
  4. The experimental validation is not sufficient. a) The datasets remain relatively small. The feature dimension is limited to 90; the number of classes is limited to 15, and the number of samples is limited to 19,000. The authors should validate the methods in more challenges scenarios with much higher feature dimensions, more number of classes and more samples. b) The compared methods are mostly old methods. There is only comparing method published in 2025. More recent and representative methods should be compared. c) With a close look at the comparison to the competitors in Tab. 2, the proposed method does not significantly outperform the compared methods. Most of the gain for the average accuracy comes from the movement libras dataset. If discarding this dataset, the gain over ICGNB is negligible. The W/T/L should be recalculated based on the statistical significance rather than simply comparing the average value.
  5. The writing of the manuscript needs significant improvement. a) Many statements are provided without references or justifications. b) Many sentences in the last paragraph on Page 1 are almost the same as in the abstract. c) The review is shallow and outdated. In most cases, the review only touches the definition of key categories, without highlighting the key technologies behind each method. d) Fig. 2 and Fig. 3 should be merged into Fig. 1, and the novel designs are not sufficiently highlighted in the block diagram, if there is any. e) There are many issues on symbols and definitions. For example, p(x∣z,v)p(\bm{x}|\bm{z},\bm{v}) is used to represent both the decoder and the output of the decoder (in Eq. (3)). f) Fig. 8 presents an ablation, but the two parameters are never defined in the methodology section.
Questions for authors
  1. The authors should better articulate their novel contributions. In the current form, this paper lacks novelties.
  2. The authors should clarify how to obtain the one-hot label encoding during inference.
  3. There are many unclear or unjustified the design choices. Please justify them.
  4. The current experimental validation is not convincing. The authors should: a) at least show W/T/L in terms of statistical significance rather than simply comparing the average. b) Comparison with more recent methods. 3) On more challenging datasets (larger datasets with more attributes, classes and samples).
Reviewer_MoJkOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes MCENB, a naive Bayes classifier that combines neural feature generation with attribute weighting. A multiple correlation encoder is trained to enhance feature–class relevance, reduce feature redundancy, and improve within-class instance similarity. The generated features are concatenated with the original features and classified using weighted Gaussian naive Bayes. Experiments demonstrate improvements over several existing naive Bayes variants.

Critical strengths
  1. The framework integrates attribute-level and instance-level correlation objectives into a unified feature learning approach.

  2. The experiments cover multiple datasets and include component ablations and synthetic analyses.

Critical weaknesses
  1. The proposed attribute correlation loss minimizes marginal Pearson correlations, whereas naive Bayes relies on conditional independence given the class. Reducing marginal correlation does not necessarily reduce conditional dependence, particularly when the original features are concatenated with their learned transformations. Moreover, Eq. (5) measures feature–class correlation using numerical class labels, making the objective potentially sensitive to arbitrary label encodings in multiclass problems. These issues weaken the theoretical justification for the proposed correlation optimization framework.

  2. The experiments mainly compare variants of naive Bayes, while the proposed method incorporates a nonlinear neural encoder trained with supervised information. Comparisons with representative tabular classifiers, such as MLP and XGBoost would help determine whether the improvements are competitive beyond the naive Bayes family. Furthermore, the instance correlation objective introduces pairwise computations with quadratic dependence on the number of training instances, yet no practical training-time or scalability comparisons are provided.

Questions for authors
  1. What motivates the hidden-layer width of (2m) and the choice to generate exactly (m) new attributes? Although Figure 8 examines the feature ratio on a synthetic dataset, could the authors clarify whether these architectural choices are robust across different real-world datasets?

  2. What justifies assigning equal weights to (L_E), (L_A), and (L_I)? While Figure 8 investigates the balance between reconstruction and correlation losses, the relative contributions of (L_A) and (L_I) are not separately examined.

  3. The accuracy differences between the full model and the loss ablation variants in Figure 5 are relatively small (0.47–1.32 percentage points). Have statistical significance tests been conducted to establish whether these improvements are consistent across datasets

Reviewer_bx9GOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
3
Soundness
1
Read review
Paper summary

This paper proposes a multiple correlation encoder-based Naive Bayes (MCENB) method that improves the classification performance of Naive Bayes by leveraging attribute-class, attribute-attribute, instance-class, and instance-instance relationships simultaneously. First, an encoder generates new attributes, and a decoder reconstructs the original input using the generated representations and one-hot class vectors. For representation learning, the paper proposes to use LEL_E--composed of reconstruction and KL terms--along with the attribute-perspective correlation loss LAL_A and the instance-perspective correlation loss LIL_I. It then combines the original attributes with the generated attributes and train attribute weights to maximize the conditional log-likelihood, thereby constructing a weighted Gaussian Naive Bayes model. The paper compares the performance of the proposed method with existing NB-based methods on 24 real-world datasets and validates the proposed method through component-wise ablation studies and analysis of synthetic data.

Critical strengths
  • In Table 2, MCENB shows a higher average accuracy than ICGNB, the most recent related method, with a win/tie/lose record of 16/1/7 across 24 datasets. The average accuracy is 80.08% for MCENB and 78.87% for ICGNB, and the Wilcoxon test in Table 3 also reports results favorable to MCENB. Therefore, this provides evidence that the proposed method has the potential to improve predictive performance compared to the NB-based methods included in the evaluation.

  • The ablation study in Figure 5 is helpful because it allows us to understand the extent to which each loss contributes.

Critical weaknesses
  • Most importantly, a significant portion of the proposed method -- combining original and generated features and optimizing the conditional log-likelihood of weighted GNB -- was adopted from ICGNB. Equations (12)-(19) in ICGNB and Equations (9)-(16) in this paper correspond almost exactly during the classification and weight learning stage. Therefore, the additional contributions of this paper are focused on the new encoder and the correlation objective function; A substantial portion of the proposed method is adopted from ICGNB, yet the manuscript presents these components as if they were introduced in this work. The authors should explicitly acknowledge that their approach builds on ICGNB and clearly distinguish the inherited components from the new contributions. Once these inherited components are accounted for, the remaining methodological novelty appears quite limited, raising concerns about whether the paper offers a sufficiently substantial advance over the prior work.

  • Minimizing the attribute loss in the form of a ratio LAL_A does not necessarily increase the attribute-class correlation. If the denominator decreases more rapidly, the loss can improve even if the attribute-class correlation of the numerator decreases. The suitability of this ratio for learning class-discriminative attributes therefore requires further justification. In particular, the authors should explain how the full training objective controls this trade-off and avoids favoring representations with reduced class relevance.

Questions for authors
  • Please address the concerns raised in the Weaknesses section.

  • Could the term ρ(bj;c)\rho(\mathbf{b}_j;\mathbf{c}) in Eq. (5) be a notational error? If c\mathbf{c} denotes the vector of scalar class codes, this Pearson correlation can change simply by reassigning the numerical codes to nominal classes, even though the underlying classification problem remains unchanged. Is this term intended to be computed using one-hot class representations? If so, could the authors revise Eq. (5) accordingly and clarify how the correlations with individual class indicators are aggregated?

  • Could the authors clarify how the instance-correlation loss LIL_I contributes to learning on datasets with only two input attributes, such as banana in Appendix C? Under the stated architecture, the embeddings would also be two-dimensional. However, Eq. (4) gives an absolute Pearson correlation of 1 for any pair of nonconstant two-dimensional vectors, so Eq. (6) reduces to the constant LI=−1L_I=-1, providing no gradient for learning instance relationships. Is this the behavior in the reported experiments, or does the implementation use a different embedding dimension or correlation computation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.