acceptodds
Under review as a conference paper at ICLR 2027

Confidence Neural Logic Networks

Abstract

Neural networks achieve strong predictive performance, but their decisions are often opaque, which is problematic in domains that require explicit, human-understandable reasoning. Neural rule learners address this by learning logical rules from data, yet they typically aggregate rules through a linear layer that has no formal logical meaning, leaving unclear what reasoning the trained network performs. We introduce Confidence Neural Logic Networks (CNLNet) and prove that conjunctive clauses followed by a linear layer exactly realize a weighted logic program: the forward pass performs weighted-satisfiability inference, and the predicted class provably coincides with the one favored by the program. Clause structure is learned end-to-end with differentiable gates. We extend this correspondence to depth with the Confidence Neural Logic Block, whose hidden units are themselves weighted-satisfiability decisions, so that deep networks remain layered logic programs, and we compose blocks into class-conditional chains that avoid gradient interference across classes. On tabular and image benchmarks with up to M samples, CNLNet achieves a better average rank than interpretable baselines, including RRL (outperformed on datasets), C4.5, SATNet, and NeuroLogic, and is comparable to strong black-box models such as PLNN, LightGBM, and XGBoost.

Primary area:
interpretability and explainable AI
Keywords:
interpretable machine learning, neuro-symbolic learning, differentiable rule learning, logic programs

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_6SDvOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper builds classifiers with a weighted-rule interpretation and extends them to deeper networks with separate chains for each class. It evaluates accuracy, program size, and inference time on 18 datasets.

Critical strengths

I like that the rules belong to the model making the prediction, rather than being fitted afterward to imitate it. The experiments cover a good range of datasets, and the size and runtime results help show what using these models would cost.

Critical weaknesses
  1. The checkpoint selection uses validation data, and Table 4 gives both variants separately. My concern is with Table 1 taking the better shallow/deep test result for each dataset. What would the result be if that choice were made on validation data? The pruning results have a similar limitation: they are clearly labeled as oracle results, but do not show how much compression a validation-selected pruning level would give.

  2. Tables 8–9 do compare shared and separate chains, but I could not find a comparison with matched parameter counts. Separate chains also give the model more capacity. Without controlling for that, I am not sure how much of the gain comes from reducing gradient interference.

Questions for authors
  1. What results do you get when both the shallow/deep choice and pruning level are selected on validation data?
  2. Do class-specific chains still help against a shared model with a comparable parameter count?
3 comments

W1 / Q1. Results when the Shallow/Deep choice and the pruning level are selected on validation?

Both are now validation-selected.

Shallow/Deep: Table 1 changes on three datasets (magic 84.80→84.4184.80\to84.41, wine 96.68→96.1496.68\to96.14, dota2 59.43→59.3659.43\to59.36); average rank 4.50→4.564.50\to4.56, still 4th of 12.

Pruning (Table 5: Before / After (val) / After (oracle)): Shallow shrinks by 1919--68%68\% on 12 of 18 datasets (banknote 254.6→82.4254.6\to82.4; adult 198.6→160.0198.6\to160.0 vs. 110.4110.4 oracle), Deep by 99--81%81\%. Test F1F_1 drops by ≤0.4\leq0.4pp on 17/18 datasets (Shallow) and 15/18 (Deep); the exceptions have tiny validation splits (wine, 15 samples: −5.6-5.6 / −2.6-2.6; tic-tac-toe Deep −1.5-1.5; car Deep −0.9-0.9).

DatasetShallow BeforeShallow After (val)Shallow After (oracle)Deep BeforeDeep After (val)Deep After (oracle)RRLSATNetC4.5
adult198.6 ± 15.6160.0 ± 36.6110.4 ± 24.3636.6 ± 137.0566.4 ± 119.7390.6 ± 187.312.2 ± 1.0260.0 ± 0.05500.0 ± 70.1
bank-marketing418.4 ± 32.5323.6 ± 69.7218.0 ± 55.2503.6 ± 3.4397.8 ± 51.3394.8 ± 69.510.2 ± 0.7176.0 ± 0.04361.6 ± 65.1
banknote254.6 ± 2.382.4 ± 33.380.6 ± 34.3229.2 ± 26.362.8 ± 64.059.4 ± 60.019.4 ± 2.576.0 ± 0.027.6 ± 1.0
car91.6 ± 14.580.0 ± 22.277.2 ± 20.8351.4 ± 19.773.4 ± 17.1104.8 ± 26.621.8 ± 1.048.0 ± 0.0111.2 ± 3.2
chess1482.4 ± 4.81453.2 ± 19.21396.4 ± 63.26609.6 ± 37.95375.2 ± 240.55117.6 ± 678.177.4 ± 1.0116.0 ± 0.07199.2 ± 41.2
connect-41491.6 ± 1.91340.2 ± 153.11347.6 ± 205.01379.6 ± 20.4659.6 ± 210.7748.8 ± 315.631.2 ± 1.7256.0 ± 0.013959.8 ± 269.1
letRecog1496.8 ± 3.71402.0 ± 82.81330.6 ± 59.013390.4 ± 189.22590.4 ± 1335.13858.8 ± 988.8181.2 ± 1.5248.0 ± 0.02065.4 ± 18.7
magic441.0 ± 8.7271.4 ± 37.1183.6 ± 69.8508.8 ± 4.4460.4 ± 38.1212.8 ± 61.613.2 ± 0.4184.0 ± 0.02741.2 ± 19.7
tic-tac-toe72.8 ± 4.745.8 ± 18.936.8 ± 12.2192.0 ± 0.066.4 ± 37.393.0 ± 22.619.4 ± 0.556.0 ± 0.080.4 ± 7.7
wine64.0 ± 0.021.4 ± 10.524.4 ± 9.7325.0 ± 20.2157.6 ± 95.5119.4 ± 76.310.2 ± 2.2240.0 ± 0.010.2 ± 0.4
activity646.8 ± 37.5504.2 ± 100.5534.6 ± 56.3189.4 ± 68.971.4 ± 8.5102.8 ± 56.047.8 ± 3.51752.0 ± 0.0282.4 ± 9.6
dota2199.6 ± 4.5143.2 ± 23.5138.2 ± 31.6714.6 ± 36.8216.8 ± 100.446.4 ± 26.32.2 ± 0.7792.0 ± 0.021279.0 ± 236.0
facebook85.4 ± 9.853.4 ± 18.350.4 ± 12.8831.2 ± 875.4384.0 ± 351.5248.6 ± 161.052.4 ± 7.61652.0 ± 0.02834.2 ± 18.4
fashion2000.0 ± 0.01921.4 ± 73.71918.8 ± 91.59853.8 ± 30.54267.4 ± 1390.24032.4 ± 472.354.2 ± 1.21588.0 ± 0.08105.4 ± 21.6
porto-seguro127.2 ± 3.585.2 ± 27.194.0 ± 23.1245.2 ± 76.5143.8 ± 92.174.4 ± 42.3127.8 ± 0.4184.0 ± 0.033527.2 ± 1487.8
forest-cover496.8 ± 4.0400.2 ± 38.5394.4 ± 20.93424.8 ± 38.01868.0 ± 414.61879.0 ± 428.3128.0 ± 0.0280.0 ± 0.053949.4 ± 331.1
credit-fraud14.2 ± 1.213.2 ± 1.611.8 ± 2.6184.4 ± 14.069.8 ± 38.655.4 ± 21.463.6 ± 0.8544.0 ± 0.0292.8 ± 15.9
higgs218.8 ± 3.5171.0 ± 23.3172.0 ± 22.1491.8 ± 9.4334.2 ± 101.4337.0 ± 107.3127.0 ± 0.9224.0 ± 0.01535650.0 ± 15585.1

W2 / Q2. Do class-specific chains still help against a parameter-matched shared model?

Added (Table 9: a shared chain with clause counts scaled by ≈C\approx C to match the parameter count). With matched parameters, they help on chess and letRecog, and tie on car. Claims in Sec. 3.4 and 5 are revised accordingly.

DatasetCShallowSharedShared (param.-matched)Class-cond.
car494.81 ± 5.5098.55 ± 1.7599.11 ± 1.2598.71 ± 1.17
chess1883.47 ± 0.9467.09 ± 0.7263.04 ± 1.0286.54 ± 0.87
letRecog2694.86 ± 0.3291.70 ± 0.8491.07 ± 0.4392.87 ± 0.40
#params (car/chess/letRecog)---35K / 160K / 579K139K / 2.75M / 14.6M138K / 2.75M / 14.6M

Thanks for new experiments which address some of my concerns. I raised my score accordingly.

Reviewer_dGFEOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

In this paper, the authors propose the Confidence Neural Logic Network, a network that automatically learns logic rules over binarized input for classification tasks. In its shallow version, it is composed of two layers: a layer of AND nodes that put the input literals in conjunction with an L0 gate, and a subsequent linear layer that weights the importance of each clause for each class. The authors show this network is equivalent to a weighted logic program. In its deep version, it composes several shallow blocks with batch normalization and residual OR. It uses separate chains for predicting in each class to avoid gradient interference. Experiments on multiple datasets show that CNLNet ranks better than comparable interpretable solutions (RRL, C4.5, etc.) on average.

Critical strengths
  • Novelty and Potential Impact. The paper proposes a novel neural rule learner whose output layer is equivalent to a weighted logic program. As it can be effectively composed through blocks, it creates an end-to-end neural model that can be read as a composition of logic rules by design, without requiring post-hoc analysis. Also, due to the equivalence of the learnt logic rules to a neural network structure, they can be executed as matrix multiplication on GPU with higher speed than several of the compared methods (up to two orders of magnitude faster than RRL on complex problems). Its capacity to learn logic rules while passing the gradients could have an impact beyond the tabular domain, as concept-based approaches are also learning logic programs on top of binary concepts.
  • Wide Evaluation and Multiple Ablation. The paper benchmarks its approach over 18 datasets and compares 11 baseline models; it also reports ablation studies on BN, residuals, depth, and width of the network. Negative results openly declared: the model collapse with 2 blocks on the activity dataset and RRL providing more compact rule sets on most datasets.
Critical weaknesses
  • Experimental Protocol and Model Selection. The main model selections are performed on the test set: in particular, the reported column in Tab. 1 (which is the maximum between the shallow and the deep version of the model) and the level of pruning. Some dataset-specific components were used (residual dropout only on porto-seguro Deep, a learned projection residual on activity and facebook, focal loss on two datasets, etc.) as well as parameters (lr, λ\lambda, budget, etc.) without explaining how they were selected, since the validation split is reported only for checkpoint selection. On the other hand, no hyperparameter tuning is reported for the baselines, and this seems to have affected their predictions, as on porto-seguro several models report the same score, which is the one of a constant predictor (49.07). Finally, no standard deviation is reported in Tab. 1, and the statistical test is not specified.
  • Rule dimensions not compared against model accuracy. Although the authors report in the appendix a table comparing the dimensions of the rule sets (i.e., the number of different clauses), it is more important to report the rule dimensions (i.e., the average number of literals used for each prediction, which actually reports the effort for humans to understand the model decisions). Possibly, comparing it against the model accuracy in the main text (e.g., as an accuracy-vs-size Pareto front) would allow one to understand whether the accuracy gain is worth the higher effort for understanding the model. To get some space, I think that the ablation paragraphs on depth, width and sensitivity could be summarized.
Questions for authors
  1. The extracted program keeps literal i in clause j iff zij>0z_{ij} > 0 (l.214), while at evaluation rjr_j is computed only over literals with ∣v~ij∣≥ϵ|\tilde v_{ij}| \geq \epsilon (l.228). Which criterion defines the extracted program? With the first one, literals with 0<∣v~ij∣<ϵ0 < |\tilde v_{ij}| < \epsilon would appear in the program but be ignored by the network.
  2. The proof of Theorem 3.1 (App. A) only uses Ij∈0,1I_j \in {0,1} and the one-hot structure of yy, not the conjunctive form of sjs_j, so it seems to apply to any linear layer over Boolean features (e.g., the output layer of RRL, or logistic regression on binarized inputs). What property distinguishes CNLNet from these models?
  3. How were the per-dataset hyperparameters and components of CNLNet (Table 2) selected, given that the validation split is used only for checkpoint selection (l.846)? What tuning was performed for the baselines? In particular, why is the RRL clone far below the published RRL results (e.g., chess 45.93 vs 78.83, connect-4 57.23 vs 71.23)? Were the hyperparameters recommended by its authors used?
  4. The model comparison is ample enough but is there a reason why the methods described as closest in Sec. 2 (DR-Net, RL-Net, Neural Rule Lists, HyperLogic, difflogic) are not included in the comparison?
  5. Lemma 3.1 assumes uncorrelated class gradients: how does it explain the collapse of the shared chain on chess (l.432–437)? Is the shared chain in Table 8 parameter-matched to the class-conditional chains, or does it use fewer parameters and possibly provide fewer clauses?
  6. (Suggestion) Confidence weights are regularized with an L1 penalty (l.348) and then pruned post hoc. Have the authors considered applying L0 gates to the confidence weights as well, which would yield exact sparsity during training instead of post-hoc pruning?
3 comments

W1/Q3. Test-set model selection; unexplained hyperparameter and component choices; untuned baselines; no std or stated test. Why is RRL far below its published results (chess 45.9345.93 vs. 78.8378.83, connect-4 57.2357.23 vs. 71.2371.23)?

Selection. The Shallow/Deep choice (higher 5-fold mean validation F1F_1) and the pruning level (per fold) are now chosen on validation (App.C). Table 1 changes on three datasets (magic 84.80→84.4184.80\to84.41, wine 96.68→96.1496.68\to96.14, dota2 59.43→59.3659.43\to59.36); average rank 4.50→4.564.50\to4.56, still 4th. With the validation-selected pruning level, Shallow shrinks by 1919--68%68\% on 12 of 18 datasets and Deep by 99--81%81\%, with test F1F_1 dropping by ≤0.4\leq0.4pp on 17/18 (Shallow) and 15/18 (Deep) datasets (Table 5).

Hyperparameters. For 14 datasets, Table 2 holds the winners of a grid sweep (2525 Shallow / 3030 Deep configs) selected on validation F1F_1; λL0=0.1\lambda_{L_0}{=}0.1 for all. The four large Kaggle datasets were tuned by hand, also on validation F1F_1 only. Focal loss (porto-seguro, credit-fraud) and residual dropout (porto-seguro Deep) target extreme imbalance where the base model collapses to the majority class; the learned projection on activity/facebook is forced by memory (H≥N≈5000H\geq N\approx5000 is infeasible). Now stated in App.C.

Baselines. sklearn-family models use library defaults with balanced class weights; PLNN and SATNet use their reference defaults (now stated in App. C.). RRL and NeuroLogic-MLP do predict the majority class on porto-seguro (49.0749.07), which App. C now notes.

RRL. The public code has no per-dataset configs (only a tic-tac-toe demo) and selects checkpoints on a 5% validation split. We re-tuned it inside our protocol following its README (1 layer, width {512,1024512,1024}, temp. {1,0.1,0.011,0.1,0.01}, lr {2⋅10−3,2⋅10−42\cdot10^{-3},2\cdot10^{-4}}, selected on validation). The gap to the published numbers remains, likely due to unreported protocol details; conclusions are unchanged.

Std and tests. Table 1 now reports std. The ∗^* marks now use the corrected resampled tt-test instead of an uncorrected one: CNLNet beats \emph{all} four interpretable baselines on 6 datasets. App.~C.6 adds Friedman (p=1.2×10−11p{=}1.2\times10^{-11}), Nemenyi and Wilcoxon--Holm tests: CNLNet is significantly better than RRL, C4.5, LR, SVM and NeuroLogic-rules, and not significantly different from PLNN, RF, LightGBM, XGBoost.

W2/Q6. Report literals per prediction against accuracy (Pareto front). Q6: L0L_0 gates on the confidence weights?

Rule length (W2). New App. D.1 (Table 6, Fig. 3) reports literals in the fired rules and the size of a sufficient (abductive) explanation, with accuracy--length plots. The result is unfavourable and reported as such: RRL's explanations are shorter than Shallow's on most datasets (adult 2424 vs. 633633; chess 376376 vs. 28722872), C4.5's paths are shortest except on dota2, and Shallow is shorter than RRL only on porto-seguro, credit-fraud and higgs. The cause is many low-confidence clauses; sufficient explanations remove most of them on binary tasks (adult 633→147633\to147; porto-seguro 298→14298\to14, below RRL's 2121) but not on many-class tasks (chess 2872→25282872\to2528). No model dominates: C4.5 is the shortest, and CNLNet is the most accurate. Summarized in the main text and Limitations; the full table and figure are in App. D.1.

Clause-level L0L_0 (Q6). Implemented, thank you. A Hard Concrete gate on each clause's confidence vector removes whole clauses (the L1L_1 term never reaches exact zero under Adam). We fine-tune the trained Shallow model and pick λ\lambda as the largest value keeping validation F1F_1 within 11pp. On 88 of the 99 datasets where a λ\lambda qualifies, explanations shrink 22--8×8\times at ≤0.9\leq0.9pp F1F_1 cost (connect-4: −2.0-2.0), e.g. banknote 545→72545\to72, magic 657→98657\to98, higgs 350→74350\to74. This makes Shallow shorter than RRL on car (1515 vs. 2222) and widens its lead on forest-cover and higgs (74 vs. 570), but RRL stays shorter elsewhere; shortening the clauses themselves is future work.

Q2. Theorem 3.1 applies to any linear layer over Boolean features (RRL's output, LR). What distinguishes CNLNet?

Agreed: the theorem needs only exact Boolean features under a linear layer, and we now say so. The difference lies in the features: exact, L0L_0-gated conjunctions, and the same reading for every hidden unit (Prop. 3.1), so the whole network is a layered weighted logic program. In the revised paper, we've presented the theorem as the property the architecture is built to satisfy, not as a deep result.

Q4. Why not compare with DR-Net, RL-Net, Neural Rule Lists, HyperLogic, difflogic?

Half of our 18 datasets are multi-class, and the five methods differ in scope:

  • DR-Net and HyperLogic learn rule sets for binary tasks only (HyperLogic builds on DR-Net and reports +0.2+0.2--1.21.2pp accuracy over it on four binary datasets, without comparing to RRL). In an independent benchmark of 22 binary datasets (NeuRules, Xu et al., NeurIPS 2025, Table~1), DR-Net is far below RRL (average F1 0.560.56 vs. 0.750.75), so RRL is the stronger rule-set baseline.
  • difflogic learns networks of two-input logic gates, not rule sets or weighted rules, and is compared only with a decision tree, LR, and an MLP on tabular data.
  • RL-Net and Neural Rule Lists learn ordered rule lists, a different model class. In the same benchmark, they score above RRL (average F1 0.770.77 and 0.790.79 vs. 0.750.75; binary tasks, multi-class only against RL-Net and CLASSY).

Due to the time required for hyperparameter tuning, we will add Neural Rule Lists (and RL-Net) and their results once available.

Q5. How does Lemma 3.1 (uncorrelated gradients) explain the chess collapse? Is the shared chain parameter-matched?

Lemma 3.1 shows that the shared-parameter gradient is the sum of class-specific gradients, δ=∑c=1Cδc\delta=\sum_{c=1}^{C}\delta_c. Under the assumptions of uncorrelated gradients and equal expected squared norms, we have E∣δ∣2=Cσ2E|\delta|^2=C\sigma^2. Therefore, the aggregate gradient norm scales as Cσ\sqrt{C}\sigma, while each class contributes only σ\sigma, or 1/C1/\sqrt{C} of the aggregate RMS norm. For chess with 18 classes, this proportion is approximately 0.236. Consequently, as the number of classes increases, each class-specific learning signal has a progressively smaller contribution to updates of the shared parameters, making it harder for a single shared chain to accommodate all class-specific objectives. This provides a theoretical interpretation of the observed collapse on chess. In contrast, class-conditional chains isolate the parameter updates for each class, avoiding this cross-class gradient aggregation. Added (Table 9: a shared chain with clause counts scaled by ≈C\approx C to match the parameter count). Despite parameter matching, class-conditional chains outperform shared chains on chess (86.54 vs. 63.04) and letRecog (92.87 vs. 91.07), while on car the parameter-matched shared chain slightly outperforms class-conditional chains (99.11 vs. 98.71). These results suggest that avoiding cross-class interference is particularly beneficial on chess and letRecog, beyond simply increasing parameter capacity.

DatasetCShallowSharedShared (param.-matched)Class-cond.
car494.81 ± 5.5098.55 ± 1.7599.11 ± 1.2598.71 ± 1.17
chess1883.47 ± 0.9467.09 ± 0.7263.04 ± 1.0286.54 ± 0.87
letRecog2694.86 ± 0.3291.70 ± 0.8491.07 ± 0.4392.87 ± 0.40
#params (car/chess/letRecog)---35K / 160K / 579K139K / 2.75M / 14.6M138K / 2.75M / 14.6M

Q1. Extraction keeps literal ii iff zij>0z_{ij}>0 (l.214), evaluation only if ∣v~ij∣≥ϵ|\tilde v_{ij}|\geq\epsilon (l.228). Which defines the program?

∣v~ij∣≥ϵ|\tilde v_{ij}|\geq\epsilon, which makes aj(x)=I[x⊨sj]a_j(\mathbf{x})=\mathbb{I}[\mathbf{x}\models s_j] exact; l.214 is corrected. The criteria differ on only 1313 of 529,490529{,}490 literals (0.002%0.002\%) across all 9090 shallow models.

Reviewer_dGFEOpenReview ↗

Post author rebuttal

I thank the authors for the detailed response and the additional experiments. Selecting the Shallow/Deep variant and the pruning level on validation, adding standard deviations with a corrected statistical test, and the new App. D.1 address several of my concerns. Two points remain open after reading the revised paper:

  • Baselines. CNLNet's hyperparameters are selected from 25–30 configurations per dataset, while all other baselines use library or reference defaults with no per-dataset tuning (App. C.4). The claim that CNLNet is on par with the black-box models is weakened considering this issue. Moreover, the RRL re-tuning described in the response is not reported in the paper, and the RRL results in Table 1 are identical to the original submission on all 18 datasets (e.g., chess 45.93, connect-4 57.23), still far below the published ones.

  • Interpretability. The new App. D.1 shows that RRL's explanations are shorter than Shallow CNLNet's on most datasets, often by an order of magnitude (e.g., adult 24 vs. 633 literals, chess 376 vs. 2872). Deep CNLNet's explanations are longer still, up to 123,050 literals on letRecog. Even with the clause-level L0L_0 gate, RRL remains shorter on most datasets. This result is not reported in the main text, which still describes the model as "fully interpretable" (l.389). In light of these numbers, the proposed models are arguably not very interpretable.

Overall, the revision improves the evaluation protocol, but the new comparison weakens the interpretability claim, which is the main motivation of the paper. For this reason, I keep my score.

Reviewer_etPoOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes Confidence Neural Logic Networks (CNLNet) for learning weighted-satisfiability inference with a differentiable objective. On the one hand, this work proves the equivalence between conjunctive clauses followed by a linear layer and a weighted logic program. On the other hand, this work proposes a training paradigm for stacking structure, resulting in Deep CNLNet. Extensive experiments show that CNLNet outperforms interpretable baselines and achieves competitive performance against black-box baselines.

Critical strengths

None

Critical weaknesses

The contributions do not seem important enough.

Regarding Exact semantics: Theorem 3.1 seems correct but straightforward. In particular, if I understand correctly, the proved equivalence seems independent of conjunctive clauses. Theorem 3.1 only requires that the truth values of the rule bodies (which are conjunctive in this work) are Boolean values. Hence, the proof seems straightforward, and I did not find an interesting part.

Regarding Deep CNLNet: Using Deep Learning techniques to extend shallow CNLNet to multiple layers is very important and helpful. However, this work only experiments with at most two hidden blocks. First, this is not deep enough (for example, a two-layer MLP is not deep neural network). Second, since Deep CNLNet cannot outperform SOTA black-box baselines, why not extend it to deeper structures? This raises questions about the practical usage of Deep CNLNet in deeper scenarios.

Questions for authors
  1. If replacing the conjunctive rule bodies with any interpretable or non-interpretable structures that output Boolean values, does Theorem 3.1 still hold?
  2. Can you experiment with deeper CNLNets with three or more blocks?
1 comment

W1. The contributions do not seem important enough.

We believe the significance lies in bridging neural learning and exact logical reasoning without sacrificing scalability. Specifically: (i) CNLNet makes the trained network itself a weighted logic program, eliminating post-hoc rule extraction while enabling exact MAP inference in one forward pass (2–390× faster than RRL on CPU). (ii) This property extends naturally to stacked blocks, with every hidden unit remaining a weighted-SAT decision (Prop. 3.1). (iii) Across 18 datasets, CNLNet significantly outperforms all interpretable baselines (Wilcoxon–Holm: RRL p=0.001, C4.5 p=0.017, LR p=0.038), while matching PLNN, RF, LightGBM, and XGBoost (new App. C.6).

W2/Q1. Theorem 3.1 is straightforward and independent of the conjunctive form; does it hold for any Boolean-valued rule bodies?

Yes; it needs only exact {0,1}\{0,1\} bodies under a linear layer, and a new remark after Thm.3.1 now says so. Its role is not technical depth but faithfulness: the trained classifier is the program read from its weights. The conjunctive form matters for interpretability: the L0L_0-gated AND node with the ϵ\epsilon-threshold (Eq.5) makes each atom an exact, human-readable conjunction, and Prop.3.1 extends the equivalence to every hidden unit. Opaque Boolean features would give a valid but uninterpretable program.

W3/Q2. Only up to two hidden blocks; can you try three or more?

We extended the depth ablation to three hidden blocks on chess and forest-cover, the two datasets where increasing depth was most promising (Table 11). The results reveal dataset-dependent effects: on chess, performance largely saturates after one hidden block, with only marginal fluctuations as depth increases (86.54 →\to 85.51 →\to 85.83). In contrast, forest-cover continues to benefit from additional depth, improving consistently from 71.07 to 73.15 and then to 78.95. These results suggest that deeper architectures provide limited benefits once the model has sufficient capacity for a dataset, whereas datasets with more complex patterns can benefit from additional representational depth.

DatasetDepth 0 (shallow)Depth 1Depth 2Depth 3
banknote99.56 ± 0.2897.57 ± 1.0598.38 ± 0.44--
chess83.47 ± 0.9486.54 ± 0.8785.51 ± 1.1985.83 ± 1.49
activity95.22 ± 0.4696.74 ± 0.4863.69 ± 6.87--
forest-cover67.46 ± 0.6171.07 ± 2.6173.15 ± 0.3978.95 ± 1.15

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.