acceptodds
Under review as a conference paper at ICLR 2027

Operators Gate Gradients: Routing, Survival, and Reasoning-Shortcut Control in Differentiable Logic

Abstract

A logical constraint can make a network learn less than it would without it: on MNIST digit-sum with the Gödel or Goguen implication, concept accuracy stays near (chance ) through labels, below the supervision alone reaches with . We trace this to the operator that combines sub-predictions, usually chosen by convention. It is credit assignment on two axes: gradient must be *routed* to a parameter, as the computation graph dictates, and *survive*, staying nonzero where training operates, as the objective and grounding dictate. No operator is universally best, but the best is predictable. Fuzzy description logic makes this exact: one implication gates every axiom through closed-form partials, so the chain rule predicts where each operator acts. Across runs, the implication explains – of operator-induced variance once structure is fixed, against – for the t-norm, even under operator-independent scoring; the Łukasiewicz implication is the safe default for pure satisfaction until, as predicted, the objective or grounding changes. Beyond logic, the same axes predict seven of eight aggregation tasks, the one miss being prospective. The paradox is a survival failure. Gödel and Goguen *saturate*, going flat on satisfied constraints, so a wrong-concept solution satisfying them, a *reasoning shortcut*, is a zero-gradient optimum, walled off from supervision by the violated-constraint loss. Swapping in a non-saturated implication lifts the barrier without extra labels, matching supervision by to : the shortcut is a property of the loss, not a shortage of labels.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
neuro-symbolic learning, differentiable logic, fuzzy logic operators, reasoning shortcuts, gradient routing, description logic, aggregation operators, credit assignment

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_iyL3OpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper builds primarily on the work of van Krieken et al. (2022), who observed that incorporating fuzzy logical constraints does not necessarily improve neural network training performance on semi-supervised MNIST and attributed this phenomenon to the gradient properties of different implication operators, particularly the Gödel and Goguen implications. Building on this work, the present paper applies the known derivatives of these operators to seven ALC axiom structures. It proves that, for certain right-quantified axioms, the two operators yield identical gradients with respect to role parameters, while for certain left-quantified axioms, there is a quantifiable difference in their first-order descent behavior. The paper refers to this analysis as "Routing". Furthermore, building on prior research into gradients vanishing and operator-related issues associated with softmax outputs, the paper introduces a "Survival" analysis to investigate, from the perspectives of gradients and loss functions, why certain logical operators can cause models to converge to incorrect solutions. As a methodological improvement, the paper replaces logical operators that tend to trap models in incorrect solutions with existing operators that have better gradient properties.

Critical strengths
  1. The research problem is reasonable, and the paper generally provides a fair distinction between its own contributions and prior work's.
  2. The experiments are comprehensive, and the results provide a coherent account that supports the paper's claims.
Critical weaknesses
  1. The main shortcut mitigation experiments in Section 5 are directly based on Survival. The authors do not demonstrate a complete pipeline in which Routing is first used to analyze a new task, followed by automatic operator selection and successful shortcut mitigation. Routing appears to be an independent theoretical contribution, but it has not yet been integrated as an essential algorithmic step in the paper's shortcut mitigation experiments.
  2. I personally find the derivations of Theorems 2 and 3 relatively straightforward, as they can be derived directly from van Krieken et al. (2022).
  3. Prior work from 2022 has already demonstrated that replacing operators can improve training performance. The methodological contribution of this paper still primarily relies on manually selecting and replacing operators, which limits its novelty.
  4. Writing Problem: a) The abstract begins by presenting the authors' own experimental findings, which may give the misleading impression that these findings reflect the established state of research. b) The paper uses fancy emphasis on its theoretical framing, while some of the theorems are relatively straightforward. The overall presentation lacks clarity.
Questions for authors
  1. Under the same training data and computational budget, can replacing the Gödel/Goguen operators with alternatives such as KD achieve both higher concept accuracy and higher logical constraint satisfaction rates, compared to retaining the Gödel/Goguen operators and adjusting the weights of the logic loss?
  2. Compared with the operator gradient analysis presented by van Krieken et al. (2022), can the proposed Routing × Survival framework predict which operator will perform best on a completely new neuro-symbolic learning task before training?
6 comments

Thank you for the fair summary and for the two questions, which led to new experiments.

W1. Routing is not part of the shortcut pipeline.

Agreed for the submission. Routing and survival are two axes of one analysis, not two stages, and the paper proposes no automatic selection procedure; the abstract and Section 6 now say so. The revision adds experiments in which routing decides the outcome of the shortcut task:

TestRouting predictsResultWhere
Digit-sum, contrapositive formGödel sends no gradient to the digits, so the swap has no effectGödel 97.7% vs. supervision alone 97.8% (3,000 labels); 19.8% in the original formTable 15
MNIST-Addition, clause form, 10 labelsGödel and Goguen get no gradient at any weight50.4% at every weight, equal to supervision alone; KD, Łukasiewicz, Reichenbach 97.0–98.2%Table 16
MNIST-Addition, contrapositive formGödel and Goguen learn97.0–98.2%Table 16

W2. Thms. 2 and 3 (now 1 and 2) follow directly from van Krieken et al.

Their per-operator analysis does not contain the per-axiom statement: on which of the seven forms two implications give identical role gradients, and the exact difference on the others. This depends on where the role sits in the axiom and which quantifier branch is selected. We also report that Thm. 2 is first-order only: left-quantified axioms have no measurable effect on the trained gap (coefficient 0.000 [−0.021, 0.021]). → Section 3.1, App. H

W3. Operator replacement was already shown to help in 2022.

Yes; we do not claim the replacement itself. The additions are:

  • the failure is a reasoning shortcut: Goguen at 30,000 labels satisfies the rule on 99% of pairs with 19–21% concept accuracy (8 of 10 seeds) → App. L.3
  • more labels do not repair it at weight 10 → Fig. 4
  • the loss slope orders the weight each implication tolerates → Table 14

W4. Writing.

  • (a) The abstract now states that its opening result is a finding of this paper.
  • (b) Facts that follow directly from the derivatives are now plain text, not propositions.

Q1. With the same data and budget, does KD give higher concept accuracy and higher constraint satisfaction than Gödel/Goguen with a tuned weight?

Yes on both, by a small margin, with one exception (Goguen at 100 labels, where the intervals include zero). Each implication at its best weight, chosen on a held-out split; KD minus the other, in points, 95% paired intervals → App. L.1

LabelsConcept vs. GödelConcept vs. GoguenConstraint vs. GödelConstraint vs. Goguen
100+4.4 [3.4, 5.5]+0.4 [−0.3, 1.1]+6.0 [4.8, 7.1]+0.6 [−0.3, 1.6]
300+1.8 [1.4, 2.3]+0.9 [0.5, 1.3]+3.5 [2.7, 4.3]+1.7 [1.0, 2.5]
3,000+0.25 [0.12, 0.38]+0.24 [0.18, 0.30]+0.45 [0.21, 0.68]+0.53 [0.40, 0.66]

At 10 labels KD is 10.3 and 9.9 concept points above Gödel and Goguen. Tuning the weight removes most of the gap; Section 5 now reports this.

The gap is large only where the cause is routing: on MNIST-Addition in clause form, no weight helps Gödel or Goguen, and KD at its worst weight is 46.7 points [45.6, 47.7] above their best with 10 labels (8.7 [8.0, 9.5] with 100). → App. L.2

Q2. Can the framework predict the best operator on a completely new task before training?

No. It predicts which operators fail and which form of a rule trains; it does not rank the remaining operators. The abstract and Section 6 now say so.

To test what it does predict, we added MNIST-Max (two images, labeled with the larger digit) and wrote six predictions before any run → App. L.2

PredictionOutcome
Gödel and Goguen in clause form equal supervision aloneHeld: run for run (50.1% with 10 labels)
Łukasiewicz, KD, Reichenbach in clause form and all five in contrapositive form learnHeld: 97.8–97.9% with 10 labels, +47.7 points [46.7, 48.8]
These combinations coincide (two predictions: clause form among Łukasiewicz, KD, Reichenbach; contrapositive form among all five)Held only with labels
SP with 10 labels ends lowerHeld (65.7%), but it stayed above supervision alone, against our expectation
The Łukasiewicz t-norm without labels does not learnHeld (11.1%)

Overall: three held fully, two only with labels, one in part (Section 6).

Other predictions:

PredictionOutcomeWhere
Three new aggregation benchmarksCelebA and bAbI as predicted; ABSA againstTable 12
Tolerated weight ordered Gödel ≤ Goguen ≤ KDHeld on digit-sum, FashionMNIST, target sums 5 and 13App. L.1
Contrapositive encoding shifts the trained Łukasiewicz–Gödel gap in four quantified structuresHeld in two; within 0.004 in the other twoApp. E.2
Smoothing Gödel's branch indicator lets it escapeFailed: 13 of 15 runs leave the shortcut and all returnApp. L.3

Thanks for the additional experiments.

I agree that the revised experiments provide stronger evidence for the proposed analysis. However, the routing theorems remain fairly direct consequences of known operator derivatives. The weight-tuning results also show that much of KD's advantage over Gödel/Goguen disappears with proper tuning.

Also, the authors have clarified that the framework cannot predict the best operator or provide an automatic selection procedure. Given this narrower scope, I still find the methodological novelty limited.

I will keep my original score of 2.

Reviewer_WdXqOpenReview ↗
Rating
2
Confidence
1
Informativeness
1
Presentation
1
Soundness
2
Read review
Paper summary

This paper proposes a routing × survival framework to analyze how differentiable logic operators affect gradient propagation and model optimization. The framework explains operator behavior in terms of gradient routing determined by logical structure and gradient survival influenced by training objectives and grounding. The authors provide theoretical analyses and validate their findings through extensive experiments on synthetic and real-world ontologies. Furthermore, they investigate the reasoning shortcut problem in neuro-symbolic learning and demonstrate that changing the implication operator can mitigate such shortcuts without requiring additional labeled data.

Critical strengths
  1. The paper provides a systematic analysis of the strengths and limitations of different differentiable logic operators. And the extensive experiments on synthetic and real-world ontologies provide substantial empirical support for the analysis. It offers a more principled understanding of operator selection in neuro-symbolic learning.

  2. The paper provides an interesting analysis of the reasoning shortcut problem in neuro-symbolic learning. The authors explain why certain operators may lead models to satisfy logical constraints without learning the correct underlying concepts. This connects the theoretical analysis to a practical challenge in neuro-symbolic learning and strengthens the significance of the work.

Critical weaknesses
  1. The paper is difficult to follow due to its dense writing and scattered experimental results. The lack of a consolidated comparison table further makes it difficult to understand the key findings and differences between operators.
  2. The novelty of the routing × survival framework is not sufficiently established, as several key insights build on previously studied gradient propagation and saturation properties. Its theoretical contributions beyond existing work need clearer justification.
  3. Figure 3 is generated by AI mostly and not clear to see the details.
Questions for authors
  1. Can you list more examples to further illustrate how the logical constraint can make a network learn less than without it. The most neuro-symbolic methods (DeepProblog) actually improve the performance with logical constraints.
  2. What is the formal language with a certain expressivity that the proposed method can be applied to? 3.Since the gradient properties of fuzzy logic operators have been studied previously (§2.2), what are the key new theoretical insights provided by the routing × survival framework?
  3. Since reducing the logic-loss weight can also mitigate reasoning shortcuts, what are the advantages of operator swapping over simply adjusting the loss weight?
1 comment

Thank you for the review. Each point is answered in order.

W1. No consolidated comparison table. Added: Table 3 (Section 6) lists, for each condition, which implications work and which fail, with the key number and its section.

W2 / second part of Q2. What is new relative to earlier gradient analyses? Earlier work (van Krieken et al., 2022) describes each operator's derivatives. We show when those properties matter:

  1. Labels. With Gödel or Goguen at weight 10, concept accuracy stays at 15–20% up to 30,000 labels, below supervision alone with 10 labels (33%). → Section 5
  2. Loss weight. Each implication tolerates a different logic-loss weight, in an order predicted from its loss slope. → Table 14
  3. Rule form. Whether a rule trains the concepts depends on how it is written. → Table 16
  4. Structure. Two theorems state on which axiom forms two implications give the same role gradients. → Section 3.1

Table 17 compares the paper with each related work.

W3. Figure 3. Redrawn from code as a vector figure.

Q1. More examples where the constraint makes the network learn less, and DeepProbLog.

Concept accuracy with the rule vs. supervision alone:

TaskWith the ruleSupervision aloneWhere
Digit-sum, 300 labels, weight 10Gödel 19.6%, Goguen 18.6%92.7%Table 14
Digit-sum, 3,000 labels, weight 10Gödel 20.1%, Goguen 18.7%97.9%Table 14
Digit-sum, 100 labels, weight 10KD 21.7%84.6%Table 14
Target sums 5 and 13, 600 labels, weight 10KD 71% and 89%94.7%App. L.1
FashionMNIST, 300 labels, weight above the tolerated oneGödel and Goguen 18–26%76.7%App. L.1
MNIST-Addition, 10 labels, SP9.7–16.3%50.4%Table 16

The rule also helps in many cases: on MNIST-Addition with 10 labels it lifts KD, Łukasiewicz and Reichenbach from 50.4% to 97.0–98.2% (Table 16).

There is no conflict with DeepProbLog. DeepProbLog uses probabilistic semantics and has no fuzzy implication, so saturation and blocked branches do not arise. On its own benchmark, three fuzzy implications (KD, Łukasiewicz, Reichenbach) in clause form have the loss 1−P(sum)1-P(\text{sum}), one minus DeepProbLog's query probability, and they learn. → Section 5

Q2. Which formal language? The theorems cover ALC\mathcal{ALC} TBoxes in the seven normal forms of Table 1, which include EL\mathcal{EL}. The MNIST experiments use first-order rules in the style of Logic Tensor Networks. Number restrictions and transitive roles are not covered. → Section 2, App. N

Q3. Why swap the operator instead of lowering the weight?

Lowering the weight removes most of the harm; Section 5 now reports this. Swapping still has three advantages:

  1. No search. The tolerated weight differs by a factor of 100 between implications (300 labels: Gödel 0.1, KD 10) and changes with the label budget (Gödel: none at 100 labels, 0.1 at 300, 0.3 at 3,000). → Table 14
  2. A small, consistent gain at the best weights (KD minus the other, points, 95% paired intervals) → App. L.1
LabelsConcept vs. GödelConcept vs. GoguenConstraint vs. GödelConstraint vs. Goguen
100+4.4 [3.4, 5.5]+0.4 [−0.3, 1.1]+6.0 [4.8, 7.1]+0.6 [−0.3, 1.6]
300+1.8 [1.4, 2.3]+0.9 [0.5, 1.3]+3.5 [2.7, 4.3]+1.7 [1.0, 2.5]
3,000+0.25 [0.12, 0.38]+0.24 [0.18, 0.30]+0.45 [0.21, 0.68]+0.53 [0.40, 0.66]
  1. The rule stays useful. At 300 labels and its best weight, KD is 1.4 points [1.1, 1.8] above supervision alone; Gödel is 0.4 below [−0.7, −0.1]. → App. L.1

Limits. KD is not exempt: with 100 labels it tolerates weight 1, and on FashionMNIST and target sums 5 and 13 it tolerates weight 3.

Reviewer_TPFhOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper studies how operator choice affects gradient propagation in differentiable logic. It separates gradient routing from gradient survival, derives role-gradient comparisons for normalized ALC axioms, and tests operator choices on ontology benchmarks, MNIST reasoning-shortcut tasks, and several aggregation tasks. The main claim is that operator derivatives, together with structural and grounding conditions, can explain failures and help select operators.

Critical strengths
  1. Separating gradient routing from gradient survival is a useful way to think about operator choice. The motivation for doing so is clear.
  2. The experiments cover several different settings. The ontology benchmarks, MNIST reasoning-shortcut tasks, and aggregation tasks give a broader view of operator behavior than a single-task evaluation.
  3. The study includes useful controls. Fixed-evaluator comparisons help separate training-operator effects from evaluation choices, while logic-loss-weight sweeps show how sensitive the results are to loss balancing.
  4. The paper reports failed predictions and limitations. I appreciate that the authors include cases that do not support their predictions; these help readers judge the scope of the proposed explanation.
Critical weaknesses
  1. The label-budget result depends on loss weighting and the training setup. The 30,000-versus-10-label comparison does not isolate the effect of the constraints. Figure 4 includes matched-budget baselines, but both failed operators recover at a logic-loss weight of 0.1 (Section 5; Appendix O). Since the supervised loss is a batch mean, more labels do not proportionally increase its weight relative to the logic loss. The evidence therefore supports failure under the chosen training setup, not operator saturation as the sole explanation.
  2. Two theoretical claims need clarification. Under Theorem 3, the role-gradient quantity gLg_L must be zero when its defining active set is empty, yet Appendix I.2 reports positive norms attributed to concept pathways. The measured quantity and the 91.7% lower-bound pass rate need clarification. Section 3 also claims unchanged KD branch reachability, but at (u,v)=(0.8,0.6)(u,v)=(0.8,0.6), adding λmax⁡(0,u−v)\lambda\max(0,u-v) changes the gradient from (0,−1)(0,-1) to (λ,−1−λ)(\lambda,-1-\lambda) for λ>0\lambda>0, activating the previously inactive antecedent branch.
  3. Several evaluation choices limit the conclusions. Section 4.3 uses the highest test accuracy reached during training (Appendix K); validation-selected or fixed-checkpoint results are needed. The Łukasiewicz/Gödel ordering reverses with the evaluator, so the default-operator recommendation and switching precision of 0.994 need an explicit evaluation setting. Within-active gradient correlations are only 0.04 and −0.08, offering limited support for magnitude prediction. Confidence intervals containing zero do not establish equivalence with the LLM baselines. The seven-of-eight predictive score also mixes evidence types, whereas only two of three prospective tests succeed (Table 3). Part of the near-zero-loss ranking is attributed to the soft surrogate, limiting its use as evidence of an intrinsic operator ordering (Appendix I.4).
Questions for authors
  1. Regarding the label-budget experiment: Since both failed operators recover at lower logic-loss weights and the supervised loss is a batch mean, what does the label-budget sweep establish beyond failure in the chosen training setup? How do the matched-budget and loss-weight controls separate the proposed operator mechanism from loss-balancing effects?
  2. Regarding the theoretical definitions and their validation: Which parameter subspace and exact quantity were measured as gLg_L in Appendix I.2? Why is it nonzero when its defining active set is empty? Did all cases that failed the lower bound satisfy the theorem assumptions? How is the hybrid-loss reachability claim consistent with the nonzero antecedent gradient introduced by the margin term?
  3. Regarding checkpoint selection and the scope of the conclusions: Can the authors provide validation-selected or fixed-checkpoint results for Section 4.3? Please also specify the fixed evaluation metric under which the safe-default recommendation and switching threshold are intended to hold. Which main conclusions remain after distinguishing peak from sustained performance and separating the different types of evidence in the predictive scorecard?
3 comments

Thank you for the careful review. W1 and W3 led to new experiments, and W2 exposed an error that we corrected.

W1 / Q1. What does the label sweep establish beyond failure in the chosen setup?

By itself, only failure at weight 10. You are right that a batch-mean supervised loss does not grow with the number of labels; Section 5 now says so. The weight grid is what separates the operator from loss balancing. Final-epoch concept accuracy (%), 10 seeds → Table 14

LabelsSupervision aloneGödel, w = 10 / 3 / 1 / 0.3 / 0.1Goguen, w = 10 / 3 / 1 / 0.3 / 0.1KD, w = 10 / 3 / 1 / 0.3 / 0.1
10084.619.2 / 19.7 / 21.3 / 35.4 / 80.419.0 / 23.7 / 37.8 / 80.2 / 84.521.7 / 56.8 / 83.4 / 84.9 / 84.8
30092.719.6 / 20.6 / 26.5 / 88.7 / 92.318.6 / 31.9 / 92.2 / 93.3 / 93.194.2 / 93.7 / 93.0 / 92.8 / 93.0
3,00097.920.1 / 20.0 / 31.4 / 96.4 / 97.918.7 / 33.3 / 97.8 / 98.0 / 97.998.2 / 97.9 / 97.9 / 97.9 / 97.9

Three observations separate the two effects:

  1. Every implication is released by a low enough weight, KD included (100 labels, weight 10: 21.7%). Saturation is therefore not the sole explanation, and the paper no longer says it is.
  2. The weight each implication tolerates is ordered by the operator. Largest weight such that it and all lower ones stay within 3 points of supervision alone (n/a = not run) → Table 14, App. L.1
Task (labels)GödelGoguenŁukReiKD
digit-sum (100)none0.10.311
digit-sum (300)0.113310
digit-sum (3,000)0.31n/an/a10
FashionMNIST (300)0.31n/an/a3
target sums 5 and 13 (600)0.11n/an/a3

This is the order of the slopes of each implication's loss on violated constraints (Prop. 11, App. L.3). It holds at cutoffs of 1, 2 and 5 points and with validation-selected checkpoints.

  1. Where the cause is routing, no weight helps. MNIST-Addition, clause form, 10 labels: Gödel and Goguen are at 50.4% at weights 0.1, 1 and 10, equal to supervision alone; KD at its worst weight is 46.7 points [45.6, 47.7] above their best. → Table 16, App. L.2

With each implication at its best weight, KD stays ahead, by a small margin (KD minus the other, concept accuracy points, 95% paired intervals) → App. L.1

LabelsKD − GödelKD − Goguen
100+4.4 [3.4, 5.5]+0.4 [−0.3, 1.1]
300+1.8 [1.4, 2.3]+0.9 [0.5, 1.3]
3,000+0.25 [0.12, 0.38]+0.24 [0.18, 0.30]

Revised claim. Saturation makes the shortcut a zero-gradient optimum; the weighted loss on violated constraints decides whether a run stays there. This also covers Łukasiewicz, which is saturated yet escapes in 4 to 7 of 10 seeds from 1,200 labels. → Section 5

W2 / Q2. What was measured as gLg^L, and why was it nonzero on an empty active set?

You are right; the measurement was wrong. The submission measured a soft-gated, batch-weighted gradient, which is nonzero without active pairs. We replaced it with the theorem's quantity, computed with hard operators in three independent ways on 513 real and 60 controlled ontologies → App. E.2

CheckResult
The three computations agreeall but 3 of 573; those 3 sit at a tie u=vu=v, which the theorem excludes
No active left-quantified pairgL=0g^L = 0 exactly, 429 of 513 real ontologies
Lower bound, with active pairsholds on all 144 (84 real, 60 controlled); one within float32 rounding (relative 6×10−86\times10^{-8})
The five earlier failuresall five satisfy the bound with the theorem's quantity
  • Did the failures satisfy the assumptions? Yes. They failed because of the wrong quantity, not because of the theorem.
  • Within-active correlations (0.04 and −0.08). These came from the same wrong quantity. With gLg^L they are 0.98, 0.97 and 0.99 at initialization. We do not count this as independent support, because the bound and gLg^L use the same pairs. Along the training trajectory the correlation is 0.07 and 0.24, not significant. We no longer use Thm. 2 to predict trained magnitudes.
  • KD and the hybrid loss. You are right: the margin term gives the antecedent a gradient that KD alone lacks. Section 3.1 now says the margin term reopens the starved branch.

W3 / Q3.

(a) Best-epoch selection. Section 4.3 rerun with a held-out selection split, concept accuracy (%) → App. G

SPReichenbachŁukasiewicz
validation-selected90.274.736.5
final epoch79.146.429.4

The order is unchanged, but Reichenbach does not sustain its peak, and the text now says so. Fig. 4 and all tables of Section 5 are final-epoch.

(b) Evaluation setting of the default recommendation. The reversal you point to is real and remains in the paper: under the four fuzzy evaluators other than its own, Łukasiewicz-trained models score below Gödel-trained ones (0 wins / 3 ties / 17 losses under the Gödel evaluator), and Gödel has the best held-out AUC. → App. E.2. The recommendation is now stated for three settings:

SettingResultWhere
Self-scored satisfactionswitching pays only above precision p∗≈0.994p^*\approx 0.994Table 9
One crisp evaluator (round at 0.5, check each axiom classically)Łukasiewicz highest; +0.022 [0.003, 0.040] over GödelTable 8
Held-out MRR, fixed evaluatorŁukasiewicz highest (0.542); p∗=0.515p^* = 0.515; online policy precision 0.309; offline classifier 13 of 17 switches correct, mean gain 0.0004Section 4.4, Table 10

(c) LLM baselines. "Statistically tied" is removed. Every interval contains zero, every point estimate favors the LLM, and the text now says the data do not establish equivalence. → App. J.2

(d) Predictive scorecard. Counted by type: all three predictions derived from Prop. 9 hold, and two of the three for new benchmarks hold (CelebA, bAbI; ABSA fails). → Table 12

(e) Near-zero-loss ranking. You are right. KD's last place is an effect of the soft surrogate (zero under hard scoring in all 120 runs). The main text keeps only what holds under both scorings: Reichenbach reaches exactly zero. → App. E.4

(f) What remains after separating peak from sustained performance. Remains: Gödel and Goguen at 15–20% at weight 10 for every label budget; the order of tolerated weights; the routing results on MNIST-Addition. Weakened: Reichenbach's and Łukasiewicz's peaks are not sustained. Withdrawn: the τ\tau-intervention (13 of 15 runs leave the shortcut and all return). → App. L.3

Thank you for the detailed response and additional experiments. The corrected gradient measurement, held-out checkpoint selection, and revised statistical wording address several of my original concerns. However, important limitations remain. The weight sweep supports operator-dependent sensitivity to loss weighting, while the advantages after tuning are sometimes small or statistically uncertain; the stronger routing results remain specific to the tested configuration. The corrected measurements support consistency with the theorem, but the weak, nonsignificant correlations along training leave its explanatory value for training dynamics limited. Furthermore, the preferred operator depends on the evaluation metric, and the reported switching strategy provides only modest practical gains. The gap between peak and final-epoch performance also limits claims about sustained optimization benefits. These revisions improve the paper’s accuracy and clarify its scope, but the remaining evidence does not sufficiently strengthen the practical contribution or the connection between theory and observed training behavior to warrant a higher assessment. I therefore maintain my original score.

Reviewer_L88SOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2(+1)
Soundness
2
Read review
Paper summary

This paper studies fuzzy logic (specifically ALC), which is used in neurosymbolic learning as a differentiable loss. The fuzzy relaxation requires a choice of logical operations (disjunction, conjunction, and implication). The authors argue that in particular, the implication can be problematic during training due to vanishing / zero gradients. This problem is classfied into "routing" (which parameters sit on a path with a nonzero partial derivative), and "survival" (which partial derivative is nonzero during actual training). The papers provides several theorems on the survival and routing behaviour for the different implication operators. Experimentally, this is verified on ontology datasets and the MNIST-addition experiment.

Critical strengths
  • The paper contains extensive details about the theoretical and experimental setting, including about negative results (e.g in Table 6).
Critical weaknesses

Novelty & Significance.

The theory contributions seems pretty slim and I find the theorems to be quite straightforward. For example, Prop. 1 is just the chain rule, and Prop. 4 is already well-known. Stripped from the all the new vocabulary (such as “routing” and “survival”), the paper’s main argument is that the parameters need a non-zero gradient during training to learn. This seems a very basic observation and somewhat tautological to me. The split in “routing” (is the gradient zero by construction?) or “survival” (is it zero in the particular region of training?). It is not clear to me how novel this is compared to the prior literature, van Krieken et al (2022) already explore vanishing and single-passing gradients in fuzzy logic.

Presentation.

In my opinion, the presentation of the paper is subpar, and needs substitutional improvement before acceptance. For instance, the preliminaries section does not actual cover the setup, and already assumes the reader knows e.g. what a TBox is. The actual background is covered in Appendix A, but in my opinion a paper should be readable without the appendix.

More generally, the heavy use of LLM-style writing and non-standard terminology make the paper hard to parse and the story of the paper gets somewhat lost due to this. For example, the experimental starts with very dense experimental details, but doesn’t really explain what the research questions are what is even really tested. For this reason, it’s also hard to really know what the conclusions are from this paper.

The appendix is also very long at 60 pages. Besides the obvious sloppy AI-generated artifacts (e.g. ”The main ranking conclusions remain visible in a reviewer-friendly format”), its structure is confused and should be cleaned up considerably.

Questions for authors
  • The evaluation of a fuzzy loss depends on not only the chosen operators, but also on the logical formulation. The paper argues that the implication is more important compared to the other operators, but this seems very dependent on the chosen formula normalization (c.f. Table 1)? For example, how do the MNIST digit-sum results change with different encodings? (And what is the used encoding, I couldn't find this anywhere.)
  • Why are the relaxations of the quantifiers in Table 1 fixed to min/max? I suppose they could be picked similar to the t-(co-)norms?
  • “Because T and S appear only inside u and v while I sits at the top level, implication is the only operator that directly gates the loss gradient” This can also happend for disjunctions/conjunctions, right (e.g. using the drastic or godel t-norm)?
2 comments

Thank you for the detailed review. We answer novelty and presentation here and Q1–Q3 in the second part.

Novelty. The revision now separates what is known from what is new.

  1. The chain-rule decomposition and saturation follow from known derivatives. The former Prop. 1 and Prop. 4, the softmax dead zone, and the stationarity of shortcuts are now plain text with attribution; only the two structural theorems remain formal results. → Section 3
  2. van Krieken et al. (2022) already contain the properties our rule-form results use: the modus ponens / modus tollens analysis, contrapositive differentiable symmetry (their Def. 8, Section 8.2), R-implications vanishing where the antecedent is at most the consequent (Section 8.2), and that the Gödel implication never makes a modus tollens correction (Section 8.3). We now say so. → Sections 1, 2.2, 3.1
  3. Thm. 2 is first-order at a fixed point and has no measurable trained effect on real ontologies. The regression coefficient of left-quantified axioms on the trained gap is 0.000 [−0.021, 0.021]. The trained gap comes from disjointness axioms (+0.093 on 157 ontologies, +0.005 on the other 356), that is, from Gödel's blocked antecedent, which is their property. → App. H

What we claim beyond their analysis. Routing and survival correspond to their single-passing and vanishing. Our contribution is to make each conditional (on the axiom or rule form; on the objective, grounding and loss weight) and to test the resulting predictions:

ClaimEvidenceWhere
With a saturated implication at weight 10, more labels do not release the shortcutGödel and Goguen: 15–20% concept accuracy from 10 to 30,000 labels; supervision alone reaches 33% with 10Section 5, Fig. 4
The slope of the loss on violated constraints orders the logic-loss weight each implication tolerates300 labels: Gödel 0.1, Goguen 1, Łukasiewicz 3, Reichenbach 3, KD 10; Gödel ≤ Goguen ≤ KD also on FashionMNIST and target sums 5 and 13Prop. 11, Table 14, App. L.1
On real ontologies, changing the form of one loss term closes the gap69 BioPortal ontologies, crisp evaluator: Gödel 0.953 → 0.973; Łukasiewicz 0.975Table 8
The best implication changes with supervisionMNIST-Addition, SP: 97.6% with no labels, 9.7–16.3% with 10 labels; when the labels arrive later, all 36 runs that enter at ≥ 95.0% end at 96.1–97.9%Table 16, App. L.2
Per axiom form, where the Łukasiewicz and Gödel role gradients coincide (right-quantified) and by how much they differ (left-quantified)Thms. 1 and 2; identity checked with hard operators on 573 ontologiesApp. E.2

These predictions can be read before training, and they held; the ones that failed are listed in Table 5.

Presentation.

  • Preliminaries. Section 2 now defines concepts, roles, TBoxes and the fuzzy grounding before Table 1.
  • Research questions. Section 4 now opens with the three questions it tests.
  • MNIST setup and encoding. → Section 5 (rule), App. L.1 (setup)
  • Appendix. Reordered to follow the main text and cut to 31 pages. The sentence you quote is removed. All negative results are kept.
  • LLM-style writing. We rewrote the main text and the appendix. Our use of LLM assistants is disclosed in the AI Use Statement; the authors verified every claim and number.

Q1a. Which encoding is used? I(A,C)I(A, C) with A=Sum9(x,y)A=\mathrm{Sum}_9(x,y) and C=⋁idi(x)∧d9−i(y)C=\bigvee_i d_i(x)\wedge d_{9-i}(y), as in van Krieken et al. → Section 5, App. L.1

Q1b. How do the digit-sum results change with the encoding? They change, and in the direction the partials give. Final-epoch concept accuracy (%) at 300 / 3,000 labels, weight 10, 5 seeds; supervision alone 93.1 / 97.8 → Table 15

EncodingGödelGoguenKDSPŁuk
original I(A,C)I(A,C)19.6 / 19.819.0 / 19.794.6 / 98.264.8 / 98.321.1 / 82.3
contrapositive I(¬C,¬A)I(\neg C,\neg A)93.4 / 97.721.7 / 98.194.5 / 98.265.1 / 98.121.5 / 82.2
digit-wise ∀i ⁣:I(T(A,di(x)),d9−i(y))\forall i\colon I(T(A,d_i(x)),d_{9-i}(y))21.1 / 71.114.7 / 36.468.5 / 95.257.0 / 89.137.4 / 93.5
curried ∀i ⁣:I(A,I(di(x),d9−i(y)))\forall i\colon I(A,I(d_i(x),d_{9-i}(y)))22.9 / 68.715.2 / 25.572.6 / 97.459.3 / 89.544.0 / 92.2
  • Contrapositive. Gödel sends gradient only to ¬A\neg A and none to the digits, so the rule neither helps nor harms: it matches supervision alone (paired difference +0.33 [−0.23, 0.89] at 300 labels).
  • Digit-level encodings. Every implication ends below supervision alone; Gödel and Goguen are below KD in all 8 cells. The trapped runs satisfy only 10–54% of the constraint, so they are not shortcuts. The shortcut of Section 5 is specific to the original encoding, and the paper now says so.

Q1c. Is the dominance of the implication an artifact of the normal form? It is conditional on the grounding, and the paper now states this. Under Table 1 one implication sits on top of every axiom, which is why it dominates. If an axiom is written as ⊤⊑¬C⊔D\top\sqsubseteq\neg C\sqcup D, the top-level operator is a t-conorm, which induces an S-implication, and the same analysis applies to it. → Section 3.1

For the contrapositive encoding of ontologies we tested this directly: Thms. 1 and 2 swap roles for Gödel (at initialization, on all 144 ontologies with active left-quantified pairs and all 323 with active right-quantified pairs), and over 900 training runs the encoding × implication interaction explains at most 0.2% of the variance. → App. E.2

Q2. Why are the quantifiers fixed to max/min? They need not be. We fix them in the ontology experiments because max/min is the standard fuzzy DL semantics and the theorems need an aggregator that selects one branch. Two checks without max/min:

  • Ontologies → App. D.5. Temperature-smoothed and power-mean aggregators keep the per-structure ranking close to the hard one (ρ = 0.900 to 1.0; first place changes between Łukasiewicz and Goguen in two of four structures). Log-mean-exp fails to train on three of four structures.
  • MNIST. All shortcut experiments use the benchmark's own aggregators, the mean of logarithms for ∀ and a generalized mean (p = 1.5) for ∃, with the Yager t-norm. → App. L.1

Q3. Can a t-norm or t-conorm also gate the gradient? Yes. The sentence was wrong and is corrected. The drastic t-norm blocks a path (Fig. 13). The claim is now: under the grounding of Table 1, the implication is the only operator on every axiom's gradient path. → Section 3.1

Comments on the paper

1 comment

We thank the four reviewers. The revised PDF is uploaded; blue marks substantive changes. This note lists what changed. Each reviewer's points are answered in the individual replies.

Corrections

PointRaised byChangeWhere
The validation of the left-quantified theorem measured a soft-gated, batch-weighted gradient, not the theorem's gLg^LTPFh W2Recomputed with hard operators on 573 ontologiesApp. E.2
Accuracies were taken at the best test epochTPFh W3Now final-epoch or validation-selected. The τ\tau-intervention then gives only transient escapes; that claim is withdrawnFig. 4, Table 14, App. L.3
The headline result depends on the logic-loss weightTPFh W1, WdXq Q3, iyL3 Q1Stated as conditional on weight 10; full weight grid addedSection 5, Table 14
"The only operator that gates the gradient"; KD branch reachability under the hybrid lossL88S Q3, TPFh W2Both sentences correctedSection 3.1
Chain-rule facts were presented as propositionsL88S, iyL3 W2 and W4Now plain text with attribution; the two structural theorems (Thms. 1 and 2, formerly 2 and 3) remainSection 3
"Statistically tied" with the LLM baselines; "seven of eight" predictionsTPFh W3Removed; predictions are counted by typeApp. J.2, Table 12
The trained Łukasiewicz–Gödel gap was attributed to left-quantified axiomsfound by usIt comes from disjointness axioms: +0.093 on the 157 ontologies that have them, +0.005 on the other 356App. H

New experiments

ExperimentResultWhere
MNIST-Addition: 3 rule forms × 6 implications × weights {0.1, 1, 10}, 10 seeds (1,320 runs)Clause form, 10 labels: Gödel and Goguen 50.4% at every weight, equal to supervision alone; KD, Łukasiewicz, Reichenbach 97.0–98.2%. Contrapositive form: Gödel and Goguen 97.0–98.2%Table 16
Logic-loss weight grid on digit-sum (640 + 200 runs), FashionMNIST, target sums 5 and 13Tolerated weight at 300 labels: Gödel 0.1, Goguen 1, Łukasiewicz 3, Reichenbach 3, KD 10Table 14, App. L.1
Three alternative encodings of the digit-sum rule, 5 seedsContrapositive: Gödel 93.4 / 97.7 at 300 / 3,000 labels; supervision alone 93.1 / 97.8Table 15
Section 4.3 with validation-selected checkpointsSP 90.2, Reichenbach 74.7, Łukasiewicz 36.5 (final epoch: 79.1, 46.4, 29.4)App. G
69 BioPortal ontologies with disjointness axioms, retrained, one crisp evaluator, 2 seedsŁukasiewicz 0.975, Gödel 0.953; Gödel with only its disjointness term in contrapositive form 0.973Table 8
Contrapositive encoding on ontologies (900 training runs)Encoding × implication interaction explains at most 0.2% of the varianceApp. E.2
MNIST-Max, predictions written before training (400 runs)Of six predictions, three held fully, two only with labels, one in partApp. L.2, Section 6

What stands. At logic-loss weight 10, Gödel and Goguen stay at 15–20% concept accuracy from 10 to 30,000 labels, below the 33% of supervision alone with 10 labels; KD reaches 94.2% at 300 labels (Section 5, Table 14). Within structure, the implication explains 73–81% of operator-induced variance and the t-norm 5–6% (Table 7).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.