- Rating
- 2
- Confidence
- 4
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper brings shifted Łukasiewicz unbound logic, a recent logic from the mathematical fuzzy logic community whose truth values range over all of R, into differentiable ILP. The first contribution is LuHorn. LuHorn forward chaining scores conjunction, rule aggregation, step aggregation and existential quantification all as sums, and uses a hard min-based gate to decide whether a ground rule fires. Theorem 4.1, proved in Appendix A, shows that the Boolean projection of this procedure recovers the classical least Herbrand model. The second contribution is DiffLu. It replaces each rule with a nonnegative weight vector over a fixed set of first-order features, folds the gate into those weights, iterates the summed update, and trains with a clipped hinge-like loss on the fixed-point valuations. Symbolic rules are recovered afterwards with a ratio-based top-K cutoff. The third contribution is vDiffLu, which puts slot attention and a bilinear fact converter in front of DiffLu and trains everything end to end with no object-level annotations. On the 17 ∂ILP benchmark tasks, DiffLu reaches a mean success rate of 84.1 against 77.6 for HRI. On Kandinsky and CLEVR-Hans3, vDiffLu averages 87.64 and 95.22, compared with 93.72 and 97.38 for αILP, which uses pretrained perception.
Critical strengths
-
A clean correctness result for a real-valued forward-chaining procedure. Appendix A splits rule applicability, the min gate, from value propagation, the additive payload, and proves through Lemmas A.3 to A.5 and Proposition A.6 that the Boolean projection catches up with classical TP within two updates and converges to LM(P). I went through the induction, the delayed-activation counterexample on p. 17, lines 876 to 882, and the termination argument, and found no gaps. This matters because it shows unbound Łukasiewicz conjunction can be reconciled with Horn semantics, which a reader from the fuzzy logic side would otherwise doubt, given that additive conjunction does not preserve falsity, p. 15, lines 756 to 762.
-
End-to-end visual ILP with no intermediate annotations, at competitive accuracy on CLEVR-Hans3. vDiffLu trains slot attention from scratch together with the rule learner, p. 7, lines 357 to 359, and comes within about 2 points of αILP on average, 95.22 against 97.38, while using about 5 percent of αILP's memory according to p. 9, line 469. Removing the need for object-level supervision is the main practical barrier for αILP and NEUMANN, so a credible result here would matter to the neuro-symbolic vision community.
Critical weaknesses
-
The mechanism the paper claims is never isolated, and the theory does not cover the trained model. The abstract and introduction motivate everything with local gradient pathologies: vanishing gradients for product operators and selective routing for Gödel and Łukasiewicz, p. 1, lines 045 to 050. The only support offered is the analytic gradient column of Table 1, p. 2. Gradient 1 for a sum is true by construction and says nothing about whether training improves. The paper itself concedes on p. 20, lines 1043 to 1045, that global gradient issues are left open, but even the local claim is untested. There is no ablation that keeps the DiffLu architecture, features and loss fixed and swaps the sum aggregators for max or product, no gradient norm diagnostics, and no comparison of training curves. On top of that, Theorem 4.1 applies to Eq. 5 with its hard gate, while DiffLu in Eq. 7 states that "The gates in Eq. 5 are absorbed into the learnable weights", p. 6, line 319. The authors concede on p. 20, lines 1047 to 1051, that the relaxation "may be not well-formed". Without the gate, each head is a nonnegative linear combination of feature valuations summed over every grounding of the existential variables. That makes it a counting threshold function, not a conjunction, and the paper never explains how it avoids being swamped by the many false groundings at value -1 or how valuations stay bounded over recursive steps. As things stand, the contributions are a correct theorem about a procedure nobody trains and an empirical model whose benefits are not traced to the claimed cause.
-
The empirical advantages are narrower than the abstract claims, and the visual comparison is not reliable. Table 2 on p. 8: DiffLu averages 84.1, ∂ILP 83.7 and LRI 81.2. I recomputed all six averages from the per-task entries and they match. The "6.5 percentage points" headline on p. 8, line 428, and p. 9, lines 478 to 479, only holds once ∂ILP and LRI are excluded as "not directly comparable". Against ∂ILP, the margin is 0.4 points over 17 tasks with "at least 10 random seeds", p. 8, line 423. DiffLu scores 0 on Cyclic and Len, where ∂ILP scores 100 and 92.5, and gains mainly on Adjacent, EvenSucc, Buzz and Fizz, by 49.5, 51.5, 55 and 30 points. That is a trade, not the dominance that "superior performance" in the abstract, line 018, suggests. DiffLu also drops auxiliary predicates and uses longer bodies, p. 4, footnote 3, and p. 20, lines 1057 to 1061, so the comparison with ∂ILP and HRI mixes language bias with the operator choice. The MSE criterion on p. 8, lines 424 to 427, follows the ∂ILP convention and is applied to the clipped continuous valuations, so Table 2 says nothing about whether the extracted symbolic programs are correct, and no learned rules are shown. DiffLu's hyperparameters, meaning V, R, the number of steps and τ, are never reported, even though p. 3, lines 130 to 131, says they are tuned per task. In Table 3, p. 9, the rerun NEUMANN numbers are at or near chance on red triangle and two pairs, 54.68 and 51.21, and have a standard deviation of 24.69 on closeby. That suggests a failed reproduction, which weakens any claim to be comparable to NEUMANN. αILP shows a standard deviation of 0.00 on every Kandinsky task. vDiffLu trails αILP by 37 points on twopairs. On CLEVR-Hans3, vDiffLu is run as "three binary classification problems", p. 9, line 457, while the CNN baseline does "3-way classification", p. 19, line 1017, so at least that comparison mixes two task formats. The confounder-free test split is skipped, p. 9, lines 458 to 459, so results come from validation data that carry the same confounders as training. CLEVR-Hans3 exists precisely to test that split, and for a model whose slots and predicates have no supervised meaning, p. 21, lines 1082 to 1084, high validation accuracy could easily come from shortcuts. On related work, Logical Neural Networks, which use weighted Łukasiewicz operators, dNL-ILP by Payani and Fekri 2019, Neural Logic Machines, Logic Tensor Networks, DRUM and TensorLog are all missing, and Payani and Fekri appear only through their 2020 reinforcement learning paper, as are end-to-end perception-plus-logic systems such as DeepProbLog and Scallop. These belong in the paper, since sum-based path aggregation and Łukasiewicz-style differentiable rules both have precedent there.
Questions for authors
- Can you add a controlled ablation: the same DiffLu features, loss, initialization and seeds, with sum replaced by max for S, R and ∃, and with product or Łukasiewicz conjunction, on all 17 tasks? Please include gradient norm statistics per layer and step. A clear gap would go a long way toward supporting the paper's thesis.
- Is success in Table 2 computed by running classical forward chaining on the extracted symbolic program, or on the continuous DiffLu valuations? If it is the latter, what are the success rates for the extracted programs, and can you list the learned rules for Fizz, Buzz and EvenSucc?
- Please report V, R, the number of forward steps, τ, the optimizer and the tuning budget for each task, and explain how valuations are kept from blowing up under summed recursive aggregation without the gate.
- The NEUMANN reruns are near chance on two Kandinsky tasks. How do your αILP and NEUMANN reruns compare with the results published in Shindo et al. 2023 and 2024, and which Kandinsky split does Table 3 report? Would you also evaluate on the CLEVR-Hans3 test split and in the same 3-way setting as the CNN baseline?
- On p. 9, line 469, the memory figure is about 5 percent of αILP. Can you give a measured table of peak memory and wall clock for each method?