- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
The paper addresses the problem of explainability of logic gate networks, a recent type of neural architectures where neurons are encoded as hard logic gates, with continuous relaxations employed at training time for differentiability. The authors present explainability in LGN as a formal reasoning task, showing how to extract compact, guaranteed abductive explanations efficiently.
Critical strengths
The paper is very well written and clear. It is grounded on automated reasoning methodologies that provide sound formal guarantees to the explainability approach being presented.
Critical weaknesses
Encoding LGN in SAT is not novel, this was already done in Kresse et al 2025 (as the authors clarify in the introduction). Explanation of LGN was also addressed in the same paper, albeit at a global model-level scale. This work contributes instance-level explanations.
The authors argue that LGN can be useful not only for efficiency but also for interpretability. Being them based on hard logic gates, I find this statement rather unsurprising. Indeed, Kresse et al already proposed explainability of LGN, albeit for model level explanations. The novelty wrt model-level explanation techniques is not clarified.
There is no attempt at inspecting the explanations produced by the approach, e.g. in terms of interpretability for humans. Their utility thus remains unclear.
Questions for authors
It would be useful to have a clearer picture of how different the machinery for instance-level explanations is wrt what was already done for global level ones.
2 comments
I would like to thank the authors for their rebuttal.
I'm still not entirely convinced about the stated novelty of the contributions. Convolutional binarized networks have been encoded in SAT before (see e.g. Truth Table Net: Scalable, Compact & Verifiable Neural Networks with a Dual Convolutional Small Boolean Circuit Networks Form). The SAT version of voting is a standard cardinality encoding (used by Kresse et al too), while MaxSAT version explicitly follows Ignatiev et al. 2022 (and it's slower on every benchmark).
While I take the point of the authors that LGN are not interpretable by default, the interpretability of the explanations extracted is still to be verified. The fact that they are on average smaller that those extracted from ReLU nets is positive, but insufficient I'm afraid to claim interpretability
Rebuttal by Authors
Thank you for raising concerns about the novelty and utility of our encoding and explanations. Kresse et al. [1] takes a model f and a perturbation distance ϵ, outputting the minimum confidence κ that any instance must achieve to be ϵ-robust for f. For instances which achieve κ confidence, a set of distance-restricted explanations [2] may be derived. However, the potential to explain instances with less than κ confidence is limited. Our approach unconditionally explains any instance with no distance restriction.
On the encoding side, we make three novel contributions. We encode convolution and pooling layers in SAT for the first time, we provide two representations of the voting mechanism, and we propose constraint-based LGN simplification.
Besides aiming for a different problem, being unable to encode convolutional and pooling layers, Kresse et al. makes use only of a cardinality constraint based formulation for the voting mechanism.
To our best knowledge, the other two contributions have never been done before.
Furthermore, Kresse et al. is scope limited to simple datasets and networks, with more advanced tasks marked for future work. As we contribute formal and tractable explanability for MNIST and Fashion-MNIST, our work addresses this identified research direction.
Finally, LGNs themselves are not interpretable by default. They most often represent intricate interactions between a large number of logic gates, which is similar to "intrinsically interpretable" ML architectures. These are difficult to interpret without computing an explanation as we do [3].
[1] Fabian Kresse, et al. Logic gate neural networks are good for verification, PMLR 2025.
[2] Yacine Izza, et al. Distance-restricted explanations: theoretical underpinnings & efficient implementation, KR 2024.
[3] Joao Marques-Silva and Alexey Ignatiev. No silver bullet: interpretable ML models must be explained, FRAI 2023.