acceptodds
Withdrawn from ICLR 2027

Causal Models of Convolutional Binary Spiking Neural Networks for Explanation

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

We propose a novel logic-based causal approach to explaining convolutional binary spiking neural networks (C-BSNNs). Specifically, we provide an SMT (Satisfiability Modulo Theories) encoding of the binary causal model (BCM) underlying a C-BSNN and use it to compute formally correct abductive explanations for its outputs. We demonstrate that our SMT-based causal analysis extends beyond static inputs, preserving the recurrent temporal dependencies of spiking dynamics to support formal explanations of complex event-based DVS-Gesture data alongside static MNIST. To enable training of our networks with Batch Normalization while preserving the Boolean activations of neuronal units required for mapping them to BCMs and subsequent SMT encoding, we introduce a novel Batch Normalization absorption procedure. Finally, we compare our logic-based causal approach with SHAP with respect to the relevance of the elements included in an explanation. While our approach provides a formal guarantee that all such elements are relevant, SHAP provides no such guarantee.

Primary area:
interpretability and explainable AI
Keywords:
Formal explainability, Abductive explanations, Spiking neural networks, Binary neural networks, SMT solver, Causal models, Neuromorphic computing

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_qZ58OpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper aims at explaining the output of convolutional binary spiking neural networks. The approach follows Satisfiability Modulo Theories, aiming at capturing causal model of the neural network. Baseline (SHAP) comparison is missing convincing arguments that the approach is better. No other baseline is providing which makes it difficult to assess. Benchmarks are MNIST and DVS-Gesture classification tasks. Issues with the complexity / scalability of the approach - no discussion on well this could scale (data / model wise). This makes the paper difficult to recommend for acceptance.

Critical strengths
  • formal guarantee but at the cost of complexity and scalability - well known issue of such methods.
Critical weaknesses
  • scalability: more information would be needed.
  • SHAP comparison is weak. First no real conclusive elements. Second missing other baselines to value the benefits.
  • Explanation not necessarily useful for users - at least some users studies should have been done
Questions for authors
  • would that approach more scalable than existing Satisfiability Modulo Theories based methods?
  • why no user studies?
Reviewer_3BdnOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper explains binarized convolutional spiking networks (C-BSNNs, ternary weights) by mapping them to binary causal models, encoding those models into linear integer arithmetic, and running a Z3-backed deletion search: an end-to-end strategy that yields subset-minimal abductive explanations, and more efficient decomposition strategies that yield sufficient explanations without guaranteed subset-minimality. Batch normalization is absorbed into a time-varying firing threshold so the weights stay ternary. Experiments cover MNIST (10 classes, two conv-pool blocks) and a deliberately reduced DVS-Gesture setup (one conv-pool block, two input channels with eight convolutional output channels, one-vs-rest or three classes, downsampled 32×32/16×16 inputs), comparing end-to-end and decomposition search strategies, with a case study showing Kernel SHAP can assign importance to pixels disconnected from the predicted output head, and an analysis of two false positives where input events provably force erroneous output-neuron firing.

Critical strengths
  1. The construction is sensible and comes with a reusable trick. The C-BSNN → causal model → LIA pipeline (Sec. 5) targeting exact AXp semantics is the right formal target, and BN absorption into a time-varying threshold (Sec. 4, Eq. 3) to preserve weight ternarity is a useful idea for SMT analysis of quantized networks.

  2. The speed–size trade-off is demonstrated, and the certification examples are concrete. Table 2 and Figure 1 make the trade-off visible: decomposition reduces the reported MNIST search time from 6.79 days (end-to-end) to 15.4 hours (n-layer) at the cost of retaining more pixels. The SHAP case study identifies pixels disconnected from the predicted output head that the tested Kernel SHAP configuration selects as relevant (Table 5), and the Section 7.2 analysis certifies sets of input events that provably force erroneous output-neuron firing — something a purely attribution-based method cannot show.

Critical weaknesses
  1. The demonstrated explanation target is limited. All explanations are computed at t* = 1 with the zero initial state of Appendix A.2, so the runs cannot exercise accumulated history, leak of previously accumulated potential, or resets following earlier spikes; using an event-based dataset does not by itself demonstrate explanations of temporal dependencies. "Preserving recurrent temporal dependencies" (abstract) is a claimed capability of the formalism — Sec. 7 notes the analysis extends to multiple time steps at higher cost — not something the experiments test. In addition, the explained target differs from the classification target: Section 3.1 obtains predictions by argmax over time-averaged output spikes, whereas Appendix A.3 (Eq. 10) explains the output spike vector at a single selected timestep. Fixing the output at t* = 1 does not generally fix the final prediction over a longer sequence, so how the reported explanations relate to the final classification is unclear — this also affects how the Section 7.2 false positives, which certify erroneous output-neuron firing, should be read relative to the sequence-level prediction. Since the temporal claim appears in the abstract, this gap matters to how the contribution should be scoped.

  2. The empirical evidence is not sufficient to characterize practical performance. The demonstrated operating range is restricted (one conv-pool block, two input channels, at most three DVS classes, downsampled inputs), and the costs are uneven: the end-to-end MNIST explanation takes 6.79 days (Table 4), with decomposition bringing the Table 2 example down to 15.4/23.8 hours — a substantial saving, but still hours per explanation — while the reduced DVS explanations complete in minutes (Table 3). What typical performance looks like between these points is not established: the number of explanation instances underlying Tables 2–4 is not stated, Table 2 corresponds to a single sample, Table 4's caption scopes its runtimes to the fastest run, and while Table 4 does report retained-fraction ranges over its evaluated samples, it does not state how many there are, nor provide representative runtime distributions or completion/timeout rates. For the n-layer strategy — the practical contribution — the actual value of n, the layer-counting convention it depends on, and the bundle boundaries are unspecified. The explanations also retain many zero-valued input constraints: in Figure 1, these account for 280 of the 354 literals in the end-to-end MNIST explanation. Their role in certifying the output is clear, but the paper gives limited guidance on how a practitioner should interpret or use these large sets of constraints.

Questions for authors
  1. Is the BN scale a = γ/√(σ²_run + ε) constrained positive during training? How are a < 0 (where dividing by a reverses the inequality) and a = 0 (where the displayed formula is undefined) handled? Could you report agreement between the original BN-equipped network and the absorbed network, preferably comparing spike traces across timesteps rather than only final class labels?
  2. The A.4.2 argument invokes AXp minimality (Definition 4), while Section 6 states the n-layer output need not be minimal. Could you provide the missing justification for the decomposition case? Separately, Theorem 2 guarantees a nonzero connection to some successor convolutional neuron, whereas Section 8 disconnection is from the predicted output head specifically — could you align the theorem, the disconnection criterion, and the explanation target?
  3. Either a multi-timestep experiment in which earlier inputs actually affect the explained output, substantiating the abstract's temporal-dependency claim, or a clarification of that claim's scope. Relatedly, how do the reported explanations relate to the final classification, given that prediction uses argmax over time-averaged output spikes (Sec. 3.1) while explanations fix the output spike vector at one timestep (A.3, Eq. 10)? How should the Section 7.2 certified erroneous firings be read relative to the sequence-level prediction?
  4. How many explanation instances underlie Tables 2–4, and how were they selected? Please provide representative runtime distributions, completion/timeout rates, and the actual n and bundle boundaries for the n-layer strategy. Which MNIST encoder and number of timesteps were used for the reported accuracy and explanation results? If Poisson encoding was used, are explanations conditional on a particular sampled spike realization, and is any guarantee intended across resamplings of the same image? Please also specify the DVS event-count thresholds and how they were chosen, or identify the relevant supplementary configuration.
  5. Does the 70/20/10 DVS-Gesture split keep subjects disjoint across train/validation/test, and how does it relate to the subject-disjoint protocol of the original benchmark (23 training subjects, 6 held out)?
  6. How sensitive are the results to the fixed deletion order? Even two or three alternative orderings would help.
  7. Kar et al. (2026) already maps BSNNs to causal models, computes explanations with SAT/SMT, and compares against SHAP, using fully connected networks with one hidden layer; Section 2 frames this submission as extending that perspective to convolutional architectures and event-based spike streams. Beyond the setting change, what new technical obstacles do the convolutional and temporal extensions introduce, and which of the paper's contributions (e.g., BN absorption, the decomposition strategies) address them? Is the OBDD/SDD knowledge-compilation approach of Shi et al. (2020) applicable to these networks, and if not, why?
  8. How was the threshold δ = 8·10⁻³ chosen, and how sensitive are the conclusions to it, to the explained output, the background dataset, and the regularization settings? I note the effect attenuates with sampling size: at 10⁵ samples only 4 of 25 draws select a disconnected pixel, with a maximum selected fraction of 2.2% of the 46.
  9. The decomposition procedures produce intermediate-layer explanations as byproducts. Could you illustrate one and discuss whether it offers a more interpretable summary than the input-level explanation?
Reviewer_HAp8OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

The paper proposes an SMT-based framework for formally explaining convolutional binary spiking neural networks (C-BSNNs). It maps networks with ternary weights to binary causal models and computes abductive explanations (AXps), i.e., subset-minimal sets of input assignments sufficient to guarantee an output. To improve scalability, the authors introduce layer-wise decomposition strategies that preserve sufficiency but not necessarily minimality. They also propose a procedure for absorbing Batch Normalization into firing thresholds. Experiments are conducted on MNIST and reduced DVS-Gesture models, with an additional comparison against Kernel SHAP.

Critical strengths

Exploiting the discrete structure of spiking networks for formal explainability is well motivated. Extending the previous causal-model approach to convolutional architectures is a useful contribution.

The distinction between exact AXps and sufficient but potentially non-minimal explanations is clear. The decomposition approach is sensible, and the sufficiency guarantee is useful.

The BN absorption procedure is potentially useful, and the experiments provide a candid illustration of the computational costs of formal explanations.

Critical weaknesses

Temporal evaluation is unconvincing. This is my main concern. All explanations are computed at (t^\star=1), with membrane potentials initialized to zero. At this step, there is no accumulated spiking history, so the experiments essentially explain a feedforward computation. The claimed ability to explain temporal dependencies is therefore not demonstrated, especially on DVS-Gesture.

Mismatch between what is explained and the actual prediction. The formal explanandum is the output-spike configuration at a single time step, whereas classification uses time-averaged firing rates. An explanation sufficient for the former need not preserve the final predicted class. This substantially limits the interpretation of the reported guarantees.

Formal correctness needs clarification. It is unclear how the SMT encoding handles membrane potentials under counterfactual input assignments. These states depend on earlier spikes and cannot simply be fixed to their observed values. Relatedly, the BN absorption derivation assumes a positive scale parameter, which is not guaranteed by standard BN. Theorem 2 also appears weaker than the claimed relevance guarantee, and its proof invokes minimality even though decomposed explanations need not be globally minimal.

Limited empirical scope and scalability. The DVS experiments use only one-vs-rest and three-class models rather than the full 11-class task. End-to-end explanation takes almost seven days for the reported MNIST example. More importantly, the results appear to cover very few instances, making it hard to assess typical runtime or explanation quality. The SHAP comparison is also fairly narrow: it mainly demonstrates finite-sample attribution errors rather than comparing methods on a common explanation objective.

Questions for authors

Why are all explanations computed at (t^\star=1)? Can the authors demonstrate explanations at later time steps, where earlier input events and membrane history genuinely affect the output?

Can the framework explain the final classification decision, rather than an instantaneous output-spike vector? If not, how should the reported guarantees be interpreted?

How exactly are membrane potentials encoded in SMT when earlier input spikes are allowed to vary? Is equivalence between the SMT representation and the original network verified under such perturbations?

Does BN absorption preserve the network's behavior for nonpositive BN scales? Also, can the authors clarify the proof of Theorem 2 for decomposition strategies, given that these do not guarantee global minimality?

How representative are the reported runtimes? It would be helpful to see results across multiple instances, including runtime variability and timeouts, as well as a clearer assessment of scalability beyond the small architectures considered.

Reviewer_iAhqOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

The paper contributes a formal model of convolutional binary spiking neural networks (C-BSNNs) using a Satisfiability Modulo Theories (SMT) encoding of the underlying binary causal model. This is used to compute formally verifiable "explanations" of C-BSNN outputs, where explanations are subset-minimal sets of observed features that fix the model's output ("abductive explanations"). The paper suggests some search techniques to trade off global minimality of the explanations against computation time and evaluates them on MNIST and reduced DVS-Gesture tasks. It then compares them with "explanations" found via kernel SHAP and finds that SHAP has a pathology that allows it to find input features that are disconnected from the output prediction at low sample sizes.

Critical strengths

Strengths: The paper provides a concrete extension of formal explanation to a specialized architecture. Incorporating convolution, pooling, and spiking dynamics goes beyond the shallow, fully connected setting of the prior work they describe. It also explicitly notes and measures the tradeoff between minimality and compute time for their search methods, while retaining the formal sufficiency guarantee.

Critical weaknesses

The temporal claims are not backed up empirically. The experimental evidence is restricted to t∗=1t^* = 1, which means they do not exercise dependencies on prior input events or accumulated firing/reset history. The evidence does not support the claim emphasized in the abstract that "We demonstrate that our SMT-based causal analysis extends beyond static inputs, preserving the recurrent temporal dependencies of spiking dynamics to support formal explanations of complex event-based DVS-Gesture data."

The authors claim to "compute formally correct abductive explanations for [the C-BSNN's] outputs". But correctness is only established with respect to the SMT encoding. The faithfulness of the encoding from the C-BSNN -> binary causal model -> SMT encoding is not established for all input stimuli considered during explanation search. The provided proofs therefore do not establish that sufficiency in the encoding implies sufficiency for the network being explained. In particular, membrane potentials and absorbed thresholds depend on firing history, yet §5 line 260 describes them as externally computed parameters. This proof is a critical piece of showing that the espoused formal guarantees provided by the method apply to real networks under study.

Questions for authors

How are the externally computed parameters treated when inputs change during explanation search? Are they held fixed at their observed values or recomputed?

Can you provide a correspondence argument establishing that the encoding and network dynamics produce identical output spikes for every admissible input sequence, including changed firing/reset histories?

Do the same explanation search methods work when t∗>1t^* > 1 and earlier input events are allowed to vary? Can you demonstrate this on an example where earlier events affect the output through accumulated membrane state or firing/reset history, and report runtime?

How would reported explanations provide guarantees about the actual final class prediction e.g. obtained from time-averaged spikes? The reported results only provide guarantees about the instantaneous output spike vector.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.