acceptodds
Under review as a conference paper at ICLR 2027

ViLo-TM: Visual-Logic Mapping from Deep CNNs to Tsetlin Machines

Abstract

Tsetlin Machines (TMs) provide explicit conjunctive rules and gradient-free adaptation, yet learning expressive visual features jointly with their discrete clauses remains difficult. We introduce ViLo-TM, a family of vision TMs that jointly learns CNN representations and visually traceable Boolean rules. Its learned spatial-evidence-to-literal (SE2L) interface maps pooled convolutional channel responses to ternary predicate states and paired literals, while continuous relaxations of literal inclusion and conjunction enable end-to-end optimization. Quantization-aware training (QAT) and export yield an integer CNN encoder and a Boolean TM in which each fired clause exposes its literal conditions and signed vote, linked to the source channel maps. The Boolean TM supports bitwise inference and multiplication-free learning through standard TM feedback (Type I/II) with the encoder and thresholds fixed. On CIFAR-10, the deployed Boolean TM exceeds the strongest compared vision-TM baseline (90.15% vs. 82.8%; initializer selected on test accuracy). On CUB-200-2011, Boolean conversion retains most of the QAT accuracy (75.92% 75.01%, mean of three runs), and the deployed TM exactly matches the reference Boolean clause outputs and class scores on all test images (5,794).

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Tsetlin Machine, Neuro-Symbolic Learning, Interpretable Computer Vision, Visual Concept Learning, Differentiable Logic, Discrete Inference

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_zBGTOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes ViLo-TM, a framework that jointly learns deep CNN representations and interpretable Boolean rules of Tsetlin Machines. The SE2L interface maps channel responses into ternary predicates, while ContinuousTM employs differentiable relaxations to allow classification gradients to update the CNN, predicate thresholds, and sparse clause supports. Quantization-aware training (QAT) subsequently enables the export of an integer CNN and a Boolean TM. The authors further demonstrate computational traceability from clauses to channel maps, fidelity of Boolean conversion, and online adaptation through Type I/II feedback with a fixed visual front end.

Critical strengths

Comprehensive interface design from CNNs to Boolean TMs. The framework integrates ternary predicates, sparse clause supports, QAT, integer export, and Type I/II adaptation into a coherent pipeline. The exact agreement between the exported Boolean model and reference clause outputs and class scores on CUB provides valuable evidence of implementation correctness.

Concrete consideration of resource-constrained deployment. The paper evaluates conversion accuracy, classifier adaptation with a frozen visual front end, and FPGA/HLS resource estimates. These analyses explicitly illustrate the trade-off between multiplication-free adaptation and increased state storage and LUT requirements.

Critical weaknesses

Test-set leakage in initializer selection undermines the main experimental results. The paper explicitly acknowledges that the initializers for MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 were selected based on official test-set accuracy. This introduces test-set leakage and compromises the validity of the reported comparisons. In particular, the headline CIFAR-10 result of 90.15% cannot be regarded as an independent held-out evaluation. The authors should select initializers using only training and validation data, fix all hyperparameters accordingly, and reevaluate the models on an untouched test set.

Insufficient evidence for visual semantic interpretability and hardware-speed claims. Semantic concepts are assigned post hoc using a frozen InternVL3.5-8B model, rather than being explicitly grounded during training. The test-set part purity on CUB drops to only 0.23 when using training-assigned concepts, and all non-wildcard conditions are VLM-confirmed in merely 0.17% of eligible clause-firing events. These results raise substantial concerns about whether the learned Boolean rules provide reliable semantic explanations of visual decisions.

Questions for authors

Please refer to Weaknesses.

Reviewer_GCDyOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

The paper introduces ViLo-TM, a family of tsetlin machines (TM) trained jointly with a CNN encoder for image classification. The method turns CNN outputs into boolean literals by learning thresholds and a ternary representation. ViLo-TM is trained on GPU but can be exported to boolean to be deployed on resource-constrained hardware. Experiments show that ViLo-TM matches or outperforms existing approaches.

Critical strengths

The high-level motivation of jointly training the CNN and the TM makes sense for improving model performance.

The method matches or outperforms existing image classification TMs on a wide variety of datasets.

Critical weaknesses

The paper is dense in technical details and is difficult to read.

One primary benefit of Tsetlin machines is its lack of reliance on back propagation. ViLo-TM requires backpropagation during training, which makes the motivation for using TMs over other resource-light architectures less clear.

The method includes many hyper-parameters that need to be tuned for each dataset. For example, appendix B shows that the weights selected for CUB drops the align and usage terms from the loss function in eq 10. It is not clear what the motivation behind these choices are.

The paper does not provide ablation experiments to show that the ternary states are necessary.

Questions for authors

Could a simpler binary representation work instead of the ternary states?

Reviewer_6uNbOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces ViLo-TM, a framework that jointly learns CNN representations, visual predicate thresholds, and Tsetlin Machine clauses for image classification. A ternary literal interface and differentiable relaxations enable end-to-end training, followed by quantization-aware training and export to an integer CNN and Boolean rule classifier. The exported classifier links clause conditions to spatial channel responses and supports Type I/II updates with the encoder and thresholds fixed. Experiments evaluate recognition accuracy, Boolean conversion, visual grounding, and classifier adaptation under image corruptions.

Critical strengths

The paper provides a complete pipeline for jointly learning CNN features and TM clauses, then exporting the model into a Boolean classifier. On CUB, conversion reduces average accuracy by only 0.91 percentage points across three runs. The exported classifier also supports Type I/II updates with the encoder and thresholds fixed. Preserving most of the recognition accuracy while retaining the ability to update discrete rules is a meaningful strength of the approach.

Critical weaknesses
  1. Test-based initializer selection weakens the main accuracy comparisons. Initializers for MNIST, Fashion-MNIST, CIFAR-10, and CIFAR-100 were selected using official-test accuracy. Although explicitly disclosed, this prevents these results from serving as independent estimates of generalization and weakens the highlighted improvements over prior vision TMs. Validation-based selection is needed to establish these gains reliably.

  2. The interpretability benefit remains insufficiently established. The rules expose exactly which channel conditions contribute to a prediction, but those conditions do not consistently correspond to recognizable visual concepts. On CUB, part purity drops to 0.23 when training-selected part assignments are fixed, and LOW responses do not reliably indicate concept absence. These results support computational traceability more strongly than semantic interpretability, leaving the practical explanatory benefit of replacing the classifier unclear.

  3. The deployment advantage is only partially demonstrated. Multiplier-free updates are useful, but the TM requires substantially more mutable state and LUT resources than the linear baseline. Hardware latency is estimated through HLS, without on-board latency or energy measurements, and the update baselines use FP32 arithmetic. The current evaluation therefore does not establish an overall deployment advantage, particularly against low-precision alternatives.

Questions for authors
  1. For CIFAR-10, I would like to see the result with a validation-selected initializer, keeping the remaining pipeline unchanged. How much of the reported improvement over prior vision TMs remains under this protocol?

  2. How much does the ternary interface contribute? A matched comparison with a simple binary-threshold interface would help establish whether the DON’T CARE state improves accuracy or reduces the loss during Boolean conversion.

  3. The hardware comparison uses FP32 linear and MLP updates. Please consider adding a low-precision linear baseline on the same frozen inputs. Comparing adaptation accuracy, mutable state, and hardware resources would clarify when multiplier-free TM updates offer a practical advantage.

  4. The paper includes several automated interpretability evaluations, but the benefit to human understanding remains unclear. The main rule visualization focuses on a single clause; broader examples across classes, including consistent predicate meanings across images and explanations of both correct and incorrect predictions, would help. A human evaluation could also test whether readers can understand the predicates and use the rules to explain the model’s decisions.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.