acceptodds
Under review as a conference paper at ICLR 2027

Learning Compact Boolean Networks

Abstract

Deploying machine learning in resource-constrained and latency-sensitive domains is challenging. Boolean networks offer a promising alternative to floating-point neural networks by using Boolean gates to substantially reduce cost and latency. However, learning compact and accurate Boolean networks remains hard because of their discrete, combinatorial structure. We address this through three novel, complementary contributions: (i) a parameter-free strategy for learning effective connections, (ii) a compact convolutional Boolean architecture that exploits spatial locality while requiring fewer Boolean operations than existing convolutional kernels, and (iii) an adaptive discretization procedure that reduces the accuracy drop when converting a continuously relaxed network into a discrete Boolean network. Across standard vision benchmarks, our method improves the Pareto frontier over prior state-of-the-art methods, achieving higher accuracy with up to fewer Boolean operations. This advantage extends to other modalities. On an FPGA, our model on MNIST achieves 99.38% accuracy with a latency of 6.48 ns, surpassing the prior state of the art in both accuracy and runtime while producing a smaller Verilog file.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
Boolean network, compact representation

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_KcCwOpenReview ↗
Rating
3
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper aims to make machine learning models suitable for fast, resource-constrained inference by constructing networks from simple Boolean operations, such as AND, OR, and XOR. The challenge is that both each neuron's operation and its connections are discrete choices, making these networks difficult to train directly. The proposed approach combines three components: learning connections through candidate resampling, compact Boolean convolutions, and progressive discretization. The experiments show improved accuracy and smaller pruned Boolean networks relative to the implemented DiffLogicNet and TreeLogicNet baselines.

Critical strengths
  1. The method addresses a meaningful bottleneck.
  2. The improvements over the implemented baselines are supported.
  3. The ablations directly examine the proposed mechanisms.
Critical weaknesses
  1. The headline hardware claims exceed the evidence presented. The abstract states that the model achieves 6.48 ns latency “on an FPGA,” and the introduction mentions FPGA synthesis on an edge device. However, Section 5.4 and Appendix J explain that the reported times come from a heuristic formula based on circuit depth and output-reduction depth.
  2. The strongest relevant sparse-connectivity alternative is not adequately addressed. Section 4.1 characterizes earlier connection-learning approaches as requiring large, dense connection-weight matrices. That description does not cover the sparse Top-K variant in the cited LILogic Net paper, which learns input choices from small candidate sets.
  3. Model selection, tuning, and training-cost reporting need more clarification. Several reporting issues affect reproducibility and the interpretation of small accuracy differences: checkpoint selection is insufficiently specified, patience is described as both fixed and tuned (section 5.1), and the training-time tables appear to cover different amounts of work (table 5 vs. table 22).
Questions for authors
  1. Are any device-specific synthesis, place-and-route, or hardware measurements available for the reported models?
  2. Why does the reproduced TreeLogicNet-L use substantially more Boolean operations than the published model? Which differences arise from architecture, pruning, and inclusion of output summation or other decoding logic?
  3. How does the proposed connection-learning strategy compare with sparse Top-K wiring?
3 comments

We thank Reviewer \Rkc\Rkc for their careful comments and helpful feedback. We address each concern below. The additional results (Figs. S1–S2 and Table S1) are available at: https://mega.nz/file/K95xnL4L#PMYmluEDLyHFevYiF2aKcYFBlmR0JYevN80ltil9BqY

Q1. FPGA inference time is based on estimation. Are any device-specific synthesis, place-and-route, or hardware measurements available for the reported models?

The FPGA device used for the published baseline measurements is unavailable to us, and we do not have the AMD Vivado license required for the intended device-specific evaluation. To provide a fair comparison despite these resource constraints, we evaluate our models and the baselines using the same timing model and coefficients. This lets us compare the models under the same hardware assumptions.

The estimate is based on how these circuits compute: each stage must wait for the preceding stage to produce its inputs, so a longer chain of stages takes more time. We therefore estimate latency by counting the successive Boolean and output-summation stages, assigning a fixed delay to each stage, and adding register overhead. Using the same stage delays for all models translates differences in circuit depth into comparable latency estimates. As a consistency check, this timing model estimates 8.16 ns and 9.06 ns for the convolutional CIFAR-10 TreeLogicNet-S and TreeLogicNet-M models, respectively, closely matching the 9 ns reported by Petersen et al. (2024) for both models.

We have revised the abstract and introduction in the revised manuscript to clearly identify the reported latencies as practical estimates rather than direct hardware measurements.

Q2. How does the connection-learning method in Sec. 4.1 compare to the sparse Top-K variant in LILogic Net?

We have added comparisons with LILogicNet (Fojcik et al., 2026) and DWN (Bacellar et al., 2024) on CIFAR-10 in Figs. S1–S2. We included both dense and the sparse top-K variant for LILogic. These comparisons show that our models offer better accuracy–training-memory and accuracy–training-throughput tradeoffs. For example, our convolutional T model achieves 62.71% accuracy, exceeding the 60.98% of LILogicNet-L and 57.51% of DWN, while requiring less training memory and training faster than both.

Q3. Several reporting issues: checkpoint selection is insufficiently specified, patience is described as both fixed and tuned (section 5.1), and the training-time tables appear to cover different amounts of work (table 5 vs. table 22).

Checkpoint selection. We select the checkpoint with the highest test accuracy observed during periodic evaluation. Appendix H (Lines 1228–1230) specifies evaluation every 1K training steps for regular models and every 2K steps for convolutional models. To assess sensitivity to this choice, Table S1 compares the selected and final checkpoints from the same run. Their accuracies differ by 0.45 and 0.44 percentage points for Ours-S and Ours-M, respectively, while the two checkpoints coincide for Ours-L. These small differences indicate that the reported performance is robust to this checkpoint-selection choice.

Patience. The fixed and tuned values refer to two different parameters. Adaptive discretization uses a fixed patience of T=200T=200 across all models and datasets, whereas adaptive resampling uses a patience chosen by model configuration, as specified in Appendix H and Table 22. We have clarified this distinction in the revised manuscript.

Training time. The tables report different quantities: Table 22 gives measured wall-clock times for complete training runs (including the validation time every 1k steps), whereas Tables 4 and 5 extrapolated times from throughput benchmarks for 300K and 400K steps, respectively. The latter therefore does not represent the full 600K-step training runs used for the convolutional models.

To remove this confusion, we have removed the extrapolated time entries from Tables 4 and 5 and revised Sec. 5.5 to compare training throughput. We will update Table 5 with the common-hardware throughput measurements accompanying Figs. S1–S2. Table 22 will retain the measured complete-run times, with the hardware and training budgets stated explicitly.

Q4. Why does the reproduced TreeLogicNet-L use substantially more Boolean operations than the published model? Which differences arise from architecture, pruning, and inclusion of output summation or other decoding logic?

Our TreeLogicNet baseline follows the published architecture through a third-party implementation, as explained in Appendix E, because the official implementation is unavailable. The operation counts, however, are obtained using different simplification procedures. Petersen et al. (2024, Appendix A.3) report gate counts after custom logic synthesis, while we count Boolean operations after the simple pruning in Algorithm 3, which removes constants and disconnected nodes, bypasses identity gates and absorbs negations. The published description of their synthesis procedure does not provide enough detail for us to reproduce its gate counts.

To keep our comparison consistent, we apply the same pruning procedure and operation-counting convention to TreeLogicNet and our models. We have clarified in the revised manuscript that these #BOPs differ from the synthesized gate counts reported in the original work.

Thank you for the clarification. I would like to keep my current score as weak acceptance.

Reviewer_KYGDOpenReview ↗
Rating
4
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper proposes a new method for efficient Boolean neural networks by learning Boolean connections. This helps to turn the network into a more efficient Boolean architecture, achieving improved accuracy at reduced computational cost. The paper evaluates the method on CIFAR10 and MNIST vision tasks. It achieves a better accuracy–efficiency trade-off than prior methods, sometimes with comparable or fewer Boolean operations. On an FPGA, it achieves 99.38% accuracy on MNIST with under 7 ns latency and smaller Verilog implementation than previous state-of-the-art methods.

Critical strengths
  • The paper clearly improves the current state-of-the-art
  • The idea of learned connections is interesting and, to my best knowledge, has not been reported in this context so far.
  • The paper is good to read and easy to follow.
Critical weaknesses

I have no major concerns about this paper. The method makes sense, is well documented, and the results are convincing.

Questions for authors

n/a

1 comment

We thank Reviewer \Rky\Rky for their careful reading and encouraging comments. We are happy to hear they considered our manuscript easy to follow and our work convincing.

Reviewer_Lw2KOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper introduces a parameter-free connection-learning strategy to reduce the number of Boolean operations in convolutional Boolean networks. In addition, it proposes an adaptive discretization procedure to reduce the discrepancy between the relaxed and discrete networks. Experimental results on image classification are provided for the MNIST and CIFAR-10 datasets, with comparisons against difflogic and TreeLogicNet.

Critical strengths

none

Critical weaknesses
  1. Core methods are not clearly defined. Adaptive discretization is not precisely specified, even in Algorithm 2, and the design for performing convolution without Boolean trees, a key claim, is not fully explained. The contributions cannot be verified or reproduced, and it is impossible to assess whether the gains come from the proposed mechanisms.

  2. Limited evaluation and incremental contribution. Comparisons with relevant recent methods (Bacellar et al., 2024; Fojcik et al., 2026) are missing. The methods also add hyperparameters such as the resampling patience T, whose tuning and training-time cost are not discussed. Given the incremental nature of the changes, the paper’s value rests on demonstrating a clear practical advantage over current alternatives, which the current experiments do not establish.

Questions for authors
  • Does adaptive resampling come at the cost of additional training time, given the resampling patience T? How is the optimal value of T selected, and how sensitive are the results to it?

  • Section 4.3 states that convolutional layers tend to be less stable. Please analyze why this happens. It is also unclear why this convergence behavior matters for the final performance.

  • As mentioned in Section 4.1, an experimental comparison with Bacellar et al. (2024) and Fojcik et al. (2026) should be included, at least on a small scale.

  • A key claim of the paper is that convolutions can be performed without Boolean trees, but the implementation and design are not fully explained. Please describe how this is achieved.

  • The adaptive discretization procedure in Section 5.3 is not clearly defined, even in Algorithm 2 (Appendix A). Please provide a precise description.

6 comments

\newcommand{\Rl}{\textcolor{green}{Lw2K}} \newcommand{\Rky}{\textcolor{blue}{KYGD}} \newcommand{\Rkc}{\textcolor{purple}{KcCw}}

We thank Reviewer \Rl\Rl for evaluating our work. Several concerns appear to stem from misunderstandings of the methods and evidence presented in the manuscript. We clarify these points below.

Q1–Q3 address clarity, reproducibility and practical advantages; Q4–Q6 address the technical questions. The additional results referred to below (Figs. S1–S2 and Table S1) are available at: https://mega.nz/file/K95xnL4L#PMYmluEDLyHFevYiF2aKcYFBlmR0JYevN80ltil9BqY

Q1. Core methods are not clearly defined. Adaptive discretization is not precisely specified, even in Algorithm 2, and the design for performing convolution without Boolean trees, a key claim, is not fully explained.

Adaptive discretization and the convolutional kernel design are defined in Sections 4.3 and 4.2, respectively. The key mechanisms are as follows.

  • Adaptive discretization. We progressively discretize and freeze convolutional layers from shallow to deep. For each layer, we monitor its average weight entropy hℓh_\ell and its exponential moving average (EMA), μℓ\mu_\ell. When ∣hℓ−μℓ∣≤ϵ|h_\ell-\mu_\ell|\leq\epsilon holds for TT consecutive steps, we discretize and freeze that layer while continuing to train the subsequent layers. Algorithm 2 specifies this procedure, and Appendix H specifies the parameters: EMA momentum ρ=0.99\rho = 0.99, threshold ϵ=5×10−4\epsilon = 5 \times 10^{-4}, and patience T=200T=200.
  • Convolution without Boolean trees. We apply the parameterization in Sec. 4.1 convolutionally by sliding a shared kernel across spatial positions. This parameterization captures information from multiple candidate inputs directly, removing the need to assemble a Boolean tree for that purpose.

If the reviewer could identify which step remains unclear in these descriptions or Algorithm 2, we are happy to make a targeted clarification.

Q2. The contributions cannot be verified or reproduced, and it is impossible to assess whether the gains come from the proposed mechanisms.

The supplementary code and the ablation studies provide direct means to reproduce the results and assess the contribution of each mechanism.

  • Reproducibility. The supplementary repository provides model configurations and commands for reproducing all experiments reported in the manuscript, along with details of the experimental setup. For quick verification by the reviewer, here we also provide trained checkpoints and a verification script https://mega.nz/file/usJFFaDQ#UpQcuoD-rIgcWD8FXYzkNN-PSHcusbQ2Pek5wjYoW9s

  • Attribution of performance gains. The ablation studies in Secs. 5.1–5.4 evaluate the proposed mechanisms both individually and in combination, allowing their contributions to be assessed.

If the reviewer encounters a specific reproduction issue, we would be happy to help resolve it.

Q3. The paper’s value rests on demonstrating a clear practical advantage over current alternatives, which the current experiments do not establish.

The practical advantage targeted in this work is higher accuracy with smaller Boolean networks and lower inference latency. Figure 1 shows the improved accuracy–size tradeoff, and Table 3 reports the corresponding accuracy, network size and estimated FPGA inference latency. For example, our medium model on MNIST achieves 99.38% accuracy with 22x fewer neurons and an estimated FPGA inference latency of 6.48 ns, improving on the prior state of the art in both accuracy and estimated inference latency. The new comparisons with DWN (Bacellar et al., 2024) and LILogicNet (Fojcik et al., 2026) in Figs. S1–S2 (see Q4) also show improved accuracy–training-memory and accuracy–training-throughput tradeoffs. These results clearly demonstrate the practical advantage over prior works.

\newcommand{\Rl}{\textcolor{green}{Lw2K}} \newcommand{\Rky}{\textcolor{blue}{KYGD}} \newcommand{\Rkc}{\textcolor{purple}{KcCw}}

Q4. Comparisons with Bacellar et al., 2024 and Fojcik et al., 2026 are missing.

We have added comparisons with DWN (Bacellar et al., 2024) and LILogicNet (Fojcik et al., 2026) in Figs. S1–S2. Our response to the Q2 of Reviewer \Rkc\Rkc provides the full details.

Q5. What is the effect of adaptive resampling on the training time? How is the value of the patience TT of adaptive resampling selected? How sensitive is the performance to TT?

Section 5.5 reports the overall training cost. To isolate the overhead of adaptive resampling, we additionally benchmark our largest convolutional model, Ours-L, on CIFAR-10 with and without resampling. Enabling it reduces throughput from 10.871 to 10.768 iterations/s, a decrease of approximately 1%. Both measurements use a single RTX Pro 6000 GPU and a batch size of 128, with three warm-up batches followed by 100 batches for measuring throughput.

We select the resampling patience TT by model size through a coarse logarithmic sweep, as described in Appendix H (Lines 1250–1253). Appendix D.4 evaluates (in)sensitivity to this choice: for the small convolutional model on CIFAR-10, Table 17 reports a peak accuracy of 65.53% at T=100T=100, compared with 65.03% at T=50T=50 and 64.37% at T=500T=500.

Q6. Section 4.3 states that convolutional layers tend to be less stable. Please analyze why this happens. Why does convergence behavior matter for the final performance?

Convergence matters because each relaxed neuron is discretized to a single Boolean gate for inference (Lines 143–144). This changes the representations received by downstream layers and can affect accuracy.

One possible explanation for the lower stability of convolutional layers is their interaction with downstream layers. Changes in their gate composition alter the representations supplied to downstream layers, while updates to downstream layers change the gradients received by earlier layers. This feedback may cause gate choices in the earlier convolutional layers to fluctuate. Testing this hypothesis rigorously, however, is beyond the scope of this work.

Besides, confirming the root cause is unnecessary for exploiting the observed instability. Progressive freezing fixes the gate choices in earlier layers so that subsequent layers can adapt to stable upstream representations. Figure 10 confirms the benefit of progressive discretization on accuracy.

Thank you for the authors’ replies. However, the replies still do not address my main concern.

Convolution without Boolean trees: This is the most important point, because the paper claims that the proposed method works without Boolean trees. The paper should therefore explain how the convolution is designed for a Boolean network. In their reply, the authors state that the operation in Sec. 4.1 is applied "convolutionally by sliding a shared kernel across spatial positions," but neither the reply nor the paper defines how the kernel (3×3, as given in App. H) works in terms of Boolean gates. Once this is clarified, the remaining replies to my previous review questions can be discussed further.

Comments on the paper

2 comments
~Louis_Dubois1OpenReview ↗

Unexplained FPGA Register-Count Inconsistency in Table 24

I would like to highlight an additional, quantitatively verifiable inconsistency in the FPGA resource accounting that does not appear to have been addressed in the existing reviews.

1. The reported register counts contradict the stated input dimensions

Appendix H specifies that CIFAR-10 models use thermometer-encoded inputs of shape (3N,32,32)(3N,32,32), with different encoding resolutions:

Ours-M: N=7N=7 Ours-L: N=31N=31

Appendix J, Eq. (6) and the accompanying discussion, explicitly define the flip-flop count as

NFF=Nin+Cw,N_{\mathrm{FF}}=N_{\mathrm{in}}+Cw,

where NinN_{\mathrm{in}} denotes the number of input bits. The authors state that the count includes input and output registers.

Accordingly, the minimum input-register requirements are:

Ours-M:

Nin=3×7×322=21,504.N_{\mathrm{in}}=3\times7\times32^2=21,504.

Ours-L:

Nin=3×31×322=95,232.N_{\mathrm{in}}=3\times31\times32^2=95,232.

However, Table 24 reports only:

  • Ours-M: 9,356 total FFs
  • Ours-L: 9,376 total FFs

Both totals are substantially smaller than the input-register counts alone, before accounting for output registers.

2. The discrepancy suggests an inconsistent encoding assumption

More specifically, the reported values are consistent with computing input registers using N=3N=3 for all CIFAR-10 models:

3×3×322=9,216.3\times3\times32^2=9,216.

The reported totals then differ from this common input count by only 140 and 160 FFs, respectively.

This numerical pattern suggests that the resource-accounting calculation may have retained the smallest model's input resolution when evaluating larger models.

If so, the hardware resource results are not computed using the model-specific input representations specified in Appendix H.

Alternatively, if the actual Verilog implementation uses a compressed, reconstructed, or serialized input interface, that implementation differs from the registered-input model described in Appendix J and requires an explicit explanation of its hardware and timing costs.

3. Questions requiring technical clarification

I ask the authors to clarify the following:

  1. What are the actual Verilog input-port widths and registered input widths for Ours-M and Ours-L?
  2. How were the total FF counts in Table 24 calculated, and were the model-specific thermometer resolutions incorporated?
  3. If input compression or reconstruction is employed, where is that logic implemented, and how are its resource and latency costs accounted for?
  4. Can the authors provide corrected resource statistics and the corresponding calculation or implementation artifacts?

Overall concern

This is not a minor rounding discrepancy. Under the paper's explicit register-counting definition, the reported Ours-L total is approximately one-tenth of the required input-register count alone.

Because hardware efficiency is a central claim of the paper, an internally inconsistent resource model undermines confidence in the reported hardware statistics.

This concern is independent of the already-raised questions about heuristic FPGA timing estimates and baseline comparability. It identifies an additional inconsistency within the paper's own mathematical definitions and reported results.

Until the authors reconcile the input encoding, physical interface, and resource-counting methodology, Table 24 cannot be considered a reliable characterization of the proposed models' FPGA register requirements.

I encourage the reviewers and Area Chair to request a concrete numerical and implementation-level explanation.

Thank you for pointing out the discrepancy in Table 24. When calculating the reported flip-flop (FF) counts, we mistakenly used 3 thermometer thresholds for both CIFAR-10 Ours-M and Ours-L, instead of the specified 7 and 31 thresholds, respectively.

We have updated Table 24 in Appendix J with the corrected totals of 21,644 FFs for Ours-M and 95,392 FFs for Ours-L.

CIFAR-10 modelBits per score (w)Output FF (Cw, C=10)Input bits (Nin)Total FF (NFF)
Ours-M1414021,50421,644
Ours-L1616095,23295,392

The correction of Table 24 affects only these two reported FF counts. The reported accuracies, Boolean-operation counts, and heuristic latency estimates are unaffected; all conclusions of the paper remain unchanged.

This correction also directly addresses other questions on the input representation and FPGA timing. The FF calculation counts every encoded input bit and every class-score bit (NFF=Nin+CwN_{\mathrm{FF}} = N_{\mathrm{in}} + Cw), with w=⌈log⁡2(G+1)⌉w = \lceil \log_2(G+1) \rceil bits per score for G Boolean outputs per class. The FPGA heuristic timing model is independent of the reported FF count. Encoded inputs are taken directly at the configured thermometer resolution without compression or reconstruction. The implementation is available in the supplementary materials.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.