acceptodds
Withdrawn from ICLR 2027

Task-Aware Discretization of Differentiable Logic Gate Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Differentiable logic gate networks (DLGNs) enable gradient-based training of highly efficient Boolean networks by relaxing discrete logic gates during training and discretizing them for inference. Standard approaches make this discretization decision locally, typically through argmax selection and confidence- or entropy-based convergence criteria. We show that local discretization can be task-suboptimal even for globally optimal relaxed solutions, with high gate confidence providing no general guarantee, and derive bounds relating task-aware gate selection to tractable interventions in the relaxed network. Motivated by these results, we study first-order downstream task information for progressive discretization and characterize when this local approximation is reliable. Experiments on convolutional DLGNs reveal a strong locality dependence: first-order scores become unreliable when directly optimized over nonlocal interventions, but accurately assess local argmax decisions for progressive freezing.

Primary area:
neurosymbolic & hybrid AI systems (physics-informed, logic & formal reasoning, etc.)
Keywords:
Fast ML, Efficient Inference, Logic Gate Networks, Discretization

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_YVTGOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses a critical limitation in Differentiable Logic Gate Networks (DLGNs): standard discretization methods that select gates based on local confidence (argmax) may not minimize downstream task risk, even when the relaxed network is globally optimal. The authors distinguish between the conventional discretization gap and the discrete optimality gap, proving that locally concentrated gate distributions do not guarantee task-optimal discrete networks. They propose a task-aware discretization framework that uses first-order downstream task information to assess when argmax decisions can be safely committed, rather than relying on local confidence metrics like entropy. Their key insight is that first-order scores are reliable for evaluating local argmax interventions but become unreliable when directly optimized over candidate gates due to nonlocal intervention effects. The proposed TaskFreeze method retains argmax gate selection and progressively freezes neurons based on predicted downstream task effects rather than entropy convergence. Experiments on CIFAR-10 with two DLGN architectures (Compact and TreeLogicNet) show that TaskFreeze is competitive with existing discretization strategies, though surprisingly, a simpler GlobalFreeze baseline performs similarly well, suggesting that adaptation to discrete computation during training may be more important than fine-grained scheduling. The work identifies intervention locality as a central consideration for task-aware discretization and highlights that developing reliable task-aware gate replacement beyond the local regime remains an open problem.

Critical strengths

This paper makes significant theoretical and empirical contributions by distinguishing the conventional discretization gap from the discrete optimality gap, proving that even globally optimal relaxed networks with highly concentrated gate distributions can remain suboptimal under standard argmax discretization. The authors derive an efficient first-order approximation for task-aware gate selection that reuses gradients from ordinary training, enabling a practical framework where task information is used to assess rather than directly select gates. A key strength lies in their empirical demonstration that first-order scores reliably predict the downstream effect of local argmax interventions but become unreliable when directly optimized over nonlocal candidates due to intervention locality. Consequently, their proposed TaskFreeze method achieves competitive performance across different DLGN architectures while highlighting that adaptation to discrete computation during training is often more critical than fine-grained scheduling.

Critical weaknesses
  1. Reliability outside the local regime. The first-order task-aware scores become unreliable when directly optimized over nonlocal interventions, as approximation error grows significantly with activation displacement. Median squared displacements for score-selected gates are 0.42 compared to 2.9×10⁻⁶ for argmax interventions. While second-order corrections reduce typical prediction error, the error remains strongly dependent on intervention magnitude. Developing reliable task-aware gate replacement beyond the local regime therefore remains an open problem. 2. Scope of the practical contribution. The motivating question is which discrete gate should be selected to minimize downstream task risk. However, TaskFreeze still chooses gates using conventional argmax; task information determines when those decisions are committed. Its practical contribution is therefore using task information to determine when to freeze argmax decisions, and it does not directly solve the broader task-aware gate-selection problem. The paper acknowledges this distinction, so this is a limitation of the algorithmic contribution rather than a correctness concern. Freezing can still affect subsequent training and the final gate assignment. 3. Empirical significance of the discrete optimality gap. Theorem 1 establishes that concentrated gate distributions do not guarantee task-optimal argmax, but the experiments do not quantify how far argmax lies from the best discrete assignment in practically trained networks. Further empirical evidence is needed to establish whether this gap is substantial in practice and how its magnitude compares with the conventional discretization gap. 4. Unclear added value of the task-aware score. TaskFreeze performs comparably to the simpler GlobalFreeze baseline, and no single freezing strategy consistently dominates across Compact and TreeLogicNet. Further evidence is needed to determine whether the task-aware score provides a benefit beyond early exposure to discrete computation and subsequent adaptation, and under what conditions that benefit is meaningful. 6. Connection between error bounds and the freezing criterion. The theoretical bounds explain approximation error through downstream mismatch and intervention locality, but the practical algorithm does not estimate these error terms. Instead, it freezes a neuron when its EMA score remains negative for the specified patience period. EMA and patience address fluctuations but do not by themselves bound systematic approximation error. It would strengthen the method to explain when the zero threshold is reliable, or to investigate a criterion that requires the predicted loss change plus an appropriate uncertainty margin to remain negative. Such a margin would need to account for the error relevant to the claimed guarantee. 8. Treatment of neurons left unfrozen. Figure 4 appears to leave some neurons unfrozen during training. The final treatment of these neurons and their proportion should be made explicit. If they undergo standard argmax discretization at the end, reporting their fraction by layer and the loss or accuracy change at that final conversion would clarify the coverage of the progressive freezing criterion and the effect of the remaining discretization step.
Questions for authors

Practical Implementation Questions - GlobalFreeze vs TaskFreeze tradeoff: Since GlobalFreeze performs comparably to TaskFreeze across architectures, under what circumstances would the added complexity of task-aware progressive freezing be justified? Are there metrics or scenarios where TaskFreeze's advantages become more pronounced? - Hyperparameter sensitivity: The TaskFreeze method requires tuning scoring interval K, EMA factor β, and patience P. How sensitive are the results to these hyperparameters, and are there principled ways to set them based on network architecture or training dynamics? - Connection resampling impact: The experiments test both continued and stopped connection resampling after discretization. What is the theoretical or empirical justification for choosing one over the other in production settings? - Neurons remaining unfrozen: What happens to neurons that have not satisfied the TaskFreeze criterion by the end of training? Are they finally discretized by standard argmax? Please report the fraction frozen by TaskFreeze versus converted at the end, preferably by layer, and the loss or accuracy change caused by this final conversion.

Experimentalal Scope - Dataset scalability: All experiments are on CIFAR-10. Do you expect the locality dependence of first-order scores to persist on larger datasets like ImageNet, or might different dynamics emerge with more complex data distributions? - Architecture diversity: The paper tests Compact and TreeLogicNet architectures. Have you considered testing on recurrent DLGNs or other variants where temporal dependencies might affect the optimal discretization schedule? - Layer-wise patterns: TaskFreeze and EntropyFreeze show distinctly different layer-wise freezing patterns. Are there architectural properties that predict which criterion would be more effective for a given network?

Quetsions about Future Work - Joint optimization: The paper notes that developing reliable task-aware gate replacement beyond local interventions remains open. Would jointly optimizing multiple discrete decisions simultaneously (rather than progressive freezing) be a promising direction, and what computational challenges would this introduce? - Hardware considerations: Given DLGNs are designed for efficient hardware deployment, have you considered how different discretization strategies affect actual hardware metrics like latency, power consumption, or FPGA resource utilization? - Downstream task generalization: The task-aware criterion is optimized for a specific downstream task. How does the discretization strategy perform when the discrete network is deployed on tasks different from the one used during training?

Reviewer_S1GeOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper addresses the discretization error that occurs when rounding differentiable logic gate networks (DLGNs) back to the discrete logic gate space they were relaxed from. Previous work employed rounding rules that relied only on the gate parameters of each differentiable logic gate (DLG), but the authors identify a shortcoming in this gate-local approach. They argue that this greedy method of choosing the optimal rounding choice for each gate separately can accumulate a global rounding bias. In their Theorem 1, they provide an example where each DLG is closest to gate AA but also has a small, non-zero weight for gate BB. Since all gates are rounded to AA, the non-negligible occurrence of BB in the soft accumulation in the final summation layer disappears, resulting in an suboptimal discretization error.

To address this issue, the authors propose to incorporate the loss gradient into the rounding decision instead of solely relying on the gate parameters like prior work. Then they theoretically study a loss-dependent, globally more optimal rounding rule that is however computationally prohibitive due to exponential complexity in the number of neurons. Towards a more efficient rounding procedure, they return to greedy gate-local rounding rules, and present a score function that incorporates both the gradient information and the local DLG parameters.

Empirically, they however observe that optimizing their score function results in substantially worse rounding decisions than the local argmax rule that prior work was applying. They do not provide a theoretical or intuitive explanation why optimizing their score function is ineffective.

Eventually, they conclude to keep the argmax rounding rule that prior work was using, and instead turn their attention towards when rounding is applied. A previous work already proposed to progressively round DLGs during training instead of rounding every DLG only after training. Following this idea, the authors put forward to use their score function to decide when a DLG is rounded. They compare several instantiations of their idea to this prior progressive rounding method and find small, mixed differences in performance that remain often within the standard deviation interval.

All in all, I feel that the authors indeed raised an important issue in previous rounding rules, but their efforts in tackling this issue lack rigor and significance.

Critical strengths

The authors identify an issue with rounding rules for differentiable logic gates that previous work in DLGN literature has indeed not considered so far. Wang et al. (2026) proposed progressive rounding during training, and Rüttgers et al. (2026) exposed how the existing DLG parametrization can produce local suboptimality of the argmax rounding rule; but both and all other works that I am aware of did only take into account local DLG parameters for rounding and have not studied global effects of applying this locally greedy rounding rule.

  1. With their Theorem 1, the authors highlight how the locally greedy rounding decisions can accumulate a global non-vanishing rounding bias. With this theoretical result, they motivate the study of incorporating information beyond single DLGs into rounding.
  2. Their proposal to incorporate information from the loss into the rounding decision appears novel to me aswell. With this idea, they set a first potential direction how information beyond single DLGs can be incorporated into the rounding decision.
Critical weaknesses

Despite identifying a novel issue and information-theoretically exposing how the suboptimality of current local rounding rules can be mitigated, the authors fail to obtain a rounding method that significantly improves upon previous rules. That way, the insights that I gained from reading this paper were mainly from the analytical study of optimal rounding and its gap to the local argmax rule.

On the constructive side, I see the following issues:

  1. Their first-order score that is derived as an efficient approximation to the theoretically more optimal "conditional task risk" is information-theoretically ineffective. As the authors show themselves, its optimum does not correlate with a decrease in loss, and this does not surprise me. After all, the score assigns a larger value to discrete gates that are farther away from the current continuous interpolation.

1.1. As the loss from the current minibatch might not be a faithful approximation of the entire population loss, greedily optimizing the rounding selection from the current minibatch results in a larger deviation from the continuous interpolation and hence increases the deviation of the network's output proportional to this deviation times its Lipschitz constant in the worst case. I would have expected the authors to recognize the these conflicting error bounds that need to be traded against each other.

1.2. In the end, the authors return to the argmax rounding rule and turn their attention from how a DLG is rounded (that was the problem they originally wanted to address) towards when a DLG is rounded. From the initial problem motivation and theoretical work, I would expect this paper to properly advance the former problem and at least result in a method that overcomes the failure of the local argmax rule under the conditions of their own Theorem 1.

  1. Their progressive freezing procedure proposed in Algorithm 1 does neither seem theoretically sound nor empirically superior to the progressive entropy freezing schedule from Wang et al. (2026).

2.1. First of all, it doesn't occur to me why Algorithm 1 would even converge. The freezing criterion might never be met.

2.2. Secondly, Algorithm 1 internally still adopts the argmax rounding rule, and thus suffers the same failure under the conditions of the authors's own Theorem 1 as would do standard argmax rounding after training. Theorem 1 serves as the initial example what the current rounding rules fall short of, but if the finally proposed procedure does not overcome the problem even in the example case that Theorem 1 showcases, then I feel the paper's output is not consistent with its motivation.

2.3. Compared to the progressive gate-local entropy rounding procedure from Wang et al. (2026), the various instantiations of the author's gradient-conditioned rounding schedules do not achieve significant, consistent improvements over this baseline. Sometimes the baseline is better, and the difference mostly remains within noise. That way, I unfortunately do not see sufficient improvements upon Wang's prior rounding schedule.

Questions for authors

1a. In Figure 2, has the change in loss been evaluated on a different minibatch or the same minibatch that the score was computed on? Also, was the rounding only applied to a single DLG or multiple at the same time?

2a. Can the authors provide any convergence guarantee of their Algorithm 1? To me, it seems this algorithm could run indefinitely when the score fluctuates across sign boundary strongly.

2b. The authors have exposed the issue of local greedy rounding rules in their Theorem 1. But would their own Algorithm 1 not suffer from the same error accumulation? Every DLG is only rounded to its argmax, so under the setting of Theorem 1, each DLG would either collapse to the constant 0 gate g0g_0 or remain soft indefinitely (if the freezing criterion is never met and the algorithm doesn't halt).

2c. Why have the authors decided to resample connections after training instead of keeping them fixed? Isn't this an avoidable source of noise that is unrelated to the rounding problem and hence perturbs the measured results?

Reviewer_WqfCOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

This paper proposes a task-aware discretization method for differentiable logic gate networks (DLGNs). It uses first-order information from the downstream task to guide progressive discretization. The experiments report layer-wise Spearman correlations, the locality of first-order interventions, progressive freezing dynamics, and CIFAR-10 test accuracy across different freezing methods.

Critical strengths

none

Critical weaknesses
  1. The central concept is undefined and untested. The paper never defines what a "task" is in DLGN training or inference. Despite the method being called task-aware, accuracy is reported only on CIFAR-10, with no comparison across tasks. Task-awareness is the paper’s main claim. Without a definition or multi-task evidence, the claim can be neither understood nor validated.

  2. The components do not form a coherent method. The paper introduces several concepts, but it is unclear how they connect, in particular how task-aware discretization relates to progressive freezing. Key terms (e.g., connection-resampling) are also undefined. Without a clear link between the theoretical definitions and the method, the reported results cannot be attributed to the proposed ideas.

Questions for authors
  1. The paper repeatedly refers to "task-aware" and "task-level" information, but it never defines what a task is in the context of DLGN training or inference, or how tasks differ from one another. Please provide a clear definition.

  2. Are there any experiments that compare the method across different tasks? If the method is task-aware, results on multiple tasks would be needed to support this claim.

  3. What is "CAGE" (Section 5.4, line 433)? What does "connection-resampling" in Table 2 refer to?

  4. It is unclear how task-aware discretization relates to progressive freezing. Please explain the connection between the two.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.