acceptodds
Withdrawn from ICLR 2027

A Mechanistic Explanation of CHSH Violations in Classical Neural Networks

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Non-classical correlations, certified by violations of Bell–CHSH-type inequalities entailed by classical probability theory, are generally assumed to lie beyond the expressive capacity of classical neural networks. In this work, we show that a CHSH-style statistic in standard feedforward multi-task networks can exceed the classical bound of inside a specific “emergence window”, thereby exhibiting non-classical correlations. We further provide a mechanistic explanation of this phenomenon: under the shared-capacity bottleneck of multi-task learning, gradient competition couples hidden neurons; under ordinary feedforward evaluation, easy and hard task pairs become unequally learnable, which unbalances the four correctness correlations and drives . We observe this pattern in numerical simulation, text classification, and image recognition, and further verify that the window is governed by the alignment between network capacity and task complexity. Using CHSH-type inequalities as a probe, this work shows that the learning dynamics of classical networks can also give rise to non-classical statistical correlations, and provides a theoretical basis for understanding non-classical statistical modeling in quantum-inspired methods.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
CHSH inequality, non-classical correlations, quantum-inspired learning, multi-task learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_xcncOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The authors show that classical neural networks can violate Bell-like inequalities in certain settings. The authors' explanation for this phenomenon is that during the training procedure the network effectively "couples" hidden neurons which directly supply nonlocal correlations to the network. The authors suggest this should only happen when the capacity of the network is limited, forcing the training procedure to couple these hidden neurons, and then perform numerical experiments to validate this hypothesis.

Critical strengths

The paper is an interesting application of ideas from quantum physics to understanding the performance of classical neural networks. I find the authors' combination of analysis and empirical study convincing and I agree with their conclusions. I also find the work novel; most previous work which studies contextual correlations and their implications for machine learning, e.g., Quantum 10, 1976 (2026) and npj Quantum Inf. 11, 127 (2025), focus on understanding the limitations of classical networks. This paper offers an interesting complementary perspective by showcasing settings where naive intuition from this previous line of work can yield incorrect conclusions.

Critical weaknesses

The authors' work is not the first to identify whether classical neural networks exhibit quantum contextual correlations; the authors cite arXiv:2603.04451, but there is also, e.g., Quantum 10, 1976 (2026) and npj Quantum Inf. 11, 127 (2025). The authors should cite these previous studies linking contextuality to machine learning.

Another weakness is that, while the authors make an interesting observation, they do not suggest any interesting implications for their observation. What should the main takeaway of the authors' work be in terms of model design?

I think even with these weaknesses the novelty of the work does enough to justify its inclusion, and I recommend acceptance.

Questions for authors

Does the authors' observation suggest any actionable advice in neural network design?

What are some implications of the authors' work on prior work (e.g., Quantum 10, 1976 (2026) and npj Quantum Inf. 11, 127 (2025)) showing that contextual correlations can be used to bound the performance of classical neural networks?

Reviewer_7nC1OpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper investigates why a CHSH-style statistic can temporarily exceed 2 in classical multitask neural networks. Following the NCnet protocol, it constructs the statistic from prediction-correctness correlations across four separately trained two-head networks. The authors propose that limited shared capacity induces gradient competition and hidden-unit coupling, making harder task pairs less learnable than easier ones and producing an imbalance among the four correlations.

Its contributions are a mechanistic account of this imbalance, a phenomenological model of coupling, and an explanation of the intermediate-capacity “emergence window.” The proposed account predicts that excesses diminish when capacity is either insufficient for most tasks or sufficiently abundant, while harder tasks shift the window toward larger networks.

Validation includes three synthetic settings, AAPD text classification, and COCO image classification, with 20 random seeds for the main experiments. Training trajectories, capacity sweeps, gradient-conflict measurements, and approximately parameter-matched architecture comparisons support the reported qualitative patterns. However, hidden-unit coupling is not directly measured, so the causal mechanism remains incompletely validated. Because the correlations come from separately trained models, the results do not establish a standard Bell violation or quantum nonlocality.

Critical strengths

The idea of investigating non-classical-like statistical behavior in classical neural networks is conceptually interesting and offers a fresh perspective on multitask learning. Building on the NCnet framework, this paper connects quantum-information-inspired diagnostics with artificial intelligence by using a CHSH-style statistic to examine correlations between prediction outcomes. Experiments across synthetic tasks, text classification, and image recognition suggest that the observed statistical excess is not restricted to a single toy setting. Beyond documenting the phenomenon, the authors propose a mechanistic explanation based on gradient competition under limited shared capacity: different task pairs become unequally learnable, creating an imbalance among the correctness correlations that constitute the statistic. This explanation helps account for both the temporary appearance of the excess during training and its concentration within an intermediate range of model capacities. The combination of empirical observations and a concrete mechanistic hypothesis makes the paper an interesting exploration of task interactions and representation constraints in classical networks.

Critical weaknesses
  1. The numerical results in Table 2 appear internally inconsistent with the stated definitions of the mean peak statistic and the violation rate. Since each correlation lies in [−1,1][-1,1], every run must satisfy Smax⁡≤4S_{\max}\le4, while any run without a violation must satisfy Smax⁡≤2S_{\max}\le2. Consequently, if a fraction pp of seeds exhibits a violation, the mean peak across all seeds must satisfy E[Smax⁡]≤4p+2(1−p)=2+2p\mathbb{E}[S_{\max}]\le4p+2(1-p)=2+2p. The reported violation rates of 75%, 90%, and 70% therefore imply maximum possible means of 3.500, 3.800, and 3.400, respectively, whereas the table reports 3.565, 3.997, and 3.890. These discrepancies are too large to be explained by rounding. The authors should clarify whether the peak statistics are averaged over all seeds or only those exhibiting violations, provide the per-seed peak values, and correct the numerical results or their reporting conventions accordingly.

  2. The interpretation of the reported high SS-values as “non-classical” behavior is insufficiently justified. Although the statistic has the same algebraic form as the CHSH expression, its four correlations are obtained from separately trained networks, so the consistency assumptions underlying the standard CHSH bound are not guaranteed. The observed pattern is compatible with an ordinary limitation of shared representational capacity: networks may learn the identity and single-XOR task combinations relatively well while struggling to learn two XOR functions simultaneously, producing high C11C_{11}, C12C_{12}, and C21C_{21}, but a lower C22C_{22}. This imbalance can yield S>2S>2 through entirely classical learning dynamics. Although the manuscript explicitly disclaims physical nonlocality, it remains unclear what formal notion of non-classicality is established by this construction. The authors should either justify that interpretation under clearly stated assumptions or describe the phenomenon more directly as an imbalance in correctness correlations across task pairs.

  3. The theoretical and practical significance of exceeding the threshold S=2S=2 remains unclear. In a properly controlled Bell experiment, a statistically significant CHSH violation rules out local hidden-variable explanations under the relevant assumptions, giving the threshold a precise operational meaning. In the present setting, however, the four correlations come from different trained models, and the manuscript does not establish an analogous class of explanations that is excluded when S>2S>2. A high value may simply reflect uneven learning across task combinations, including poor performance on the most difficult pair, rather than a desirable or distinctive property of the network. Consequently, it is difficult to assess what additional understanding or practical benefit this statistic provides. The paper would be stronger if it demonstrated that SS predicts task interference, informs capacity selection, or supports training decisions beyond what can already be inferred from task accuracy, loss, error correlations, or gradient-conflict measurements.

  4. The contribution relative to Zhao et al. (2026) appears incremental and should be delineated more explicitly. As acknowledged in Section 3.2, that work introduced NCnet and applied the CHSH-style statistic to neural-network predictions. It also reported the intermediate-capacity regime and discussed shared-parameter gradient competition and differences in task difficulty as explanations for the excess. The present manuscript adds experiments on additional datasets and architectures, together with a more detailed account involving hidden-unit coupling, but the distinct explanatory advance remains unclear. The proposed coupling dynamics are phenomenological, and the gradient–SS trajectories, capacity sweeps, and architecture comparisons provide primarily indirect evidence for the proposed causal chain. In particular, hidden-unit coupling is not directly measured. A clearer comparison with the prior explanation, supported by direct coupling measurements and interventions that selectively alter gradient competition or representation sharing, would help establish both the novelty and the validity of the claimed mechanism.

Questions for authors

Please explain why Table 2 seems inconsitent

Reviewer_RnoWOpenReview ↗
Rating
2
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper challenges the common view that classical neural networks cannot produce correlations violating Bell–CHSH-type inequalities. It builds a CHSH-style quantity S from predictions in neural networks trained on pairs of tasks. The authors report that S can rise above the classical value of 2, but only within a limited range of model capacity they call an “emergence window.” They attribute this to the constraints of multi-task learning with shared parameters. When model capacity is constrained, the gradients of the two task heads conflict and the hidden units become coupled. Pairs containing an easy task stay well learned, while the pair of hard tasks falls behind. This imbalance among the four correctness correlations pushes S past 2. The effect appears in synthetic experiments as well as in real world data. They conclude that CHSH-type excesses can arise from the training dynamics of purely classical networks.

Critical strengths

The paper is clearly written. The protocol, the construction of S, and the proposed mechanism are stated explicitly, which makes the claims straightforward to evaluate. The question it raises, whether and in what sense trained classical models can produce Bell–CHSH-type statistics, is of some interest, given how central these inequalities are to understanding the limits of classical probability.

Critical weaknesses

The central weakness is that the reported “violations” do not carry nearly the same weight as in the original Bell inequality derivation. The CHSH bound of 2 only holds if Alice’s answer to her question doesn’t depend on which question Bob is asked. That lets the same four answers be reused across all four terms of S. Here, each pair of questions gets its own separately trained network, so Alice’s answer to the same question comes from a different network depending on what Bob was asked. Every term of S is then computed from its own unrelated outcomes, nothing ties the terms together, and S can go as high as 4 without anything non-classical going on. The authors partly acknowledge this, yet the abstract and conclusion still present the results as exceeding a classical bound, which seems misleading. Once this framing is set aside, what remains is that pairs containing an easy identity target are learned well before the pair of two XOR targets. The paper explains this by gradient competition between heads coupling hidden units under limited capacity, which leaves the harder pair behind. Even if this is the case, a detailed account of why one task pair is learned more slowly than others seems of limited interest.

Questions for authors

1.Since each setting pair is trained as a separate network, Alice’s outcome can depend on Bob’s setting, and S is bounded only by 4 under this protocol. Could the authors clarify what they view as non-classical about S > 2 in this setting? Do they agree that a single model with local heads would satisfy S ≤ 2 for any weights? 2. Given the above, what do the authors see as the main value of a detailed mechanistic account of when S exceeds 2 during training? 3. Which applications or downstream questions do the authors have in mind for this line of work?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.