acceptodds
Under review as a conference paper at ICLR 2027

A Crossover, Not a Constant: When Complex-Valued Networks Actually Help?

Abstract

Complex-valued neural networks (CVNNs) are widely motivated in domains where the information is carried in magnitude and phase. The paper asks when that motivation is borne out and finds that the answer is a narrow regime and not a general advantage. On a geometry-controlled seven-class RadioML 2018.01A task, covering phase-dominant, amplitude-dominant, and joint modulations across nine SNR levels, a complex model beats the best real baseline by pp against the Cartesian-only baseline set conventionally used in this literature, and by pp once a polar real baseline is included, then loses to it by pp under independent per-family tuning of the same search space, and by pp under a stable complex activation. What is left is not a fixed advantage but a crossover: complex arithmetic is advantageous up to pp in the marginal-SNR regime where phase estimates are noisy and is overtaken by an explicit polar representation as soon as the signal is clean. For all three configurations we test, the crossover is between and dB, regardless of the class composition and the sequence length. Three separate confounds inflate the apparent advantage outside this regime: (i) an impoverished baseline set, under which the capacity-matched Cartesian baselines become seed-unstable at the selected learning rate while polar and magnitude baselines do not; (ii) the selection rule, since matched-shared-trial selection measures robustness to a shared trial rather than peak performance; and (iii) training budget, since a pp complex advantage on phase-shift-keying data falls to pp at convergence. A fourth axis is not a confound but a design choice with confound-sized effects: complex activation moves accuracy by up to pp on amplitude-structured EEG tasks, and the sign of the effect is task-dependent. Especially, -gated activations recover amplitude structure but degrade phase-dominant tasks across RF, quantum, and EEG data, so magnitude-gating is a diagnostic to report rather than a prescription to follow. Finally, we reinterpret the complex layer architecture as a -equivariant subspace of a stacked-real two-channel layer architecture, not a gain but a reduction in expressiveness in line with symmetry, and test this interpretation by constructing a constrained real network and finding that it matches the complex network to a tolerance of pp while outperforming a matched unconstrained real network by to pp. In attributing results to complex-valued arithmetic, it is the symmetry constraint that matters, not the data type.

Primary area:
learning on graphs and other geometries & topologies
Keywords:
complex numbers, neural networks, representation learning

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_sXoKOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper presents a empirical investigation with useful insights into the evaluation of complex-valued neural networks. However, its methodological novelty is relatively limited, and the broader claims are not yet sufficiently supported by experiments across diverse architectures and real-world tasks.

Critical strengths
  1. Using the geometrically controlled RadioML task, the paper demonstrates that complex-valued networks offer advantages only within a narrow signal-to-noise ratio (SNR) range (approximately 2–6 dB) characterized by low SNR and significant phase noise; this crossover point remains stable across varying class compositions and sequence lengths. The authors identify three confounding factors—incomplete baseline sets, selection bias, and insufficient training budgets—that effectively explain why the advantages of complex-valued networks were overstated in previous literature.

  2. The paper reinterprets complex-valued layers as U(1)-equivariant subspaces formed by stacked real-valued dual-channel layers, positing that symmetry constraints—rather than the complex data type itself—are the true drivers of performance; this hypothesis is validated through experiments with constrained real-valued networks. Furthermore, the authors systematically analyze the task-dependency of complex activation functions, proposing that |z|-gating should serve as a diagnostic indicator rather than a fixed prescription.

Critical weaknesses
  1. Limited methodological innovation. The experimental analysis in this paper focuses on existing complex-valued networks rather than proposing new network architectures or learning algorithms. Furthermore, the two theoretical propositions rely primarily on established mathematical results; consequently, the originality regarding theory and methodology is relatively limited.
  2. Insufficient experimentation. The primary experiments utilize small-scale 1D CNNs and subsets of the RadioML dataset, while the EEG and quantum tasks rely mainly on synthetic data. The lack of validation on large-scale networks and across a broader range of real-world application scenarios makes it difficult to fully substantiate the paper's generalized conclusions.
  3. Uncertain generalizability of the SNR crossover phenomenon. Although the authors observed similar crossover phenomena across three experimental configurations, all were based on the same RF dataset. Without validation across diverse network architectures, datasets, and noise distributions, it remains unclear whether this phenomenon is universal.
  4. Minor question: The paper's structural layout is somewhat unusual—for instance, placing the "Related Work" section at the end—which differs from the official ICLR template.
Questions for authors

see above

2 comments

Thank you for recognizing the value of the evaluation controls and activation analysis. We agree that the paper should state its contribution and empirical scope more precisely.

  1. Methodological and theoretical novelty: We do not propose a new learning algorithm or network architecture, and we will make that explicit. The mathematical statements are established facts used to organize the comparisons, not new mathematical discoveries. The intended contribution is empirical: measuring how the apparent CVNN advantage changes when real-valued representations, selection rules, training budgets, activations, and SNR are controlled. We will foreground the resulting evidence and evaluation guidance, and reduce the prominence of the propositions.

We will also narrow the interpretation of the constrained-real experiment. It establishes that complex-linear kernels admit a real-coordinate implementation and provides evidence about one structured parameterization. The current readout difference and the absence of comparable alternative constraints prevent it from proving that U(1) symmetry causes the observed crossover.

  1. Experimental scale and real-world coverage: We agree that small 1D CNNs on RadioML subsets, together with synthetic EEG and quantum tasks, do not support broad claims about large-scale CVNNs or real-world applications generally. RadioML is our real-data RF benchmark; the other domains are controlled diagnostic settings. We will clearly distinguish these roles and restrict conclusions to the tested models and data. The controls are useful because they show that the sign and size of a reported advantage can change within a fixed evaluation setting. They do not establish how every architecture or application behaves. We will state the missing larger-model and independent real-dataset validations as limitations rather than imply that the existing controls provide them.

  2. Generalizability of the crossover: We agree that the crossover has not been shown to be universal. We will clarify that 2–6 dB brackets the observed sign change under shared-trial selection; it is not itself the interval in which complex models necessarily have an advantage. Additional analysis of existing independently selected runs supports a more qualified result. At −6/−2 dB, the complex-minus-polar gaps are +8.13/+7.36 pp for seven classes at length 128, +5.54/+7.04 pp for ten classes at length 128, and +12.36/+7.69 pp for seven classes at length 256. All three turn negative at 2 dB, placing their principal sign change between −2 and 2 dB under independent selection. The qualitative reversal therefore persists in these configurations, while the numerical crossover interval is selection-dependent. This strengthens the within-RadioML observation without extending it to untested datasets, architectures, or noise models.

  3. Organization and related work: Thank you for flagging the organizational difficulty. We will bring the relevant prior work earlier, introduce the baseline and training comparisons together, and organize the results around the central evaluation questions. We will also check the manuscript against the venue’s formatting requirements. The revised discussion will connect the symmetry interpretation directly to prior co-domain symmetry work, including Singhal et al. (CVPR 2022).

Reviewer_sXoKOpenReview ↗

Official Comment by Reviewer sXoK

I thank the authors for their clarification regarding Question 1. Furthermore, I appreciate their acknowledgment of the limitations in the experiments and the issues with the writing. Given that Questions 2, 3, and 4 remain unaddressed, I am maintaining my score.

Reviewer_JXn4OpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper studies counterfactual behavior in pretrained representation spaces, emphasizing that crossing a classifier’s decision boundary need not produce an endpoint supported by observed target-class data. It proposes a diagnostic framework that profiles encoder–classifier pairs using targeted search success, endpoint support, displacement, and optimization effort. It then relates these outcomes to boundary proximity, target-class k-nearest-neighbor radius, and a local tangent-residual curvature proxy.

The evaluation spans six datasets covering vision, medical imaging, text, multimodal learning, and single-cell biology. Accuracy- and loss-matched comparisons show substantial differences in counterfactual outcomes, while fixed-encoder experiments examine the contribution of classifier-head variation. Adding support and curvature features improves held-out prediction of search outcomes beyond boundary proximity. The proposed search method combines a target-neighbor penalty with curvature-dependent step sizes, increasing dataset-averaged supported success from 0.67 to 0.79. A single-cell case study additionally compares decoded expression changes against observed population-level response differences.

Critical strengths
  • The motivation is clear. The paper targets an important problem: whether the reported advantages of complex-valued neural networks reflect genuine architectural benefits or differences in evaluation settings. The authors show that the apparent advantage on RadioML largely disappears once stronger real-valued baselines and independent hyperparameter selection are used. This is central to my assessment because it highlights the importance of fair comparisons and provides useful guidance for future CVNN evaluations.

  • The paper identifies an SNR-dependent crossover between CVNNs and polar-representation real-valued networks, and the result is consistent across the tested class configurations and sequence lengths. The comparison with U(1)-constrained real-valued networks is also useful. Since the constrained model performs comparably to the CVNN, the gain seems to come from the symmetry constraint rather than complex arithmetic itself. I find this important because the paper doesn't stop at showing that the advantage shrinks, but also gives some insight into when CVNNs help and why.

Critical weaknesses
  • Overstated attribution to the symmetry constraint: Section 4.4 argues that the remaining performance difference is attributable to the U(1) constraint rather than the complex data type. However, the complex model uses a magnitude-based readout, whereas the constrained real network uses a linear readout, leaving an additional architectural difference in the comparison. Although the authors acknowledge this limitation, the main text still presents the result as proving the attribution. Moreover, the proposed explanation for the SNR-dependent crossover is not directly validated. In particular, the paper lacks per-SNR results for the constrained real network, making it difficult to determine whether the symmetry constraint alone can explain the same crossover behavior.

  • Limited generalizability of the SNR-dependent crossover: The SNR-dependent crossover is one of the paper’s main findings, but it is demonstrated only on subsets of RadioML using similar CNN architectures. Although the authors vary the class composition and sequence length, these experiments remain within the same dataset and model family. It is therefore unclear whether the crossover reflects a general property of CVNNs or is specific to the evaluated setting. The per-SNR results also rely on matched-shared-trial selection, the same rule the paper criticizes as biased, leaving open whether the crossover persists under independent per-family selection. Additional evidence from different datasets, architectures, or selection rules would strengthen this claim.

Questions for authors

Q1. Could the authors evaluate the U(1)-constrained real network with a modulus readout to better isolate the effect of data type? It would also be helpful to see whether this model reproduces the same crossover across SNRs, as that would more directly support attributing the crossover to the symmetry constraint.

Q2. Since the CVNN trails the polar real baseline overall under independent per-family tuning, does its low-SNR advantage still hold in that setting? Reporting per-SNR results under independent selection would clarify whether the crossover point and the magnitude of the low-SNR advantage are robust to the selection rule.

2 comments

Q1. Matched modulus readout & per-SNR constrained-real

Readout differences prevent isolating dtype: complex uses a complex linear head + modulus, whereas constrained-real uses a real linear head. Complex-linear convolution is exact in real coordinates, but full equivalence requires matching activations, biases, pooling, readout, initialization, and training. Shared-trial per-SNR accuracies diverge despite similar aggregates:

SNRComplexConstrainedPolar
-6 dB26.15%17.47%16.48%
-2 dB38.46%34.89%27.86%
2 dB53.79%55.27%49.34%
6 dB59.95%66.26%63.52%
18 dB62.42%66.92%67.36%

We will report full curves and withdraw claiming the aggregate agreement proves a symmetry-based crossover explanation. We will note a modulus-matched real model is unevaluated—and even exact agreement only shows implementation equivalence, still needing controlled comparisons to explain performance gains.

Q2. Independent family selection

A marginal-SNR gain persists, though the crossover shifts. Selecting each trial by 5-seed mean validation accuracy over SNR (without test or per-SNR inspection) yields held-out test differences (Complex minus Polar-real, pp):

Config-6 dB-2 dB2 dB6 dB18 dB
7 cls, L=128+8.13+7.36-2.31-5.11-5.88
10 cls, L=128+5.54+7.04-0.50-3.50-5.27
7 cls, L=256+12.36+7.69-1.65-8.46-9.89

We will qualify the abstract and text: (1) reversal persists in all three configs, shifting from 2–6 dB (shared-trial) to -2–2 dB (independent); (2) seed-averaged point estimates on a discrete grid neither locate an exact threshold nor prove per-bin significance, and near-chance accuracy at the lowest SNRs means "low-SNR advantage" does not apply at every low-SNR level; (3) for 7 classes (L=128), the -0.58 pp aggregate gap reflects high-SNR deficits outweighing -6/-2 dB gains. We will report aggregate and SNR profiles with actual capacities and selected hyperparameters.

Scope & generalization: We will restrict crossover claims to tested RadioML configs (not universal across datasets, architectures, or noise models), clarify synthetic EEG/quantum tasks isolate activation sensitivity and budgets rather than real-world crossovers, and frame noisy phase estimates strictly as a hypothesis.

Thank you for the additional experiments results. The rebuttal addresses my main concerns regarding to Q2. The experiment demonstrates that the SNR-dependent crossover persists under independent per-family selection. I find this to be a meaningful and convincing result.

However, my concern regarding Q1 remains partially unresolved. I appreciate the authors' acknowledgment of the limitation, but the requested matched modulus experiment remains unevaluated. Therefore, the role of the U(1) constraint to the crossover remains unclear. Given these remaining concerns, I will maintain my original rating of 2.

Reviewer_LszpOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary

The paper tackles an important question of whether complex-valued neural networks are actually helpful for complex-valued data, and if so, what property helps. The authors run careful controlled experiments and show that the SNR of the data has a huge effect (i.e., complex nets are helpful for low SNR data but real-valued nets are better in high SNR regimes). They also study which properties of complex-valued nets are actually responsible for performance, and claim that the U(1) symmetry is the core property.

Critical strengths

The paper studies a genuinely important problem: empirically and rigorously verifying whether complex-valued neural nets are actually needed. As the authors point out, many of the gains shrink significantly (or go away entirely) when properly tuning the baselines. Figure 2 in particular is an interesting finding, and worth studying further.

Critical weaknesses

My main concern is the readability of this paper and some of the arguments made by the authors. I'm willing to increase my rating if the authors rewrite the paper to be clearer and address my concerns.

The relevance of Section 2.1 is unclear. Proposition 2 states the consequence of Liouville’s theorem that a nonconstant complex activation cannot be both entire and globally bounded. However, the manuscript does not establish how this result informs the comparison between real- and complex-valued networks or predicts the observed performance differences. In particular, the asserted connection between this theorem and the activations’ dependence on magnitude requires further explanation. Since the experiments evaluate existing nonlinearities, the authors should clarify what analytical insight this section contributes to interpreting those experiments. The subsequent paragraph explains that nonholomorphic activations can be trained using real differentiation, but its relevance to the paper’s empirical questions is also insufficiently developed.

Section 4.3 lacks the context needed to assess its argument. The section opens by stating that prior literature attributes real-baseline instability to excessively large first-step gradients, but provides no citations identifying the relevant studies or explanations. It then introduces gradient-clipping results on a three-class configuration and attributes failure to a dead ReLU readout, without adequately introducing that configuration or explaining its relationship to the main RadioML experiments. The authors should identify the prior claims being examined, describe the relevant architectures and experimental conditions, and explain which evidence supports the proposed failure mechanism. As written, it is difficult to determine the scope of the observation or how it supports the paper’s broader conclusions.

The activation discussion is insufficiently connected to the central question. Section 5.2 reports substantial performance differences among nonlinearities, particularly on the synthetic EEG tasks. However, the manuscript should explain how these findings inform its principal claim about the SNR-dependent advantage of complex networks. It should also clarify what is meant by “magnitude-blind”: the absence of an explicit modulus operation does not imply that an activation’s output is insensitive to input magnitude. More generally, the authors should explain what these observations add to existing design considerations concerning amplitude and phase, and how they support the proposed interpretation of the crossover.

The experimental presentation makes the comparisons difficult to interpret. Section 3 introduces several baseline families and evaluation axes, but the architectural and training differences among the families are not described together in sufficient detail. The current presentation requires substantial reconstruction by the reader to connect the reported results to the central argument.

I am not convinced that Section 4.4 establishes the proposed explanation for the low-SNR advantage. Proposition 1 correctly identifies the equivalence between complex multiplication and a real two-channel linear map restricted to the subspace (aI+bJ). Consequently, implementing convolution with real kernels restricted to this subspace implements the same complex-linear operation in real coordinates, which does not actually distinguish between complex vs real arithmetic (please correct me if I am misreading this argument). The comparisons do not establish whether the observed benefit is specific to the phase-equivariant structure of (aI+bJ), or could also arise from other restrictions of comparable dimensionality. In particular, the authors should explain how their evidence accounts for the crossover itself, rather than only showing that a constrained model can outperform an unconstrained baseline.

Finally, the symmetry discussion should engage more directly with prior work on complex-valued network design. For example, Co-Domain Symmetry for Complex-Valued Deep Learning (Singhal et al., CVPR 2022) explicitly studies complex-scaling symmetries and develops equivariant and invariant operations. Although that work considers complex scaling more broadly than the (U(1)) phase rotations discussed here, it is directly relevant to distinguishing complex arithmetic from symmetry-respecting architecture design. The manuscript should explain how its observations relate to this literature and what additional understanding its experiments provide.

Questions for authors

Can you please clarify your aI+bJ (U(1) symmetry) experiment and confirm that it doesn't simply re-implement complex arithmetic?

1 comment

We will revise the paper to separate empirical findings, mathematical equivalences, and proposed explanations:

  1. Liouville’s theorem: It predicts neither real-vs.-complex accuracy, SNR crossover, nor amplitude use—only that boundedness and entire holomorphicity exclude nonconstant activations. Moving this condensed note from the empirical argument, we state nonholomorphic activations train via real/imaginary derivatives, unlinked from accuracy.

  2. Gradient clipping: Introduced upfront as a separate 3-class RadioML control (BPSK/QPSK/8PSK; length 128; 8 SNRs, -10 to 18 dB; 16 trials; 6 seeds; unclipped vs. norm 1.0), distinct from the 7-class task, and citing or cutting unsupported prior claims. Clipping cannot verify dead ReLUs—our real model has conv ReLUs and a linear head—so we will fix "dead ReLU readout," frame it as a hypothesis, and restrict conclusions to stability.

  3. Activations & diagnostics: "Magnitude-blind" is inaccurate as CReLU(rz)=rCReLU(z)\text{CReLU}(rz)=r\text{CReLU}(z) (r≥0r\ge 0) depends on magnitude; we will use componentwise rectification vs. explicit modulus-dependent gating, decouple activation sensitivity from the RF crossover, and treat synthetic EEG/quantum tasks as diagnostics, not crossover validations.

  4. Comparisons: One table will list input representation, conv widths/depth, activation, pooling, readout, parameters, compute, optimizer, search budget, and selection rule. We will note the readout mismatch (complex linear + magnitude logits vs. constrained-real linear head), clarify independent selection equalizes search effort, not capacity, and show shared-trial (complex-selected) alongside independent results.

  5. aI+bJaI+bJ & equivariance: For z=x+iy,W=A+iBz=x+iy, W=A+iB, complex multiplication is [x;y]↦[[A,−B],[B,A]][x;y][x;y]\mapsto[[A,-B],[B,A]][x;y] (coordinate equivalence, not a new mechanism). Since our constrained-real model differs in readout, lacks comparable-dimension alternative constraints, and trails at -6 dB (26.15% complex vs. 17.47% constrained-real, 16.48% polar-real), it does not isolate the low-SNR cause. Componentwise CReLU and nonzero biases break phase equivariance, so we use complex-linear kernel structure over whole-network U(1)U(1) equivariance; symmetry claims need matched readouts and alternative constraints.

  6. Co-domain symmetry: We will cite Singhal et al. (Co-Domain Symmetry for Complex-Valued Deep Learning, CVPR 2022) in our kernel analysis, avoid claiming novel symmetry design, and frame our work as controlled evaluation of representation, selection, budget, activation, and SNR dependence.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.