acceptodds
Under review as a conference paper at ICLR 2027

Structural Inductive Biases for Quantum Sequence Modeling on NISQ Hardware

Abstract

Quantum sequence models combine architectural choices in information routing with the constraints of parameterized quantum circuits and near-term hardware, yet these effects are rarely compared under a common evaluation protocol. We empirically characterize quantum recurrent (QRNN), convolutional (QCNN), and projected-attention (QPA) models from controlled Boolean logic to constrained language modelling and physical IBM quantum processors. On Boolean tasks, multi-seed decision-geometry analysis reveals distinct but multi-axis architecture-dependent behaviour rather than a single smooth-versus-fragmented ordering. Controlled re-uploading ablations change both spectral structure and fit reliability, while a restricted single-query QPA admits a conditional additive decomposition that motivates the cumulative pooling used in the evaluated model. On the TALES language benchmark, classical baselines remain stronger on most generative metrics, and the relative ordering of the quantum architectures changes across tasks. Finally, we distinguish zero-shot execution of simulator-trained parameters from direct QPU optimisation. The two regimes need not agree: QRNN and QCNN both show measurable descent under multi-sample SPSA, while their hardware-training behaviour differs substantially despite comparatively modest zero-shot degradation. Overall, the results characterize architecture-dependent trade-offs in representation, optimisation, and physical execution rather than a uniform quantum advantage.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
quantum machine learning, quantum sequence modeling, quantum neural networks, inductive biases, quantum natural language processing, NISQ hardware

Reviews

Among:
Shape:
1234this paper, Bottom 2%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.33 over 3 reviews, above 2% of the others.
Reviewer_KTGKOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

This paper investigates the structural inductive biases of three low-capacity quantum sequence architectures: Quantum Recurrent Neural Networks (QRNNs), Quantum Convolutional Neural Networks (QCNNs), and Quantum Projected Attention (QPA) models. The evaluation encompasses controlled Boolean tasks, DisCoCat classification, the TALES synthetic language modeling benchmark, zero-shot execution on IBM quantum processors, and direct QPU optimization.

Critical strengths

The experiments in this paper are very thorough.

Critical weaknesses
  1. In my view, this paper fails to contribute new insights to the quantum machine learning community; instead, it resembles a set of experiments conducted on previously proposed quantum neural network architectures, accompanied by some rather obvious analyses. Consequently, I am inclined to recommend rejection.
  2. AI-assisted writing in the paper results in a text that appears overly defensive, making it difficult for readers to grasp the core ideas.
Questions for authors
  1. While the scope of the experiments is quite broad, the depth and attention to detail fall short of the standards expected for machine learning papers; for instance, could you explain the architectures and specific configurations used in these two hardware experiments? Since factors such as architecture, device noise, width, depth, and connectivity cannot be decoupled from one another, it is difficult to demonstrate that the difference in hardware trainability between QRNNs and QCNNs—which the paper aims to highlight—stems specifically from structural bias. Moreover, even if such a demonstration were possible, I believe it would hold little practical academic significance.
  2. I suggest reducing the amount of AI-generated experimental content and text in your paper, as this tends to result in an overly defensive tone, as well as theoretical proofs and experimental details that are superficial and limited in scope. While AI might appreciate a paper that is rich in content and honestly reports negative results, this particular paper offers little of value to readers in the quantum machine learning community.
2 comments
AuthorsOpenReview ↗

Response to Reviewer KTGK

We thank Reviewer KTGK for recognizing the thoroughness of our experimental scope across Boolean logic, DisCoCat, TALES, zero-shot QPU inference, and direct QPU training. The revision directly addresses both of your concerns by clarifying our hardware comparison and sharpening the presentation of our core findings.

Q1. What are the hardware configurations, and what does the physical comparison establish?

First, we summarize the exact circuit architectures and hardware configurations from §4.3 and App. E (Tables 7–11, Figs. 14–17):

  • QRNN on TALES-light (E=3E=3, Figs. 14 & 15a, Table 10): Uses 3 token-embedding qubits and 3 persistent hidden-register qubits (6 physical qubits total). At each of the 6 sequence steps, a token is encoded via RyR_y rotations on the embedding register, entangled with the hidden register via CNOTs, and processed by a temporally weight-shared recurrent PQC block (Ry,RzR_y, R_z, CNOT chain), followed by a prediction PQC and Z,ZZZ, ZZ Pauli readout into a classical linear layer. Total trainable parameters: 90 quantum (shared across steps) + 168 classical = 258. Circuit resources: 100 logical gates, 34 2Q gates, 2Q depth = 22 (mapped onto heavy-hex ibm_strasbourg with zero SWAP overhead: 322 native gates, 34 2Q gates, 2Q depth = 22, Table 11).
  • QCNN on TALES-light (E=3E=3, 2-3 pooling, Figs. 16 & 17a, Table 10): Encodes all 6 context tokens in parallel across 18 physical qubits. Layer 1 applies 2-token convolutional blocks and traces out half the qubits; Layer 2 applies a 3-register pooling block into a 3-qubit readout register. Total trainable parameters: 148 quantum + 168 classical = 316. Circuit resources: 180 logical gates, 42 2Q gates, 2Q depth = 10 (mapped onto heavy-hex ibm_brussels with zero SWAP overhead: 579 native gates, 42 2Q gates, 2Q depth = 10, Table 11).

The reviewer is correct that the two physical runs executed concurrently on ibm_strasbourg and ibm_brussels, with different widths, depths, and backend noise. This is consistent with the role of the hardware study in our evaluation: to establish end-to-end physical training feasibility and quantify the resource requirements of two distinct sequence-model implementations under realistic deployment conditions. The experiments demonstrate complete optimisation loops driven by finite-shot QPU measurements and provide auditable circuit width/depth, transpilation, wall-clock, and QPU-time accounting. Because architecture and backend vary together, we do not use the endpoint difference to claim intrinsic QRNN-versus-QCNN noise robustness.

Architecture-level conclusions instead come from the controlled multi-seed simulations summarized in the General Response: the parameter-group interventions, restricted-QPA analysis and pooling ablation, and matched 256-shot QRNN clean/noisy controls.

Q2. How has the presentation been revised to make the contribution clearer?

We appreciate your candid feedback on the writing style. We agree that some repetitive AI-assisted formulations and defensive qualifiers, particularly at the ends of sections, obscured the main takeaways instead of clarifying their scope. In the revised manuscript, we removed this repetitive phrasing throughout §1–4, consolidated the relevant scope conditions and limitations in §5, and reorganized the presentation around three direct findings:

  1. Where adaptation occurs: QRNN, QCNN-R, and cumulative QPA rely differently on embedding and circuit adaptation under matched parameter-group interventions (Table 1).
  2. Which structural restrictions matter: single-final-query QPA has a precise conditional-additivity obstruction removed by cumulative pooling (App. C.2), while controlled re-uploading changes fit reliability and extended-domain geometry (Table 5).
  3. What survives in language and hardware settings: strong count baselines identify the local structure in TALES (Table 3), readout-only controls confirm the necessity of upstream representation adaptation (§4.3), and zero-shot and direct-QPU experiments provide concrete physical deployment and resource benchmarks.

This revision makes the novelty, evidence, and scope of each result visible where it is introduced, while keeping limitations together rather than interrupting each experimental conclusion.

The response did not address my concerns. I am keeping my rating as is.

Reviewer_Z4vWOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper compared recurrent, convolutional and projected attention, three types of quantum sequence models. The experiment covers Bool logic, language synthesis tasks and quantum devices. Authors examined decision geometry, repeted encoding inputs, and additive limits on restricted attention. As a result, model performance varies over tasks, and overall, classifc baseline models perform better on language tasks. Experiments on hardware shows simulator-trained inference can be separated from direct optimization on hardware. Both recurrent and convolutional models were improved. however, under the experimental conidtions and hardware settings, the results varies.

Critical strengths

The paper studied a question closely relevant to machine learning, namely how a model architecture can impact the learned behaviour. Authors faithfully reported the negative results, and use multiple random seeds. Re-uploading controls help provide some understanding about mechanisms. It is also useful to look into restructed attention. It avoids overstatements by separating hardware inference from training, making the experimental claims more clear.

Critical weaknesses

The paper included many experimental observasions, but its main scientific conlcusion yet is unclear. Different architectures can perform differently on different tasks, and this is quite a common sense, not a surprise. More importantly, we want to know which architecture caused these behaviours, and why this explanation can work under certain conditions. Bool tests give some insights into these questions. However, geometric analysis often depends on the learned embeddig coordinates. Several results in summary is simply based on very few successful runs. This reduces the underlying link between results and architectures.

Other experiments seem not filling this gap. Language tasks are small, and synthesis based. Hardware training is compared based on tests on different devices. These experiments can prove it is feasible, but not sufficient to establish reliable evidence over architectural effects. Authors carefully discussed about these limits, though such a cautiousness may not replace an expected confirmable research result. For a top ML paper, I would hope to see a fully evidenced explanation.

Questions for authors

My questions for authors:

  • About inductive bias, which is/are the most strong findings? Please put it into a verifiable claim, indicating which behaviours can be tested by this prediction, instead of describing the difference after the training.
  • Will the geometric differences present after fixing or normalizing embeddings with comparable optimization budgets? That would help establish if routing structure causes the effect.
  • Besides training feasibility, what can be established from hardware comparison? Different models use different devices and circuits. If no controlled tests can lead to a meaningful archtectural conclusion, does the remaining evidence supports the main contribution. Please add what you hope other ML researchers to continue in the future work from your work.
1 comment
AuthorsOpenReview ↗

Response to Reviewer Z4vW

We thank Reviewer Z4vW for pushing us to anchor the contribution in explicit, testable claims about structural inductive bias rather than a broad list of empirical differences. We designed our new 10-seed controls specifically to answer your three questions.

Q1. What is the strongest predictive finding about inductive bias, and which experiment tests it?

Our central finding, connecting Fig. 2, App. C.2, and the direct interventions in Table 1, is that routing structure predicts where task adaptation occurs: reliable QRNN and QCNN-R fitting depends strongly on joint embedding/circuit adaptation, whereas cumulative QPA retains most fitting reliability through embedding adaptation over a fixed quantum feature map. We test two coupled predictions:

1. Once cumulative pooling removes single-query additivity, QPA adapts primarily through its embeddings. Restricted single-final-query QPA is conditionally additive over earlier positions and cannot implement the operator-dependent AND/XOR switch (0/10 endpoints). Cumulative pooling restores the missing intermediate interaction (9/10). Because token mixing then occurs through classical products of token-local quantum expectations, we predict that embedding adaptation over a frozen circuit should retain most fitting reliability. Across 10 matched seeds, it retains 8/10 endpoints (9/10 under noiseless evaluation; loss 0.127 vs. 0.079 jointly).

2. Coherent state routing relies on repeated encoding and joint embedding/circuit adaptation. QRNN carries history through a persistent register, while QCNN-R routes repeatedly encoded tokens through spatial entangling/pooling blocks. Freezing either group reduces QRNN from 10/10 joint fits (0.128 loss) to 1/10 (0.353/0.428), with joint training lower in every seed; QCNN-R similarly falls from 7/10 to 2/10 and 0/10. Complementarily, re-uploading raises single-pass QFF/QCNN fit counts from 2/10 to 6/10–7/10 and increases DC-corrected extended-domain low-frequency energy (55.155.1\\%\rightarrow68.4\\% and 70.570.5\\%\rightarrow79.1\\%). The QFF comparison is strictly parameter matched (96 each).

Q2. Do the findings survive controlled embeddings and comparable budgets?

Yes. Table 1 settings share initial weights, data order, 1,024-shot SPSA perturbations, and architecture-specific step budgets. Canonical fixed ±π/2\pm\pi/2 embeddings improve QCNN-R circuit+head training from 0/10 to 2/10, still below joint training (7/10), showing coordinate sensitivity without removing the joint-adaptation result. After DC removal, both extended-domain re-uploading shifts persist (QCNN: 70.570.5\\%\rightarrow79.1\\%; matched QFF: 55.155.1\\%\rightarrow68.4\\%), while QPA moves from 84.984.9\\% to 52.652.6\\%, separating its class imbalance from non-DC spectral energy.

Q3. What does the hardware comparison establish, and what can ML researchers build on?

The physical study establishes an operational baseline, not a cross-device ranking. Concurrent QRNN/QCNN runs on ibm_strasbourg and ibm_brussels represent complete deployment configurations combining architecture, circuit footprint (6 vs. 18 qubits), transpiled layout, finite-shot optimiser, and backend. They demonstrate closed hybrid training loops on current QPUs and quantify loss trajectories, circuit resources, wall-clock cost, and QPU usage.

Architecture-level conclusions come from controlled simulation. At a fixed 256-shot budget, the matched QRNN control reaches 5.14±1.025.14\pm1.02 PPL clean and 5.61±0.575.61\pm0.57 under generic depolarizing noise (paired difference +0.47±0.99+0.47\pm0.99), directly quantifying the incremental noise penalty without conflating it with historical 1,024-shot runs.

For future work, the paper contributes three reusable tools: a parameter-group intervention for identifying where adaptation occurs; a formal and empirical test for aggregation bottlenecks in projected attention; and an auditable protocol connecting logical architecture, transpiled resources, finite-shot optimisation, and physical execution. These tools can be applied to larger sequence tasks, alternative circuit families, and newer processor topologies without assuming that the rankings observed here will transfer unchanged.

Reviewer_dnQqOpenReview ↗
Rating
1
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper compares three hybrid quantum sequence architectures under one protocol: a quantum RNN (QRNN, a persistent hidden register with fresh token injection each step), a quantum CNN (QCNN, parallel token encoding followed by hierarchical pooling/trace-out), and quantum projected attention (QPA, token-local quantum Q/K/V features with classical unnormalised dot-product attention and cumulative pooling). The evaluation has three levels:

Boolean AND/OR/XOR. The authors interpolate between learned operand embeddings on a 101×101 grid and summarise the XOR decision surface with three descriptors (region imbalance, point-reflection asymmetry, low-frequency FFT energy). They also report successful-fit counts over 10 seeds. Re-uploading ablations (QCNN vs QCNN-R, QFF vs QFF-R) raise fit rates (2/10→7/10 and 2/10→6/10) and shift the descriptors. App. C shows that single-final-query unnormalised attention with a linear head is additively separable over earlier positions given the query, so it cannot solve the joint AND/XOR task. Cumulative pooling removes this obstruction, and a 2×2 ablation (pooling × softmax) shows only CP/no-softmax fits reliably. Language tasks. DisCoCat MC/RP (10 seeds) and a new 7-token synthetic grammar, TALES (5 seeds), are compared against ~400-parameter classical MLP/RNN/CNN/Transformer baselines. Classical models win on most TALES metrics, and the ordering among the quantum models changes from task to task. IBM hardware. Simulator-trained parameters are run zero-shot on IBM QPUs and lose a modest amount of performance. A single direct-QPU SPSA training run per model on a reduced TALES-light reaches PPL 4.82 for QRNN (sim 4.39) and 8.65 for QCNN (sim 5.30). The main stated conclusion is that architecture causes trade-offs rather than a universal ordering, and that zero-shot hardware fidelity does not predict hardware trainability.

Critical strengths
  • The paper repeatedly separates what is shown from what is not (no quantum advantage, no long-horizon memory advantage, no intrinsic noise robustness). Classical baselines are included, and the paper says plainly when they win. This candour is more than many QML papers offer.
  • Simulator, zero-shot QPU and direct-QPU training are kept as three distinct settings. This is a useful framing for the community, and the point that inference fidelity ≠ trainability is worth making.
  • The derivation in App. C.2 is short and right. The AND-vs-XOR contradiction at b=1 is clean, and the 2×2 CP/softmax ablation (Fig. 6) gives a concrete design lesson. The authors also say correctly that the result says nothing about softmax (App. C.3).
  • The parameter-matched QFF/QFF-R pair is a sensible attempt to separate repeated encoding from added depth, and fit reliability is reported next to conditional geometry statistics instead of hiding failed seeds.
  • SWAP-free heavy-hex layouts, logical vs transpiled resource tables (Tables 8–9), and wall-clock/QPU-time accounting are useful to practitioners.
  • Multi-seed reporting throughout, with seed-level landscapes for every configuration in the appendix.
Critical weaknesses
  • Contribution 3 and the abstract both claim that zero-shot fidelity does not predict hardware trainability. The evidence is one QRNN run on ibm_strasbourg and one QCNN run on ibm_brussels, each for 20 epochs. App. E.2 admits that backend and architecture are confounded. A single SPSA run at 256 shots has large variance, so the QCNN endpoint of 8.65 could easily be one unlucky trajectory. The paper also does not say how the "Sim" curves in Fig. 3 and Table 3 were trained (SPSA at 256 shots, or exact/parameter-shift gradients). If they were not trained with SPSA, the sim-vs-hardware gap mixes optimiser and noise effects. The missing control is a QCNN trained in simulation with the same SPSA settings (p=8, σ=0.05, 256 shots), and ideally under a noise model of each backend. Fig. 13a provides this only for QRNN. Without it, the QCNN failure cannot be attributed to the hardware at all, let alone to architecture.
  • L includes the DC component (Eq. 16–17), and the mask radius is about 2.5 frequency bins. For a surface ΔP ∈ [−1, 1], DC power scales with (mean ΔP)², which grows with decision-region imbalance. Table 4 bears this out: the configurations with the highest L also tend to have the highest imbalance (QPA: I=76, L=84.9). Re-uploading raises both together (QCNN→QCNN-R: I 5→26, L 70→82; QFF→QFF-R: I 23→40, L 58→79). The claim that re-uploading "produces the clearest spectral shift" (§4.1) may therefore just be the imbalance shift counted a second time. The authors should exclude DC, or mean-centre ΔP before the FFT, and report the correlation between the metrics.
  • The grid spans [−3, 3]² in interpolation coordinates, while the Boolean points sit at the False/True endpoints. If these are 0 and 1, the region between training points covers under 3% of the grid, and every metric is dominated by extrapolation. With Ry encodings, extrapolation mostly reflects the periodicity of the encoding, not anything learned. The point-reflection metric (Eq. 13) reflects about (0, 0). XOR's truth table is point-symmetric about the midpoint of the four Boolean inputs, so unless False/True sit at ±1 the metric does not test the symmetry the text uses to justify it (B.2). The paper should state the exact coordinates of the Boolean points and justify the grid range. B.2 itself notes that coordinates are not comparable across models. Together these make the cross-architecture geometry comparison, the paper's first contribution, hard to interpret.
  • The joint Boolean task has 12 examples (accuracies come in steps of 1/12). A 106-parameter MLP reaches 100% in only 6/10 seeds after 1000 steps, and a 202-parameter Transformer in 0/10. For networks of this size, fitting 12 points should be close to trivial with a tuned learning rate and optimiser, which points to a fixed, untuned hyperparameter. The statement that QRNN "exhibits the highest fit reliability" (§4.1) depends on this. Training budgets also differ by family (200/500/1000 steps). Similar concerns apply to TALES: the paper gives no hyperparameters for the classical baselines and no evidence of tuning for either side.
  • The perfect-model PPL is 4.52, the classical models score 6.06–6.43 and the quantum models 6.44–7.92. There is no n-gram (unigram/bigram/trigram) baseline and no "linear head over bag-of-embeddings" baseline. These would show how much any model, quantum or classical, uses sequence structure beyond local statistics. The hybrid models' classical component (252 parameters) is a linear readout, so an ablation with frozen or randomised circuit parameters would also show how much the quantum processing contributes.
  • Many of the differences the text reads as reordering fall within one standard deviation with 5 seeds. Examples: QCNN 6.44±0.26 vs CNN 6.43±0.40 PPL; minimal-pair accuracies of 69–82% with SDs of 5–11 points, on 50 pairs; QPA "highest quantum MP mean" at 73.0±11.2. The paper reports no confidence intervals or tests. The zero-shot rows sometimes improve on the simulator (QCNN MP 69.2→72.4; QPA MP 73→74), which signals evaluation noise of this order. With small eval sets (100 held-out sentences, 50 probes, 31 RP test sentences), the "no single ordering" conclusion may partly reflect the noise floor rather than architecture. *The paper's main findings are mostly negative or null: no model dominates, the ordering is task-dependent, classical models are better, and the hardware results are not conclusive. These are hedged to the point that little falsifiable insight remains. The abstract and conclusion consist largely of qualifications. *The one formal result (App. C.2) is elementary: single-query linear attention with a linear head is additively separable given the query. This is standard and not specific to quantum circuits. Moreover, in the evaluated QPA the attention itself is computed classically from measured token-local Pauli features (Fig. 1). The quantum part is therefore a per-token feature map, and the "quantum attention" framing overstates what is quantum. *All models are extremely small (36–162 quantum parameters, ≤6-token contexts). It is unclear which lessons would carry over to any scale where the ICLR audience would care. *Related comparative studies of QML architectures and benchmarking efforts (for example, systematic benchmarks of quantum classifiers against classical baselines) are not discussed. Neither are recent quantum sequence models beyond the few cited.
Questions for authors
  1. The paper does not say whether any error mitigation (readout mitigation, dynamical decoupling, twirling) was used in the zero-shot runs. This matters a lot for interpreting the small degradation.
  2. The QCNN/QCNN-R pair is not parameter-matched (54 vs 66 quantum parameters), unlike QFF/QFF-R. The text should not describe it as isolating re-uploading.
  3. In Table 1, the "Perfect Classification 100.0*" footnote for RP contradicts B.3: since all four both-OOV examples carry label 0, a deterministic classifier can get them right.
  4. §3 says simulator training uses 1024 shots, yet Table 1 groups simulator rows apart from "autograd/backprop". The paper should explain how gradients are computed in simulation (shot-based parameter shift? exact statevector?).
2 comments
AuthorsOpenReview ↗

Response to Reviewer dnQq (part 1/2)

We thank Reviewer dnQq for an exceptionally careful and technically engaged review. We particularly appreciate the reviewer’s recognition of six strengths that are central to the paper: its candid reporting of classical baselines and negative results; the explicit separation of simulator training, zero-shot QPU execution, and direct physical optimisation; the clean restricted-QPA derivation and accompanying pooling/softmax ablation; the parameter-matched QFF re-uploading control; the practical heavy-hex layouts and logical/transpiled resource accounting; and the complete multi-seed reporting of fit reliability and decision landscapes. These observations capture both the methodological purpose and the practical value of the study. We are equally grateful for the reviewer’s precise technical suggestions, which motivated the corrected spectral analysis, stronger baselines, parameter-group interventions, and clearer execution details addressed below.

Q1. Are the direct-QPU results confounded by backend, optimiser, and finite-shot effects?

The reviewer is correct that the QPU runs executed concurrently on ibm_strasbourg and ibm_brussels, with different widths and depths (6 qubits/depth 22 for QRNN; 18/depth 10 for QCNN). Their role is an operational baseline: demonstrating closed finite-shot training loops on contemporary QPUs and quantifying loss, circuit, transpilation, wall-clock, and QPU-time costs. Because architecture and backend vary together, we do not infer architecture-independent noise robustness from their endpoints; architecture-level conclusions come from controlled multi-seed simulation.

We also clarify a key protocol detail: in both simulator and hardware runs, embeddings and circuit parameters use finite-shot multi-sample SPSA (p=8p=8) + Adam, while the linear head uses analytic gradients + Adam (not exact statevector or parameter-shift gradients).

We added five matched 256-shot QRNN seeds on TALES-light (identical initial weights, data order, and SPSA perturbations). At epoch 20, clean simulation reaches 5.14±1.025.14\pm1.02 PPL vs. 5.61±0.575.61\pm0.57 under depolarizing noise (p1=0.001,p2=0.01p_1=0.001, p_2=0.01), directly quantifying the incremental noise penalty (+0.47±0.99+0.47\pm0.99 PPL with high seed variance). Multi-seed noisy SPSA simulation of the 18-qubit QCNN was computationally prohibitive within the rebuttal window, so we scope QCNN's 8.65 endpoint strictly as an operational baseline rather than attributing it to hardware noise or architecture alone.

Q2. Does removing DC invalidate the geometry findings, and where are the Boolean points?

In our grid construction (e(x)=eTrue+eFalse2+xeTrue−eFalse2e(x) = \frac{e_{\text{True}}+e_{\text{False}}}{2} + x\frac{e_{\text{True}}-e_{\text{False}}}{2}, App. B.2), the discrete Boolean inputs sit at (−1,−1),(−1,+1),(+1,−1),(+1,+1)(-1,-1), (-1,+1), (+1,-1), (+1,+1), placing (0,0)(0,0) at the exact midpoint of the XOR truth table. These coordinates parameterize linear interpolation between learned embeddings rather than raw RyR_y gate angles. Thus, [−1,1]2[-1,1]^2 is pure interpolation and [−3,3]2[-3,3]^2 probes extrapolation across the extended surface. We now state this explicitly in §3, Fig. 2, and App. B.2.

We recomputed all spectral scores after mean-centring ΔP\Delta P and excluding DC (updated Fig. 2a and Table 5):

  1. Spatial asymmetry SS and decision-region imbalance II are unchanged.
  2. Re-uploading still increases extended-domain low-frequency energy on [−3,3]2[-3,3]^2: rising from 70.570.5\\%\rightarrow79.1\\% for QCNN→\rightarrowQCNN-R and from 55.155.1\\%\rightarrow68.4\\% for the strictly parameter-matched QFF→\rightarrowQFF-R control (96 parameters each).
  3. Why QPA changed: Removing DC drops QPA from 84.984.9\\% to 52.652.6\\% because its uncentred score reflected its imbalanced decision region (I=76.0I=76.0\\%), separating QPA's localized-island geometry from the extended-domain re-uploading shift. App. B.2 additionally reports a valid non-DC inner-domain diagnostic; its ordering differs, so our spectral claim is explicitly scoped to the extended domain rather than universal interpolation smoothing.

Q3. Are the Boolean conclusions artifacts of untuned classical baselines or unequal budgets?

We evaluated 200 classical learning-rate configurations. Tuned fit rates reach 9/10 (CNN), 7/10 (MLP), and 7/10 (RNN); Table 5 retains the original common-LR cohort only so its geometry means correspond to the archived surfaces. The 202-parameter 1-layer Transformer (learned positions, order-sensitive readout, no input collisions) remains at 0/10 after 5,000 steps, whereas a 338-parameter 2-layer control reaches 5/5—a depth/capacity threshold of this compact baseline, not a general limitation of attention.

The matched 10-seed intervention directly tests what drives quantum fitting: QRNN reaches 10/10 jointly vs. 1/10 when either embeddings or circuit weights are frozen; QCNN-R reaches 7/10 jointly vs. 2/10 and 0/10 (2/10 with canonical fixed embeddings).

AuthorsOpenReview ↗

Response to Reviewer dnQq (part 2/2)

Q4. Do TALES models exploit structure beyond local nn-gram statistics and the classical linear head?

We added smoothed Unigram, Bigram, and Trigram count predictors fitted strictly on TALES training counts (Dirichlet smoothing α=1\alpha=1, using no test or generator probabilities), along with a 5-seed Head-Only (Frozen Quantum) control on TALES-light where the embeddings and quantum circuit are frozen at initialization and only the linear readout head is trained:

TALES Count BaselineTest PPL ↓\downarrowKL Div. ↓\downarrowMinimal-Pair Acc. ↑\uparrowSemantic Acc. ↑\uparrowGrammatical Acc. ↑\uparrow
Unigram (1-gram)27.621.7656.0%20.0%20.0%
Bigram (2-gram)6.250.3586.0%88.0%100.0%
Trigram (3-gram)6.000.2386.0%92.0%100.0%

Bigram and Trigram perform comparably to the ~400-parameter classical (6.06–6.43 PPL) and quantum (6.44–7.92 PPL) models, showing how much aggregate TALES performance is captured by local transitions. Conversely, on TALES-light, readout-only training over the initial frozen representation yields 18.20±0.8018.20\pm0.80 PPL for QRNN and 18.08±0.5018.08\pm0.50 for QCNN, versus 4.39±0.724.39\pm0.72 and 5.30±0.795.30\pm0.79 jointly. It also yields 0/10 Boolean endpoint fits, confirming that the initial representation plus linear head is insufficient and upstream adaptation is essential.

Q5. Are the reported model reorderings distinguishable from seed and evaluation noise?

We agree that differences within one SD or 1–2 items on a 50-probe set do not establish reliable reorderings. We removed such claims for Minimal Pairs and base conclusions on clear separations: 10-seed Boolean fits and interventions, the formal QPA restriction, and the larger DisCoCat RP and TALES KL gaps.

Q6. Is QPA genuinely quantum, and is the theorem specific to quantum attention?

QPA is a hybrid model. Quantum circuits extract token-local Pauli features for Q,K,VQ,K,V, and attention and pooling are computed classically. The restriction follows from token locality, a single final query, and an affine readout, so it also applies to classical models with this structure.

Appendix C.3 extends the AND/XOR result to single-query softmax. Multiplying the logit difference by the positive softmax denominator preserves the predicted class and gives an additive score over the earlier tokens. Both single-query variants had 0/10 endpoint fits. Cumulative pooling allows the earlier operand to query the operator. With pooling, we observed 9/10 endpoint fits using unnormalised attention and 0/10 using softmax. The proof concerns deterministic predictions. The occasional finite-shot 12/12 evaluations were transient.

Q7. Specific implementation questions

  1. Mitigation: No readout mitigation, dynamical decoupling, or Pauli twirling was explicitly configured; runs used transpiler level 1 and the available IBM Runtime defaults (App. E.2).
  2. Parameters: QCNN-R (108) vs. QCNN (96) is now a within-family intervention; two-repetition QFF vs. QFF-R (96 each) is the strict parameter-matched control.
  3. RP footnote: Corrected: 27/31 sentences have an in-vocabulary final word. A zero default handles the four both-OOV cases. The 100% figure assumes the other 27 are classified correctly.
  4. Gradients: §3, Tables 2–3, and App. B.4 now state SPSA+Adam for embeddings/circuits and analytic gradients+Adam for the head.

Q8. Scope, related work, and scale

We expanded §2 to connect our study to systematic QML benchmarking (Bowles et al., 2024; Jerbi et al., 2023; Schuld & Killoran, 2019) and prior QNLP hardware demonstrations. This literature motivates fair classical baselines and separating the effects of data encoding, circuit structure, classical readout, optimisation, and physical execution. Our study extends this methodology to a common cross-architecture sequence setting: rather than evaluating each model in isolation, we use matched interventions to identify which routing restrictions prevent reliable fitting, where task adaptation occurs, and how logical architecture translates into physical resource requirements.

This produces three reusable results: (i) a precise single-query aggregation restriction and a pooling intervention that removes it; (ii) parameter-group interventions that distinguish joint embedding/circuit adaptation from embedding adaptation over a fixed quantum feature map; and (iii) auditable width, depth, shot, and transpilation accounting from simulation to physical execution. These results define concrete scale-up tests for longer contexts, larger vocabularies, alternative circuit families, and newer processor topologies.

Comments on the paper

1 comment
AuthorsOpenReview ↗

General Response

We thank all three reviewers for their detailed and constructive feedback. We are glad the reviews recognized the breadth and thoroughness of our experiments, our open reporting of classical baselines, multi-seed distributions, and negative results, the separation of simulator training, zero-shot QPU execution, and direct on-chip optimization, the restricted-QPA analysis, and our physical resource accounting.

We have uploaded a revised manuscript (with major updates highlighted in blue). Rather than adding new tasks, we used the discussion period to run targeted controls that isolate where adaptation occurs across these routing structures and make the evidentiary scope of each experiment explicit:

1. Isolating where adaptation occurs: parameter-group interventions (§4.1, Table 1 & App. D.1).

On the Boolean task (where QRNN and QPA share an identical split of 18 embedding, 18 circuit, and 42 readout parameters), we evaluated four training settings across 10 matched seeds sharing identical initial weights, data order, and 1,024-shot SPSA perturbations: joint training, freezing the circuit (Embeddings + Head), freezing the embeddings (Circuit + Head), or freezing both (Head Only). Cells report endpoint perfect fits (/10) and mean cross-entropy loss:

Architecture (Parameters)Joint TrainingEmbeddings + HeadCircuit + HeadHead Only
QRNN (18E + 18Q + 42C)10/10 (0.128)1/10 (0.353)1/10 (0.428)0/10 (0.577)
QCNN-R (18E + 48Q + 42C)7/10 (0.262)2/10 (0.465)0/10 (0.413)*0/10 (0.616)
Cumulative QPA (18E + 18Q + 42C)9/10 (0.079)8/10 (0.127)2/10 (0.308)0/10 (0.486)

*With canonical fixed ±π/2\pm\pi/2 embeddings, QCNN-R circuit+head training reaches 2/10 fits (0.375 loss).

The architectures therefore adapt differently on the same relational task: reliable QRNN and QCNN-R fitting depends on joint embedding/circuit adaptation, whereas cumulative QPA retains most fitting reliability through embedding adaptation over a fixed quantum feature map.

2. Mean-centred (DC-corrected) Boolean geometry analysis (§4.1, Fig. 2a, App. B.2).

We agree with Reviewer 1 that the original FFT statistic included the zero-frequency (DC) component. We recomputed all spectral scores after mean-centring (ΔP−ΔP‾\Delta P - \overline{\Delta P}) and clarified that the normalized Boolean inputs sit at (±1,±1)(\pm1,\pm1) with (0,0)(0,0) at the exact XOR midpoint. Crucially, the extended-domain re-uploading shift on [−3,3]2[-3,3]^2 holds after DC removal: low-frequency energy rises from 70.5% to 79.1% for QCNN→\rightarrowQCNN-R and from 55.1% to 68.4% for the strictly parameter-matched QFF→\rightarrowQFF-R control (96 parameters each). Removing DC shifts QPA from 84.9% to 52.6%, separating its high class-region imbalance (I=76.0I=76.0\\%) from the extended-domain re-uploading shift.

3. Stronger classical, nn-gram, and readout-only baselines (§4.1–4.3, Tables 3–5).

Classical Boolean fit rates after a 200-configuration sweep reach 7/10 (MLP), 7/10 (RNN), and 9/10 (CNN); a deeper Transformer control reaches 5/5. On TALES, we added smoothed Unigram (27.62 PPL), Bigram (6.25), and Trigram (6.00) baselines. On TALES-light, readout-only training yields 18.20±0.8018.20\pm0.80 PPL (QRNN) and 18.08±0.5018.08\pm0.50 (QCNN) vs. 4.39±0.724.39\pm0.72 and 5.30±0.795.30\pm0.79 jointly, establishing the need for upstream representation adaptation.

4. Operational hardware baseline and matched 256-shot controls (§4.3, Table 4).

The QPU runs on ibm_strasbourg and ibm_brussels establish an operational baseline for end-to-end physical training and resource scaling. Strictly matched 5-seed QRNN controls at 256 shots reach 5.14±1.025.14\pm1.02 PPL clean and 5.61±0.575.61\pm0.57 with generic depolarizing noise.

5. Single-query classification restriction (App. C.2–C.3).

The AND/XOR obstruction holds for single-query QPA with and without softmax. With softmax, multiplying the logit difference by its positive denominator yields an additive decision score with the same predictions. Cumulative pooling introduces an operator–operand interaction. In the ablation, it yields 9/10 endpoint fits with unnormalised attention and 0/10 with softmax.

6. Clearer contribution and reusable experimental framework.

Across the reviews, the thoroughness of the experiments, multi-seed reporting, separation of execution regimes, re-uploading controls, and physical-resource accounting were consistently recognized as strengths. By removing repetitive qualifications, consolidating limitations in §5, and organizing the paper around architecture-specific adaptation, pooling/re-uploading interventions, and physical execution, the revision provides a clean, reproducible framework that future researchers can directly build on.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.