acceptodds
Under review as a conference paper at ICLR 2027

Entanglement as a Structural Complexity Axis: A PAC-Bayesian View of Generalization in Quantum Policies and Value Functions

Abstract

Parameterized quantum circuits (PQCs) are increasingly used as policies and value functions in quantum reinforcement learning (QRL), yet almost all evaluations report only average return, leaving open the question of when and why a quantum policy generalizes. We give a PAC-Bayesian answer. We derive a generalization bound for stochastic quantum policies whose complexity term is controlled not by the raw number of circuit parameters, but by the log-determinant complexity of the Fisher geometry induced by the circuit—a quantity that entangling connectivity inflates by expanding the readout's causal cone. Empirically, in a controlled setting that fixes the number and layout of trainable rotations and varies only the entangling connectivity, this acts as an independent axis of complexity: at a fixed parameter count the train-to-test generalization gap grows with the circuit's Fisher log-determinant complexity, whereas raw parameter count is the weakest predictor of the gap. The bound acts as a ranking certificate: at fixed parameter count it consistently distinguishes the non-entangled circuit from entangled alternatives, something a parameter-counting bound cannot do at all. We confirm the mechanism across settings—supervised classification (– qubits, synthetic and real data), a reward-only quantum contextual bandit, and multi-step value-function generalization—where entangled circuits generalize worse than non-entangled ones of identical parameter count and the gap shrinks with sample size as predicted. Our strongest evidence is in these low-variance decision models; in genuine end-to-end multi-step policy learning the standard entangler's effect is statistically significant but return variance leaves the full ordering only partially resolved, so we frame the paper as a controlled study of the entanglement–generalization trade-off in quantum decision models rather than a solved account of end-to-end quantum RL. A partial-correlation analysis shows screens off the entangling pattern (), consistent with readout causal-cone expansion as the operative mechanism and with the Meyer–Wallach entangling power as an empirical correlate rather than the driving variable; controls (matched training accuracy, alternative readout and optimizer) rule out optimization confounders, and the effect survives execution on an IBM Heron processor under real noise. Our results reframe quantum-policy design around an entanglement–generalization trade-off rather than expressivity alone.

Primary area:
learning theory
Keywords:
PAC-Bayes, generalization bounds, quantum machine learning, entanglement, quantum reinforcement learning

Reviews

Among:
Shape:
1234this paper, Top 4%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.33 over 3 reviews, above 96% of the others.
Reviewer_iGxnOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3(+1)
Soundness
2(+1)
Read review
Paper summary

So, this paper is all about why PQCs with the same number of parameters generalize differently. The authors frame this question around QRL policies and relevant value functions. Now on the theory side, Thm 1 is sort of the pretty standard McAllester-Maurer PAC-Bayes generalization bounds for iid environments. The authors proceed to Thm 2 which takes an isotropic Gaussian prior and a Gauss-Newton posterior such that the KL term picks up a Fisher log-determinant, ok. Then Prop 1 goes on and shows that the backward causal cone of a local readout can only grow when entangling gates are added, which caps the Fisher rank. Finally the authors say that only this cone monotonicity is proved and the step from rank to deffd_eff to the gap is treated as empirical.

Regarding the experiments conducted the authors use a small test bed with a 300 point correlation grid, together with a matched CNOT control and a global readout control, scaling to n=16 (which is for 2026 standards a bit low) then Monte-Carlo value regression, REINFORCE over four small real datasets and one (again, needs be more) IBM Heron run. The main finding is that at fixed dd the gap grows with connectivity and with deffd_eff, and deffd_eff (rho=0.82)(rho=0.82) predicts the gap better than benchmark.

Critical strengths

First, the authors are honest about their work and its strenghts/weaknesses. They do separate proved from measured claims (for example Figure 1 and Remark 2) and report results that cut against its own story. And honenstly this kind of reporting is rare in quantum ML and makes the empirical picture easier to trust while it also demonstrates the hardness of the field.

Now, the derivation the paper mosty relies on is correct and checkable (and I checked with AI and was able to re-derived Eq. (2) and found that the dropped term is non-negative, so dropping it is conservative (authors better recheck). Also Remark 1 is right that gamma can depend on the data without a union bound since it only shapes the posterior. Also Table 2 bound values can be reproduced from the stated C, m and delta, and several controls are really quite informative.

Critical weaknesses

The unentangled baseline differs in feature access and not just entanglement. With product encoding, no two-qubit gates and a Pauli readout, the circuit factorizes and the output depends only on x_1 and the appendix itself gives ⟨Z_0⟩ so none is a rank-2 model of one feature, while linear and full see all four. Then, a smaller gap is what any capacity measure would predict really? On the PCA datasets, none is a classifier on the first principal component and the matched-CNOT disjoint control is identical to none by construction. The global readout control makes the point directly actually since a no-entanglement (classical) circuit reaches d_eff 14.0 and gap 0.095, about the same as the local full circuit (12.9, 0.104) and (again, nicely) the authors accept this objection as the mechanism, but then what is left is that higher effective capacity gives a larger gap and realy this is well known which weakens the main claim. The title and abstract still present entanglement as an independent complexity aspect (hard one) for which I don't think the evidence supports that, maybe not at the level for acceptance.

As far as the theory is concerned, it is arguably quite thin. d_eff enters the bound mainly through the choice of posterior, Thm 2 is a Gaussian KL identity and Prop 1 is graph reachability. In general, circuits with more curved directions are penalized more because the posterior was sharpened more and a fair comparison would optimize the posterior per circuit. Separately AI catched this: the certificate is vacuous relative to chance in every cell ie implied test-error bounds are about 0.60 for none at N=64, above 0.9 for full at N=16, and about 0.54 in the best case. (non-vacuous should go.)

A few smaller issues: Lemma 1 is applied to 0/1 accuracy gaps and does not cover, and τ^2=1 is hardly local. Also, the bandit has two actions, so a zero reward reveals the label, and it is classification with a different loss and the value-function task is regression.

Questions for authors
  1. Can you build an unentangled baseline that sees all inputs at matched d, for example re-uploading all features onto wire 0, or comparing entangled and product circuits under a global readout at matched live-parameter count? Does any entanglement measure still predict the gap at fixed d_eff? This is the question that matters most for my score.
  2. What are rho(d_eff, gap), rho(norm, gap) and their difference within each N (and each N, L), with the bootstrap clustered by design cell?
  3. For each circuit, what is the best certificate over a posterior family that includes N(theta, tau^2)? Does the none < entangled ordering survive?
  4. Disclose how many hardware seeds were run, and can you report test accuracies with error bars?
  5. Can the bandit result be shown with K>2 actions, where zero reward does not reveal the label?
2 comments

The authors ran every experiment I asked for and corrected two errors in the original submission themselves. The main claims now largely match the evidence, so I am raising Soundness from 1 to 2. My rating stays at weak rejection for now, for three reasons.

First, the key experiment went against the original framing. Once D_eff is controlled for, entanglement does not predict the gap as it seems. Yet the paper title, contribution #3, the introduction and parts of the appendix still present entanglement as the governing complexity which is odd to me.

Second, the certificate only clearly separates "full" from the "none" circuit that sees just one feature. Unentangled circuits that see all features get bounds as loose as full. So the certificate is again picking up feature access and not entanglement.

Finally, the policy and value-function experiments still compare against the one-feature "none" baseline. The feature-complete baselines exist only for supervised classification.

I would move to weak acceptance with three changes: a title and text framed around Fisher log-det complexity, a feature-complete baseline in at least one policy setting (eg., the 4-action bandit), and a certificate claim narrowed to what Tables 4, 6 and 7 support.

We thank the reviewer for the follow-up and for raising Soundness. We have made the three changes you listed. The updated PDF is uploaded (table numbers refer to it; new changes are in blue).

1. Title and framing around Fisher log-determinant complexity

  • Title is now "Beyond Parameter Count: A PAC-Bayesian View of Fisher Complexity and Generalization in Quantum Policies and Value Functions".
  • Contribution 2 is now "Fisher complexity, not parameter count, tracks the gap". It states that the gap follows DeffD_{\rm eff} at fixed dd for product and entangled circuits alike. Entangling connectivity is presented as one structural route to a larger DeffD_{\rm eff}, alongside global readout and data re-uploading. Once DeffD_{\rm eff} is controlled, the entangling power carries no further information.
  • Introduction. "None isolates entanglement as the governing complexity" is replaced. The pathway paragraph now starts from circuit structure rather than from entanglement.
  • Also reworded: the abstract, Related Work, the Section 4 title ("Entangling Connectivity Raises Fisher Complexity at Fixed Parameter Count"), the Figure 1 caption, the Discussion, and captions that called the gap "the entanglement cost".
  • "Entangled circuits generalize worse". Every remaining sentence of this form (Section 6; Appendix B near Tables 6, 14 and 21) is now scoped to the controlled setting with fixed encoding and readout.
  • Why entanglement remains in the experiments. It is the variable we manipulate in the controlled sweeps, because it changes DeffD_{\rm eff} while dd, the encoding and the readout stay fixed. No claim rests on entanglement beyond that.

2. Feature-complete baselines in the policy setting (new Table 16, paragraph in Section 6)

Protocol. Both bandits use their original protocols unchanged; only the circuit family varies.

  • Common to both: d=16d=16, 16 seeds, N∈{16,40,100}N\in\{16,40,100\}, 816 runs.
  • K=2K=2: the circuit families of Table 3.
  • K=4K=4 (Born policy on wires 0,1):
    • a re-uploading circuit whose two readout wires jointly see all four features, with none/linear/full connectivity;
    • a two-wire product circuit in which each readout wire cycles through all four features.

The none/linear/full rows are reruns of Tables 14 and 15 and lie within about one s.e. of them.

Results. The picture matches supervised classification.

ρ(Deff,gap)\rho(D_{\rm eff},\text{gap}) within NNρ\rho(entangling power, gap)partial ρ\rho of entangling power given DeffD_{\rm eff}
K=2K=20.73 – 0.810.10 – 0.13−0.13 – −0.04
K=4K=40.61 – 0.790.12 – 0.22−0.10 – 0.00

Gap at N=40N=40 (± s.e.):

local nonelocal fullproduct, sees all features (largest DeffD_{\rm eff})
K=2K=20.069 ± 0.0180.156 ± 0.0150.369 ± 0.012 (1-qubit re-upload, DeffD_{\rm eff} 46.1)
K=4K=40.139 ± 0.0180.258 ± 0.0170.417 ± 0.013 (2-wire re-upload, DeffD_{\rm eff} 54.4)

The policies with no entangling gates and the most feature access have the largest DeffD_{\rm eff} and the largest gaps. They also have the lowest test reward (0.26 for the 2-wire policy at K=4K=4, chance 0.25), so the table supports DeffD_{\rm eff} as a predictor of the gap, not of test reward; the paper now says this explicitly and does not equate a larger gap with worse test performance.

Within a fixed encoding and readout, connectivity still raises the gap in 29 of 30 paired comparisons:

  • K=2K=2: 17 of 18. The local readout gives 3.5 to 9.7 s.e., re-uploading 2.7 to 5.4 s.e., and the global readout −0.4 to 2.6 s.e.
  • K=4K=4: 12 of 12, at 2.4 to 8.3 s.e.

As in classification, the gap tracks DeffD_{\rm eff}, not entanglement.

3. Certificate claim narrowed (Table 2, Section 5, abstract, Discussion)

You are right that the certificate responds to complexity rather than to entanglement. Table 2 now includes the two unentangled circuits that see every feature, next to their DeffD_{\rm eff}:

circuitDeffD_{\rm eff}bound N=64N=64N=256N=256N=1024N=1024
none (local)2.60.7290.5890.504
linear (local)7.90.7270.6030.524
full (local)12.80.8010.6180.519
none, global readout14.10.8090.6740.561
re-upload, 1 qubit43.50.8190.6560.565

The text now claims only what Tables 2 and 5 to 7 support:

  • it separates circuits of equal dd whose Fisher complexity differs widely: the single-feature none lies below full at every NN;
  • it does not order every pair: none and linear are tied at N=64N=64 (0.729 vs 0.727), linear and full at N=1024N=1024 (0.524 vs 0.519), and linear is separated from none only at N=1024N=1024;
  • it does not separate entangled circuits from feature-complete product circuits, which receive bounds as loose as full or looser.

The abstract and Limitations say the same ("it does not order every pair of circuits and does not single out entanglement").

Reviewer_Rm4eOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3(+1)
Read review
Paper summary

The submission investigates generalization in parameterized quantum circuits (PQCs), arguing that entangling connectivity acts as a structural complexity axis governing generalization independently of rotation count dd. The authors derive a PAC-Bayesian bound (Theorem 2) for stochastic quantum policies using the empirical Fisher log-determinant deff(γ)=log⁡det⁡(I+γF^)d_{\text{eff}}(\gamma) = \log\det(I + \gamma\hat{F}), which expands through the backward Heisenberg causal cone under CNOT gates (Lemma 2, Proposition 1). Across fixed-dd circuits (none, linear, full) and a matched 3-CNOT control (disjoint, linear, star, Table 5), deffd_{\text{eff}} tracks the generalization gap; on a 300-configuration synthetic grid it achieves Spearman ρ=0.82\rho = 0.82 with the gap, ahead of the learned norm (0.730.73), entangling power (0.720.72), and parameter count (0.450.45). The gap ordering is further examined on a contextual bandit, real tabular datasets (Iris, Breast Cancer, Wine, Digits), and Monte Carlo value regression.

Critical strengths
  1. Informative Matched-Gate Control in Table 5 (Lemma 2, Proposition 1, Section 4, Table 5): The paper provides a valuable empirical control in Table 5 by holding both d=16d=16 and gate count fixed at 3 CNOTs: disjoint (cone {0}\{0\}) matches none on gap (0.1010.101), accuracy (0.5870.587), and deffd_{\text{eff}} (3.33.3), showing empirically that uniform count-based bounds (Banchi et al. 2021; Caro et al. 2022) cannot separate these circuits; meanwhile linear and star share 3 CNOTs and cone {0,1,2,3}\{0,1,2,3\} yet differ by 2.3×2.3\times in deffd_{\text{eff}} (9.39.3 vs 21.821.8) and 2.5×2.5\times in gap (0.1460.146 vs 0.3720.372 at N=16N=16). Across 300 configurations, deffd_{\text{eff}} achieves Spearman ρ=0.82\rho = 0.82 with the gap (vs 0.730.73 for norm, 0.720.72 for QQ, 0.450.45 for dd) and screens off connectivity.

  2. Claim Discipline and Self-Reported Negative Results (Remark 2, Appendix A.4, Tables 3, 4, 8, 9, 10, 11): The authors candidly document key boundary conditions: the linear/full ordering swap for the stochastic predictor at N≥32N \ge 32 (Table 3); absorption of the n=4n=4 contrast under global readout (+0.033+0.033, 0.9σ0.9\sigma, Table 4); FQI TD bootstrapping washout (Section 6); barren plateaus (Table 10); Iris counterexample (0.760.76 vs 0.700.70, Table 9); and 4-bit parity requiring entanglement (0.900.90 vs 0.500.50, Table 8).

Critical weaknesses
  1. Certificate-Empirical Ordering Mismatch and Wide Derandomization Envelope (Appendix A.3 Lemma 1, Appendix B Table 3): A structural tension separates Theorem 2 from empirical generalization:

    • Author-Disclosed Stochastic Ordering Swap (Table 3): Theorem 2 certifies randomized policy πQ∼N(θ,τ2(I+γF^)−1)\pi_Q \sim \mathcal{N}(\theta, \tau^2(I + \gamma \hat{F})^{-1}). As the authors note in discussing Table 3, from N=32N=32 on "linear and full exchange places" (within one standard error at N=32N=32 and 6464, outside it at N=128N=128), because posterior averaging over full's eleven nearly flat Fisher directions shrinks its empirical stochastic gap, a shrinkage the bound's complexity term does not model. At N=128N=128, full's empirical stochastic gap is smaller than linear's (0.006±0.0190.006 \pm 0.019 vs 0.034±0.0100.034 \pm 0.010), yet Theorem 2 assigns full (0.240.24) a looser bound than linear (0.220.22), so the bound's linear/full ranking does not match the empirical ranking of the certified object.
    • Wide Derandomization Envelope (Lemma 1, Table 3): Passing from πQ\pi_Q to deterministic point policy θ\theta via Lemma 1 incurs additive slack bounded by the worst-case envelope Δderand=τ2d/γ=0.32\Delta_{\text{derand}} = \tau^2 d / \gamma = 0.32 (τ2=1,γ=50,d=16\tau^2 = 1, \gamma = 50, d = 16), the envelope the authors themselves cite. This slack envelope is strictly larger than all empirical deterministic θ\theta gap contrasts (full minus none) across all sample sizes in Table 3 (0.2800.280 at N=16N=16, 0.1770.177 at N=32N=32, 0.1120.112 at N=64N=64, 0.0680.068 at N=128N=128), so this envelope cannot resolve the deterministic contrasts, and the paper provides no certificate for the deterministic policy θ\theta (the authors state that they do not claim Lemma 1 as a rigorous transfer, Appendix A.3).
  2. Causal Cone Prior Art Overlap, Detached Dummy Parameters, and Readout Dependence (Section 4, Section 5, Section 7, Appendix A.4 Lemma 2, Table 4): The paper's novelty claims regarding causal cones and entanglement complexity need sharper positioning against prior light-cone literature and against the readout-geometry control:

    • Prior Art Operator Locality: The backward Heisenberg causal cone expansion of local observables (Lemma 2, Proposition 1) and vanishing gradients outside the cone closely overlap arguments in the barren-plateau and light-cone literature (e.g., Cerezo et al. 2021, which the paper cites only in its barren-plateau discussion), and the authors should position Lemma 2 against this work. Under local observable ⟨Z0⟩\langle Z_0 \rangle at n=4n=4 (d=16d=16), off-wire rotations commute with Z0Z_0, leaving none with a potential cone of only the 4/164/16 rotations on wire 0, of which just 22 are numerically live (Table 4: 2/162/16, deff=2.6d_{\text{eff}} = 2.6 at N=64N=64), while wires 1…31 \dots 3 act as detached dummy variables with zero gradient (∂θf≡0\partial_\theta f \equiv 0).
    • Global Readout Control: When readout is global (1n∑iZi\frac{1}{n}\sum_i Z_i, Table 4), none's live parameters rise from 2/162/16 to 7/167/16 at n=4n=4 (full: 12/1612/16), and the gap contrast between full and none shrinks from +0.112+0.112 (4.1σ4.1\sigma) to +0.033+0.033 (0.9σ0.9\sigma, not significant). At n=6n=6, under the global readout none has 11/2411/24 live parameters (deff=23.6d_{\text{eff}} = 23.6) vs. full's 17/2417/24 (deff=35.4d_{\text{eff}} = 35.4), and the contrast persists (+0.114+0.114, 3.8σ3.8\sigma local vs. +0.115+0.115, 4.2σ4.2\sigma global). The n=4n=4 result suggests that much of the local-readout contrast reflects recruitment of otherwise inactive parameters (which the authors accept as the mechanism, Section 7) rather than multi-qubit entanglement as such.
  3. Quantum RL Scope Disconnect, FQI Washout, and Unreported Fisher Rank at Larger nn (Section 2, Section 3, Section 6, Appendix A.4):

    • Scope Disconnect & TD Washout: Related Work positions the contribution as "return-based generalization under Markov dependence," yet Theorem 1 is stated over i.i.d. environment draws M∼DM \sim \mathcal{D} (Majumdar et al. 2021); the authors note that "Theorem 1 is the i.i.d. statement", that the Markov mixing-time correction (Zitouni et al. 2025) is one they "neither reprove nor rely on", and they instantiate it on fixed-environment i.i.d. contexts without distribution shift. In bootstrapped Fitted-Q Iteration (Section 6), the gap contrast washes out, and deffd_{\text{eff}} is not reported under the TD objective.
    • Fisher Rank at Larger nn: Appendix A.4 says that the deffd_{\text{eff}} values in Table 6 come from a rank-≤40\le 40 Fisher estimate (M=40M=40), but Table 6 reports only gaps. Since rank(F^)≤min⁡{M,∣L(θ)∣}\text{rank}(\hat{F}) \le \min\{M, |L(\theta)|\}, whether M=40M=40 actually constrains deffd_{\text{eff}} there depends on ∣L(θ)∣|L(\theta)|, not on dd; reporting ∣L(θ)∣|L(\theta)| and deffd_{\text{eff}} for the n≥10n \ge 10 rows would let readers check this.

Score rationale

Presentation (3). The submission is clearly structured and consistent. Remark 2 and Appendix A.4 carefully separate proved claims from empirical regularities, and negative results are transparently documented.

Informativeness (2). While the matched 3-CNOT control in Table 5 provides an informative empirical demonstration that uniform gate-count bounds (Banchi et al. 2021; Caro et al. 2022) cannot separate fixed-dd, fixed-TT topologies, the theoretical contributions do not significantly shift perspectives on quantum generalization. The backward causal-cone parameter activation mechanism (Lemma 2, Proposition 1) closely follows operator-locality (light-cone) arguments already used in prior work (e.g., Cerezo et al. 2021), Theorem 2 applies standard Gaussian PAC-Bayes (Dziugaite & Roy 2017; Rodriguez-Grasa et al. 2026) to the Fisher log-determinant functional of Abbas et al. (2021) evaluated at the trained parameters, and the certificate's linear/full ordering disagrees with the empirical stochastic ordering at N≥32N \ge 32, while the derandomization envelope (τ2d/γ=0.32\tau^2 d/\gamma = 0.32) exceeds every deterministic full-none contrast.

Soundness (2). Soundness is rated 2 for three substantive but fixable technical concerns:

  1. Certificate Ordering & Derandomization Envelope (Lemma 1; Table 3): Theorem 2's bound orders linear and full opposite to their empirical stochastic gaps at N≥32N \ge 32 (within one standard error at N=32,64N=32, 64; at N=128N=128 it ranks full looser, 0.240.24 vs 0.220.22, although full's stochastic gap is smaller, 0.006±0.0190.006 \pm 0.019 vs 0.034±0.0100.034 \pm 0.010), while the Lemma 1 envelope (τ2d/γ=0.32\tau^2 d/\gamma = 0.32) exceeds all deterministic full-none contrasts (0.280→0.0680.280 \to 0.068).
  2. Prior Art & Readout Confound (Section 7; Table 4): Backward causal-cone expansion closely overlaps light-cone arguments in prior work (e.g., Cerezo et al. 2021). Under local readout ⟨Z0⟩\langle Z_0 \rangle, unentangled off-wire parameters are detached dummy variables, and at n=4n=4 the full-none contrast largely disappears under global readout (+0.033+0.033, 0.9σ0.9\sigma), although it persists at n=6n=6 (+0.115+0.115, 4.2σ4.2\sigma).
  3. RL Framing vs. i.i.d. Validation (Section 2; Section 3; Section 6): Theorem 1 is an i.i.d. statement over environments M∼DM \sim \mathcal{D} whose Markov mixing-time extension the authors explicitly do not use, so the certificate is evaluated only on static i.i.d. contexts despite the policy/RL framing, and gap contrasts wash out under Fitted-Q Iteration.

Rating (2). Table 5 provides commendable empirical controls (showing via matched 3-CNOT topologies that gate count alone does not explain the gap) and negative results are reported transparently. However, the submission does not yet meet the acceptance threshold:

  1. The causal-cone mechanism (Lemma 2) closely overlaps operator-locality arguments in prior work (e.g., Cerezo et al. 2021), and the full-none contrast largely disappears under global readout at n=4n=4 (0.9σ0.9\sigma, Table 4), though not at n=6n=6.
  2. The PAC-Bayes certificate's linear/full ordering disagrees with the empirical stochastic ordering at N≥32N \ge 32 (Table 3); at N=128N=128 it ranks full (0.240.24) looser than linear (0.220.22) although full's empirical stochastic gap is smaller (0.006±0.0190.006 \pm 0.019 vs 0.034±0.0100.034 \pm 0.010).
  3. The Lemma 1 derandomization envelope Δderand=τ2d/γ=0.32\Delta_{\text{derand}} = \tau^2 d/\gamma = 0.32 exceeds all deterministic gap contrasts (0.280→0.0680.280 \to 0.068).
  4. Multi-step QRL support is limited: the effect washes out under bootstrapped fitted-Q iteration, and end-to-end policy evidence covers only linear vs. none at small NN. Transparent reporting of negative results cannot substitute for a predictive theory. The paper should be revised with anisotropic priors, tightened derandomization, and an MDP-level (non-i.i.d.) analysis.

Post-rebuttal update (Nov 19). I read the responses and the revised manuscript. Soundness 2 → 3. My three soundness concerns were resolved by narrowing the claims: the bound is now a ranking certificate, Lemma 2 is presented as the standard light-cone argument, the "independent axis" claim is withdrawn, and the gap–DeffD_{\rm eff} link is limited to fixed-target losses. Rating 2 → 3. The claims now match the evidence. I still find the contribution close to Abbas et al. (2021) plus a standard Gaussian PAC-Bayes bound, and the multi-step RL evidence limited.

Questions for authors
  1. [Clarification / Theory]: Given that Theorem 2's certificate orders linear and full opposite to their empirical stochastic gaps at N≥32N \ge 32 in Table 3, and that the Lemma 1 envelope Δderand=τ2d/γ=0.32\Delta_{\text{derand}} = \tau^2 d/\gamma = 0.32 exceeds deterministic gap contrasts (0.280→0.0680.280 \to 0.068), would an anisotropic prior prevent flat Fisher directions in full from dominating the perturbation while tightening Δderand\Delta_{\text{derand}}?
  2. [Clarification / Readout & Prior Art]: In Table 4, why does none have only 11/2411/24 live parameters under global readout at n=6n=6 vs. 17/2417/24 for full? How is Lemma 2's causal-cone formulation formally differentiated from operator locality in barren-plateau literature (e.g., Cerezo et al. 2021)? Also, can you report ∣L(θ)∣|L(\theta)| and deffd_{\text{eff}} for the n≥10n \ge 10 rows of Table 6, so readers can see whether M=40M=40 constrains the Fisher estimate there?
  3. [Clarification / RL Scope]: What is the measured deffd_{\text{eff}} under Fitted-Q Iteration (Section 6) where the gap washes out, and given the static i.i.d. evaluation, would the authors align the title with the framing already used in the abstract and Scope paragraph (a controlled study of quantum decision models), with Monte Carlo value regression as the multi-step evidence?
Reviewer_2KLEOpenReview ↗
Rating
4
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The manuscript presents a study on generalization capabilities of parameterized quantum circuits when used as a value function or a policy decision in a reinforcement learning system. The topic is important and has many open questions to be answered. The present work makes a good contribution to address these points.

There are three major contributions of the work. i. First it provides a PAC Bayes characterization of reinforcement learning policies that employ a parameterized quantum circuit. The key contribution here is demonstration of Fisher log-determinant as a capacity measure instead of the parameter count. This model dependent characterization extends similar studies in classical reinforcement learning to variational quantum circuits. This is an important contribution in terms of the analytical framework adapted for PQCs.

ii. The most important contribution is perhaps the introduction of entanglement architecture as a complexity parameter. It effectively separates the parameter complexity of the PQC and the entanglement complexity as two different components of the structural complexity expression. This is a valuable insight in the sense that this is where the quantum advantages are directly evident.

iii. The experimental design for evaluating the generalization ability of PQC as a value function or policy in RL is also an important contribution which clearly lays out the path for analysing the capacity of these models in a systematic manner.

Critical strengths

The paper, probably for the first time, differentiates the circuit complexity of PQC and the read-out complexity of the entanglement as two distinct components of the structural complexity of a quantum circuit based reinforcement learning system. This is an useful insight that may help design quantum ML systems with better generalization complexity. It may also provide pathways to reduce the variance via the entanglement pathway. This might have interesting counterparts in the boosting based approach in classical ML algorithms.

The derivation of the PAC Bayes characterization of the model capacity is also interesting. This provides a nice framework for model dependent structural complexity measures in terms of the Fisher log-determinant. It will be interesting if a single criteria encompassing the end-to-end quantum ML system can be developed in this framework.

The empirical evaluation is well planned, systematic, and illustrates the main questions in studying generalization capabilities of quantum RL systems.

Critical weaknesses

The paper is overall of high standard both in terms of theoretical and empirical significance. A more detailed derivation of the PAC Bayes bound in the quantum context might be helpful for the readers.

Questions for authors
  1. How can the presented results and theoretical characterizations of capacity of a quantum RL circuit be used to design better systems? I agree this should be a future work But some light on the possible directions that may be taken might be useful.

  2. How does the scheme work on high variance POCs in terms of empirical results? Do the results degrade? How exactly is the bias-variance trade-off handled in this setting. Just some intuitive discussion might help readers get a better perspective of the working of the system.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.