acceptodds
Under review as a conference paper at ICLR 2027

Policy RBMs on Superconducting Quantum Computers

Abstract

Restricted Boltzmann machines (RBMs) are Ising models, and a quantum processor that holds the Gibbs state of an Ising model returns a joint sample of all visible and hidden units of the RBM in a single measurement, without Markov chains or burn-in. Two obstacles stand in the way: the Boltzmann factor is not unitary, so the state is hard to prepare, and superconducting qubits couple to few neighbours, so an RBM whose graph does not match the hardware costs circuit depth that the quantum processor does not have. We treat both as a representation-learning problem in which the quantum processor defines constraints on the model space. A neural network policy for power-grid control, trained by AlphaZero-style self-play, is distilled into a conditional RBM whose 118 units and 210 weights occupy every functioning qubit and coupler of a 120-qubit IBM Nighthawk r2 quantum processor, whose action code follows the measured qubit noise and whose binary input state code is learned with the model. A quantum imaginary-time evolution prepares its conditional Gibbs state. A series of experiments isolates the effect of each technique. The quantum-processor policy agrees with the teacher’s preferred action in about half of the held-out states, against 27 % for the state-independent prior, and lies within a few hundredths of a nat of its exact read-out, whereas QAOA-type circuits fall far short of it. To our knowledge this is the largest RBM sampling experiment so far on a gate-based superconducting quantum processor with a read-out that recovers the trained model.

Primary area:
probabilistic methods (Bayesian methods, variational inference, sampling, UQ, etc.)
Keywords:
restricted Boltzmann machine, knowledge distillation, policy imitation, closed-form estimation, sparse models, planar graphs, power grid control, quantum sampling, QAOA, quantum imaginary-time evolution, Ising model

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_sFeKOpenReview ↗
Rating
2
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper presents a framework for implementing a distilled neural-network policy on a superconducting quantum processor. A policy for power-grid control is first distilled into a conditional restricted Boltzmann machine (RBM), whose graph and binary input and action representations are adapted to the hardware. The resulting RBM defines a classical Ising model, and quantum circuits are used to prepare and sample its conditional distribution. The paper compares QAOA-type circuits with imaginary-time-evolution-based preparations. Its most successful preparation exploits small conditional star components and produces policy scores close to those of the exact classical RBM, achieving approximately 49% top-1 agreement with the teacher. The experiments investigate the effects of model structure, representation choices, state preparation, and hardware noise.

Critical strengths
  1. The authors propose a complete pipeline from policy distillation to execution on real quantum hardware. The paper connects RBM training, hardware-compatible representations, circuit preparation, and sampled policy readout. This provides a concrete demonstration of the practical steps required to implement a learned probabilistic model on a superconducting processor.
  2. The paper shows that input encoding, action encoding, and model connectivity substantially affect both predictive capacity and the feasibility of quantum readout. The comparison between sparse action codebooks and fully assigned registers is particularly informative because it identifies a mismatch between the distribution optimized during training and the distribution obtained after decoding samples.
Critical weaknesses
  1. The significance beyond the hardware demonstration remains insufficiently established. The authors explicitly make no claim of quantum acceleration, but the successful conditional model decomposes into small, classically tractable components whose probabilities are calculated before circuit execution. Consequently, the nominal 118-unit model does not demonstrate accurate sampling of a comparably large connected conditional distribution. The paper provides limited evidence that the successful preparation extends to models with more complex conditional dependencies, leaving the scope of the reusable methodological contribution unclear. The reported classical students also achieve stronger policy imitation, so the experiments do not establish an advantage for policy learning or approximation.
  2. The practical consequences of the policy approximation are not evaluated. Although the paper reports offline agreement with the teacher and comparisons with the exact RBM, it does not evaluate the resulting policies in closed-loop power-grid control. It therefore remains unclear whether the observed approximation errors preserve useful control performance, such as cumulative reward, episode survival, or constraint satisfaction. This limits the practical significance of the chosen application: agreement on states collected from the teacher does not establish how the distilled or quantum-sampled policy behaves when its own actions determine subsequent states.
Questions for authors

See the weaknesses section.

Reviewer_NGnZOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper distills a 118 unit Boltzmann machine (RBM) on a superconding quantum processor, being the largest RBM based experiment on gate based quantum hardware to date. The authors run a power grid task and distill an AlphaZero teacher model into a student RBM.

To make the RBM run on quantum hardware they introduce 3 specific techniques.

  1. Grid embedding: They distill the teacher into the classical RBM while also learning a threshold layer to binarize the inputs to state bits. This grid runs on the QPU’s 2D grid without expensive swap operations. They restrict hidden units to have 2-4 connections and also aim to make hidden units connect to a state/action bits when possible so they don’t become “dead” (for 36 units they can guarantee no hidden bit is dead).

  2. Action Register: They reduce the number of action qubits from 36 bits to 12 bits. This reduces errors and ensures you get valid bit strings.

  3. QITE: They create a star like architecture where 1 action qubit has 4 hidden qubits around it. This creates independent probabilities allowing them to compute the rotational angles cheaply classically and run the circuit directly. QAOA on the other hand they show achieves poor results and also needs to be optimized which is expensive. And other approaches are expensively trying to map the Gibbs state directly.

Combining these methods they are able to reach 49.4% of top-1 agreement with the teacher and beat QAOA approaches by large margins. Also their method is cheap considering the rotational angles and conditional probabilities become independent.

Critical strengths

I found this work overall impressive. Running a distilled RBM of this scale on a 118 qubit QPU and showing it works somewhat well compared to the classical teacher is impressive. Much work in ML -> Quantum distillation is typically of much smaller scale. This work combines practical quantum engineering (e.g. grid setup / noise aware register / star graph architecture) with machine learning techniques (e.g. distillation and sparse connectivity) which is a smart combination.

I found the grid based setup particularly strong. Mapping hidden, state, and action units directly onto the 2D QPU grid (with the star graph / sparse connectivity), ensures the removal of swap based operations that would be a main hindrance for running RBM’s on superconducting quantum hardware. The star like setup also makes this problem tractable as the Gibbs state factories directly to p(σc)∏i=14p(σhi∣σc)p(\sigma_c) \prod_{i=1}^4 p(\sigma_{h_i} \mid \sigma_c) which makes the number of rotational and double qubit gates small. Specifically comparing this quantum state preparation to standard QAOA makes their method seem much more efficient and useful.

Overall this work opens up for a lot of new explorations into how RBM’s can be run on quantum hardware and I can see many of the techniques used here explored and refined further.

Critical weaknesses

I have two main concerns, regarding the general applications of this work for different problems and the uses of it.

While I thought the star-like setup was strong for their particular problem it seems specific to their setup. The action bits in their case become independent and their marginals are therefore set by single rotational gates. For more complex tasks, action bits must be correlated so their setup doesn’t work and the other methods (like QAOA) would struggle and not be scalable on quantum hardware. In many tasks I would assume more action units that share hidden units are needed.

Another concern of mine is the usefulness of this work. Their solutions require a classical teacher, distilling it into an RBM, and making constraints on it to fit it on a 2D QPU grid. This loses accuracy while also being a “complicated” step by step process. People would in reality just run the teacher or RBM on CPU/GPU’s directly which is more accurate and computationally cheap. In other terms this is not a “quantum problem” which is also noted in this work. While it’s true that quantum Gibbs sampling returns joint samples in a single shot without MCMC, distilling an efficient model onto a QPU does not give this benefit. While this work opens up for many more innovations a clearer discussion of tasks this can be used for would benefit even if it might not be useful today. Specifically, a stronger motivation is needed or a deeper discussion of limitations.

On a minor note. The work relies on several parts of the appendix requiring some back and forth reading. I also think certain parts of the teacher distillation sections could be cut as section 4 is far more interesting and important.

Questions for authors

What would you do for tasks where the action units are not independent and must share hidden units (i.e. a loop based graph and not simple tree structures).

Reviewer_H3HyOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper describes how to implement a policy for power-grid control via RBM emulation on a quantum computer. The stated contributions are: (1) A "distillation pipeline" for lattice-structured RBMs, here the a classical dense RBM is first trained and then distilled into a classically implemented RBM that respects the architectural constraints of the quantum system and then the lattice-structured RBM is implemented in the quantum computer; (2) binary action codes, that are compatible with the quantum sampler (noise aware); and (3) a demonstration of the RBM implementation on quantum hardware.

Critical strengths

The paper presents a reasonably careful and substantial experimental demonstration of an RBM on quantum hardware. This is a critical step in the path toward using quantum hardware into the AI wheelhouse.

Critical weaknesses

Assessment of Significance, Impact and Motivation:

At the high-level my objection with this paper is that it is motivated as policy model for a control task, and claims novelty from that vantage point, yet the treatment (and evaluation) are entirely in a standard prediction model (classifier) context, using cross-entropy and accuracy as evaluation metrics. More on this point below.

As I mentioned above, the paper presents a substantial experimental implementation of RBMs on quantum hardward, however its broader scientific impact is limited by several considerations:

(1) The central limitation is the absence of any demonstrated quantum computational advantage. The successful Gibbs-state preparation relies on a model whose conditional distribution decomposes into small, classically tractable components. The 118-qubit implementation does not demonstrate an ability to perform computations that are difficult for classical methods. The tractable structure that enables accurate Gibbs-state preparation is also what makes classical sampling efficient. It is therefore unclear whether the proposed approach would extend to models with sufficiently complex dependencies to make classical inference intractable.

(2) The most successful quantum state-preparation approach relies heavily on classical inference to determine the parameters of relatively shallow quantum circuits. While the authors acknowledge this point, it still brings into doubt the novelty of the quantum computational contribution. The work is arguably better characterized as hardware-aware compilation and reproduction of a classical probabilistic model than as an advance in quantum sampling.

(3) Finally, the resulting policy underperforms relatively simple classical alternatives. Evaluation is restricted to reproducing the teacher's action distribution rather than measuring downstream control performance. Consequently, the experiments provide limited evidence that the proposed approach offers practical advantages for reinforcement learning or real-world control applications.

Presentation:

The paper is challenging to read and is not suited for a typical ICML reader. There are many terms from the physics community that are not defined. This is perhaps fine if there existed a sufficiently large sub-community of ICML that exists at the AI-Quantum interface. I don't think this community is currently sufficiently large to justify this level of specialization of language.

Notational issues:

The notation in Equation 7 would benefit from greater precision. In particular, the authors should explicitly define the edge set E\mathcal E and clarify how the visible and hidden indices vv and hh relate to the qubit indices used to define σi\sigma_i.

Questions for authors

Q: As I've stated above, the treatment of the control task is superficial. What properties of policy distributions, rather than arbitrary conditionals, does your quantum approach exploit?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.