acceptodds
Under review as a conference paper at ICLR 2027

Distilling Datasets into Shallow Circuits for Quantum Machine Learning

Abstract

In quantum machine learning, training a quantum model requires each sample to be prepared as a quantum state by a loading circuit that must be re-executed for every shot at every training step. The total burden therefore scales with both the number of samples and the cost of preparation. Existing approaches reduce the loading cost of individual inputs or distill data and compress their representations before applying a separate quantum encoding. This separation can leave resulting samples costly to prepare or cause additional loss of the information during subsequent compilation into shallow circuits. We propose quantum dataset distillation (QDD), which distills the full dataset directly into a small set of shallow circuits. Each synthetic sample is parameterized as a staircase circuit corresponding to a low-rank tensor network, making the sample and its loading circuit the same object. The circuit parameters are optimized classically with exact gradients using distribution matching under an explicit loading budget, without post-hoc state-preparation synthesis. On MNIST and Fashion-MNIST, only 10 circuits per class ( the full dataset size) achieve accuracy comparable to full-data training and outperform selection baselines under matched sample and loading budgets. In finite-shot training, QDD reaches 95% of the full-data accuracy with more than fewer cumulative shots. Additionally, we validate QDD on real quantum hardware, demonstrating its practical deployment potential.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Quantum Machine Learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_CqXaOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

QDD distills a whole classical dataset into a small set of shallow staircase circuits rather than into images or feature vectors. Each synthetic sample is a parameterized staircase circuit equivalent to a low-rank quantized tensor train, so the sample and its loading circuit are the same object, and no post-hoc state-preparation synthesis is needed. Circuit angles are optimized offline in exact simulation with exact gradients, matching class-conditional mean measurement features of a randomly initialized quantum model and the class mean density matrix, under an explicit two-qubit-gate budget. On 8-qubit MNIST and Fashion-MNIST at 16x16 resolution with a rank-8 interface and 42-CNOT loaders, QDD is compared with six selection baselines compiled to the same circuit and with a distill-then-compress control. The paper also reports finite-shot parameter-shift training cost and inference on three IBM processors.

Critical strengths
  1. The formulation is clean and the control is the right one. Making the synthetic sample and its loader the same object removes a real failure mode, and the distill-then-compress arm isolates exactly that choice. The +9.25pp gap at 14 CNOTs (Table 2, IPC 10) is a measurement of what post-hoc compilation costs, not an assertion. 
  2. Table 4 is the most useful result in the paper. Under identical 42-CNOT test loaders, exact preparation of the same rank-8 targets falls to 6.25 to 24.22 percent while staircase loading stays at 78 to 83 percent. That is a direct argument for approximate loading on present devices, and it holds across three processors.
Critical weaknesses
  1. The 100x shot saving is arithmetic, not an algorithmic result. The reference arm is full-batch training with b = 60000, so the compact set wins mostly by visiting 100 samples instead of 60000 per step. The matched per-step comparison sits in Appendix D.1 and gives 2.3x and 1.9x at S = 1000 and 10000, and at S = 100 the reference does not reach the target at all.

  2. The joint-optimization claim reverses at the setting the authors actually use. At 42 CNOTs and IPC 50, QDD is 82.25 and distill-then-compress is 82.63 (Table 2). The claimed advantage exists only at 14 CNOTs, where both methods sit between 60 and 74 percent.

Questions for authors
  1. Table 2, IPC 1 at 14 CNOTs: the distill-then-compress arm is 26.70 plus or minus 5.25, close to chance. Does the +2.74pp gain there measure the method or the collapse of the control?

  2. At 8 qubits the full-data reference is 82.81 percent under the same rank-8 interface (Table 4), and ten circuits per class match it. Is this a statement about dataset distillation, or about a state that carries 256 amplitudes and cannot use 60000 samples?

  3. Section 4.2 says an L-layer staircase yields an MPS of bond dimension at most 2L. Counting each layer as at most doubling the Schmidt rank gives 2^L, which exceeds 2L already at L = 3. What is the derivation of the 2L bound?

  4. QDD-10 in Table 4 reaches 83.59 percent under exact simulation, above the full-data model at 82.81. Is that a seed effect or a real gain, and why does it only appear under the staircase loader?

  5. Distillation uses exact contraction and exact gradients at 8 qubits. Up to what qubit count does the offline classical cost stay below the quantum cost it is meant to save, and does the 10-qubit result in Appendix D.4 change that estimate?

  6. Section 5.2 says QDD has the highest accuracy in five of six settings. The sixth is IPC 1 on Fashion-MNIST at 58.06 against 58.44 for k-means, a 0.38pp difference with standard deviations of 2.14 and 0.53. Is that difference intended as a claim?

Reviewer_XCkcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes quantum dataset distillation (QDD), which reduces a training dataset to a small set of shallow quantum circuits. Instead of distilling samples first and then compiling them into loading circuits, the authors optimize the circuits directly under a fixed gate budget. They evaluate the method on MNIST and Fashion-MNIST against several data-selection baselines and a distill-then-compress approach. QDD generally achieves higher accuracy under matched sample counts and loading budgets. The authors also evaluate finite-shot training costs and test the trained models on three IBM quantum processors.

Critical strengths

S1. Distilling samples directly into shallow circuits is an interesting approach to reducing both dataset size and loading cost. Table 2 gives a useful comparison with distill-then-compress and shows that joint optimization can improve accuracy, particularly under tight gate budgets.

S2. The authors compare several data-selection methods across different sample counts and loading budgets. These comparisons are useful because they test QDD under the same constraints as the baselines. I also appreciate the evaluation on three real quantum processors, which provides evidence that the resulting models can be used with shallow test loaders under hardware noise.

Critical weaknesses

W1. The reported shot savings of more than 100× are measured against full-batch training on all 60,000 examples. Appendix D.1 provides a mini-batch comparison, where the savings at 95% of the full-data reference accuracy are around 2× for the 1,000- and 10,000-shot settings. The mini-batch baseline does not reach that target at 100 shots. This is still useful, but the smaller improvement deserves greater emphasis. The finite-shot experiments also leave out k-means and distill-then-compress, which are competitive with QDD in the exact-gradient experiments. Without these comparisons, it is difficult to tell how much of the finite-shot advantage is specific to QDD rather than using a strong compact training set.

W2. The offline distillation uses exact classical simulation, but its runtime and computational cost are not reported in enough detail. The main experiments use 8-qubit models, with a smaller 10-qubit evaluation. It is unclear how costly the distillation process becomes as circuit size increases, or how much downstream training is needed to make up for its initial cost. More complete optimization settings would also help with reproducibility.

Questions for authors

Q1 (Clarification). The abstract reports shot savings of more than 100×, which the experiments measure relative to full-batch training. Appendix D.1 reports much smaller savings against mini-batch training. Could the authors explain why the full-batch result is emphasized and how the mini-batch comparison should affect the interpretation of the practical savings?

Q2 (Suggestion). The finite-shot experiments compare QDD with random selection and full-data training, but not k-means or distill-then-compress, which are competitive in the exact-gradient results. Comparing these methods under the same finite-shot budget would help determine whether QDD retains an advantage over stronger compact baselines.

Q3 (Clarification). What is the runtime and computational cost of the offline distillation process? Since it uses exact classical simulation and class-mean density matrices, how does this initial cost compare with the savings from downstream training, particularly if the distilled dataset is reused?

Q4 (Suggestion). The experiments mainly use one downstream quantum model architecture. Evaluating the distilled circuits with a different architecture would help show whether their usefulness depends strongly on the model family used during distillation.

Reviewer_ZwqrOpenReview ↗
Rating
3
Confidence
4
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

The paper is concerned with quantum machine learning and addresses the fundamental problem of preparing (training) data as quantum states to be loaded by quantum circuits. Quantum state preparation can be (exponentially) expensive as well as lossy and is therefore researched extensively. The authors propose a novel scheme for quantum dataset distillation (QDD) which maps input data to shallow quantum (staircase) circuits. Their approach relies on classical (digital) optimization and achieves high accuracy / faithfulness in experiments with MNIST and Fashion MNIST data.

Critical strengths

This is a well written paper that is actually easy to follow for domain experts (something worth pointing out in the era of AI generated research papers). The proposed approach appears to be novel, is entirely convincing, and experimentally shown to work well in practice. The idea of distilling classical training data into (batches of) representative examples has been considered before. What appears to be original is the idea of distinlling directly into circuit parameters for architectually simple (two qubit gate) circuits rather than into quantum states. The process is guided by two optimization objectives (measurement distribution matching and a density matrix based discrepancy measure). For the qubit counts considered (up to 10), the optimization can be done classically (via gradient descend) and, as it yields loading circuits, does not require subsequent (costly) state preparation steps. For people working on quantum machine learning, this is an interesting read.

Critical weaknesses

The main issue with this contribution consists in the question: “Is this paper of interest to (the majority of) the ICLR audience?” Even though the paper contains a concise introduction to quantum computing concepts, even this introdcution is likely impssible to comprehend quantum computing experience (e.g. the role of tensor products is not clarified, rotations and CNOTs are taken for granted as is the need for several measurement shots, etc.). The authors could strengthen their report by trying to answer the above question. Furthe minor criticism pertains to the meaning of "IPC" which could be better clarified and to the experimentation on real quantum hardware. The latter is a strong point of the paper which makes it sobering to see that details are delgated to the additional material.

Questions for authors
2 comments

Thank you for this clarification.

Dear Reviewer Zwqr,

Thank you for the response. We again appreciate your time and your positive assessment of our work.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.