acceptodds
Under review as a conference paper at ICLR 2027

The Cost of Quantum Features in Reacher Control: A Resource-Accounted Simulator Study

Abstract

Small quantum circuits have few parameters, but a robot controller may need to run one every time it chooses an action. We study the resulting cost on Gymnasium MuJoCo Reacher-v5. A two-qubit feature map with finite-shot sampling (Q1) and a compact classical control share the same observations, action bounds, PPO schedule, and 50,000 training interactions. We evaluate each of 20 training seeds on the same 100 cases. Q1 finishes 0.00569 m closer to the target on average. Whether it meets the predeclared 0.010 m non-inferiority margin depends on how uncertainty is estimated: the one-sided bootstrap bound is +0.00946 m, while the Student-t sensitivity bound is +0.01074 m. Success remained low: 9.4% for the compact control, 10.3% for Q1, and 11.3% for SB3. Q1 used 57.6 million shots per seed and took 0.915 ms per action at the median, about four times the compact policy’s latency. A later 500k-step comparison retained the full state alongside the features. There, Q1 finished 0.02721 m farther from the target over 18 completed pairs (95% interval [+0.01840, +0.03647] m), and two Q1 runs timed out. The follow-up failed its frozen zero-failure rule and remains exploratory. Q1 success was 16.2% on completed seeds, versus 26.8% for sine. A separate 20-seed SB3 screen improved return on every seed, yet two missed the 10% success floor. The evidence therefore concerns the cost of fixed representations in controllers with limited task success. It does not establish practical robot competence or predict performance on a quantum processor.

Primary area:
reinforcement learning
Keywords:
reinforcement learning, quantum machine learning, robotics, continuous control, reproducibility, resource efficiency

Reviews

Among:
Shape:
1234this paper, Bottom 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.00 over 2 reviews, above 0% of the others.
Reviewer_Ur7ROpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
3
Read review
Paper summary

The paper investigates the computational cost and performance of fixed 2-qubit quantum feature maps on reinforcement learning experiments, mostly Gymnasium MuJoCo’s Reacher-v5, against a classical feature map and an SB3 PPO reference. The main experiment does not statistically prove a performance advantage, and the conclusion on non-inferiority is dependent on the uncertainty calculation method (positive result with bootsrap but not with student-t). Meanwhile, its training time and action latency are approximately 4 times greater than those of the classical counterpart. The authors acknowledge the limits of these results throughout and are honest with the scope.

Critical strengths

Here are the strong points I see about this paper:

  • The paper is openly honest about its limited success, statistical uncertainty, and negative results.
  • The use of binomial noise on the classical features to investigate finite-shot effects is appropriate.
  • The reporting of circuit calls, number of shots, training time and latency provides useful context elements.
Critical weaknesses

Here are the weaknesses I would like to highlight to the authors to justify my recommendation:

  • Limited scientific value: The main experiments only produce a narrow negative result for a fixed arbitrary quantum feature map. The paper doesn’t discuss why it fails to show a benefit or what conclusions can be taken from these results in general. The paper reads like a technical report of a Reacher-v5 experiment with a quantum component.

  • Missing ablation: The 2-qubit circuit can be reproduced with an exact equivalent classical representation, but there is no benchmark for it. Without this ablation, the cost study mostly characterises the simulator implementation and we cannot infer how much of the cost is attributed to the quantum part.

  • Difficult readability: The labels “v9”, “V9”, “v11”, “v15” are repeatedly used without being properly defined until section 5, making it difficult to track architectures, protocols and status of each results. “Q1” is defined first in the abstract and then used in the main text, which is highly inelegant. The prose throughout the text is unnecessarily dense.

  • Weak architectural justifications: The paper doesn’t explain why the authors chose padding 10 features to 32 values and compressing them to 2 angular features. Such choices feel arbitrary.

  • Limited reproducibility: The authors don’t show any explicit intention to make the experiments reproducible after publication. They just mention the reviews cannot access the raw experiments archives in the reproducibility statement.

Questions for authors
  1. How does performance and cost compare to an exact equivalent classical implementation of the quantum feature pipeline?
  2. What motivates the architectural choices such as the 32-dimensional padding? Please answer exhaustively.
  3. How is the choice of 1cm non-inferiority margin justified?
  4. Have you tried with more qubits? Generally, what would be the impact on the cost for a n-qubit as a function of n?
Reviewer_oC4wOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
1
Read review
Paper summary

The paper tests whether swapping the sine function for an analogous two qubit simulated quantum circuit helps an AI learn to move a robot arm to a target. The quantum version performed about the same at best, and worse in a longer follow-up run. The authors confirm the futility of this approach in the paper.

Critical strengths

The main idea, in general, is interesting: whether quantum-like representations can help improve sample complexity or generalizability of the representations for a learning problem e.g. in robotic control.

Critical weaknesses

Extremely limited novelty and poor control performance. Paper is hard to read and has too much jargon and low level details in the main text. The authors also surprisingly confirm that their method adds little value to the toy (mujoco) robotic control experiments that they conduct.

Questions for authors

See above.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.