acceptodds
Under review as a conference paper at ICLR 2027

Learning and Interpreting Policies for Simultaneous Entanglement Requests in Quantum Networks

Abstract

Future quantum networks will make use of entanglement to perform numerous tasks, such as sending quantum information over long distances, distributed quantum computing, and quantum sensing. In general, these tasks will need to be performed simultaneously in various regions of a network, while minimizing resources and latency. We will thus require policies for scheduling link-level entanglement resources, and using the link-level entanglement to create various forms of multipartite entanglement required for every task. In this work, we address this problem using reinforcement learning. We formulate a Markov Decision Process for the problem and use double deep Q-networks (DQN) with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training to obtain policies. The key physical parameter is the probability of link-level entanglement generation, i.e., the link activation probability. We show that our policies maintain 100% success for up to 71% lower link activation probability than the baseline heuristics for a set of physically relevant network topologies. We then examine an additional constraint where experiment (task) placements are restricted to specific hardware types and demonstrate a similar advantage in performance over heuristics, with our policy maintaining at least an 80% success rate for up to a 59% lower link activation probability. Finally, to understand how our policies obtain their advantage, we explore methods to interpret the learned policy by defining metrics enabling conclusions to be drawn about the model's behavior and by tasking a large language model (LLM) to derive a novel heuristic given example actions taken by the DQN-trained policy. We find that the LLM heuristic performs similarly to the DQN-trained policy in performance, indicating a promising method for interpretable policy extraction for large quantum networks, where direct training becomes computationally expensive.

Primary area:
applications to physical sciences (physics, chemistry, biology, etc.)
Keywords:
scheduling, quantum networks, deep q-networks, large language models, interpretation

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 3 reviews, above 27% of the others.
Reviewer_2pPkOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the scheduling of entanglement resources when multiple quantum experiments are executed concurrently in a quantum network. The goal is to improve the task success rate and shorten execution time under constraints such as random activation of physical links, the finite lifetime of entanglement resources, and the need to generate virtual links through entanglement swapping. The scheduling process is modeled as a Markov decision process and solved using double deep Q-networks (DQN) combined with Message Passing Neural Networks (MPNNs), experience replay buffers, and curriculum training. In addition, an LLM is used to extract interpretable policies from DQN trajectories. The paper has two main contributions. First, through reward function design, it guides the model to jointly consider link lifetime, topology structure, bottleneck nodes, and experiment complexity, and uses reinforcement learning to obtain an efficient policy. Second, it explores the use of large language models to extract interpretable policies, extending the solution to large-scale networks that are difficult to train with reinforcement learning. Simulation experiments are conducted on three network topologies, and the proposed method is compared with three greedy heuristic algorithms. The experimental results show that the proposed method achieves a higher task success rate and shorter average execution time under low link activation probability, and the performance of the policy extracted by the LLM is similar to that of the DQN-trained policy.

Critical strengths
  1. The task setup covers random link activation, lifetime, swapping consumption, experiment placement, and node compatibility, requiring a unified decision that trades off waiting against resource consumption.

2.Three behavioral metrics make waiting and path preferences observable, and the concise LLM-generated policy offers a low-cost deployment candidate.

Critical weaknesses

The experimental evidence is very limited and cannot prove that the method described in this paper is truly applicable. The experimental setup only uses a fixed number of nodes, entanglement lifetime, and workload, which does not support the effectiveness of the method in comparison to the baseline on other task sets, and the selected values have not been explained. Moreover, the lack of ablation experiments makes it difficult to evaluate the design of the reward component.

Questions for authors
  1. The current baseline consists of three simple heuristic greedy algorithms, which seem rather weak. Was it because there were no similar previous works available as the baseline?
  2. Could the author provide more evidence for different workloads?
  3. How stable are the strategies generated by LLM in terms of repeated generation, different prompts, or different LLMs?
1 comment

Thank you for taking the time to review our work and for your questions.

The use of greedy baselines should not be interpreted as a claim that no relevant prior work exists. There are related optimization and learning-based approaches to quantum-network resource scheduling. In our introduction, we highlight the body of literature devoted to solving the problem of routing entanglement swaps through the network. We consider this problem a subroutine within distributed quantum computation. In our work, we do not optimize the route satisfying a virtual link request but instead optimize the timing and choice of endpoints to support subsequent experiment placement.

The three heuristics allow us to assess whether the learned policy improves on simple scheduling rules under our formulation. They were selected to represent complementary scheduling priorities of minimizing virtual-link age, minimizing virtual-link span, and incorporating an initial form of topology-aware decision-making through the DCTR baseline. However, we agree that covering these complementary priorities does not establish competitiveness against stronger optimization or learning-based methods that may combine them. We reference Ni et al. (2026) in our introduction as their work similarly formulates quantum job scheduling as a queue of jobs that need to be assigned to available QPUs on a network. Although their work motivates our application-level scheduling problem, the scheduling interfaces differ. Ni et al. consider a fully connected QPU network and abstract entanglement provision as a network service (Section 3.2). Their schedulers select jobs and allocate QPUs using quantities such as resource demand, EPR usage, and estimated execution time. In contrast, our policy explicitly chooses among waiting, virtual-link operations, and experiment placements, with placement feasibility depending on the current availability and age of the required entangled pairs. Their algorithms do not directly specify the intermediate virtual link requests required by our environment.

We agree that evaluation across a broader range of workloads would strengthen the evidence for our approach. The current experiments establish performance under the workload settings reported in the paper, but do not establish robustness to changes in workload size, experiment composition, or resource requirements. A systematic assessment of whether the observed advantages over the baselines persist for different workloads is left to future work.

We did not systematically evaluate stability across repeated LLM generations, prompt variations, or different models. Our results characterize the performance of the particular generated strategies evaluated, rather than the consistency of the generation procedure. We therefore regard generation stability as an open question and a limitation of the present evaluation.

Reviewer_a7mVOpenReview ↗
Rating
2
Confidence
1
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies scheduling simultaneous entanglement requests in quantum networks using reinforcement learning. The authors formulate the problem as an MDP and train Double DQN policies with MPNNs, replay buffers, and curriculum learning. Experiments on several simulated network topologies show improved success rates and episode lengths compared with heuristic baselines. The paper also studies hardware-placement constraints.

Critical strengths
  1. The paper addresses simultaneous experiment scheduling rather than individual entanglement requests, including hardware-placement constraints.

  2. The behavioral metrics and LLM-based extraction provide an interesting attempt to understand what the RL policy has learned rather than treating it entirely as a black box.

Critical weaknesses
  1. The main experiments use three 9-node topologies, with separate models trained for each topology. Therefore, the results mainly demonstrate performance within the training topology rather than generalization to unseen network structures.

  2. The learning framework primarily combines established components such as Double DQN, MPNNs, replay buffers, and curriculum learning. The comparisons are against relatively simple hand-designed heuristics, making it difficult to determine how much improvement would remain against stronger optimization or learning-based scheduling methods.

Questions for authors
  1. How well does a policy trained on one topology generalize to unseen network topologies or different network sizes without retraining?
  2. Could the authors compare against stronger optimization or learning-based scheduling baselines beyond the three greedy heuristics?
1 comment

Thank you for taking the time to review our work and for your questions.

Regarding your first question, we designed the training architecture to achieve the best performance on a fixed physical topology that is known a priori. This setting is motivated by experimental demonstrations: Pompili et al. (2022) execute multiple applications consecutively on an unchanged network setup, while Craddock et al. (2024) demonstrate entanglement distribution over deployed fiber for 15 days with 99.84% uptime. These demonstrations motivate reusing a topology-specific policy across multiple scheduling tasks. We acknowledge that the fixed topology prerequisite of our policy leaves it susceptible to unpredictable topology changes caused by hardware failures, for example. However, we do investigate the ability of a policy that is classically trained on a small instance of a network topology to inform a heuristic that generalizes well to larger topologies of the same family. Figure 13 in Appendix F shows sustained superior performance of an LLM-generated heuristic over baselines for increasing sizes of uncolored starlink topologies. Note that the LLM was only given several policy trajectories for the N=9N=9 case, and yet the LLM-generated policy maintains a strong advantage for network sizes up to N=29N=29, the largest size tested. This generalization to larger topologies of the same network family was obtained without further reinforcement-learning retraining of the policy.

We agree that comparison with stronger optimization or learning-based schedulers would strengthen the evaluation. Our three greedy heuristics were selected to provide interpretable reference policies that emphasize temporal, spatial, and topology-related scheduling priorities. We acknowledge this limitation and leave a systematic comparison with optimization and learning-based baselines to future work.

Reviewer_eDoDOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines how to schedule entanglement in a small quantum network so that a set of “experiments” can all be placed as quickly as possible. An experiment is a graph of entangled pairs that must all be active simultaneously. Links between neighboring nodes are entangled pairs that randomly appear, age, and expire. Longer links are created by swapping along a path. The authors model this problem as an MDP and train a double DQN using a message passing network, a replay buffer, and a curriculum over increasing noise levels (essentially decreased probabilities of success). On three 9-node topologies, the learned policy continues to succeed at lower link activation probabilities compared to three greedy heuristics.

Critical strengths
  • The paper is well-written and generally easy to follow. The problem is set up step-by-step in Section 2, the figures are clear and positioned close to the text they support, and each reward term includes a brief explanation of its purpose, alongside a worked example for the edit distance term.

  • The problem is timely and motivated. Quantum network testbeds are being deployed in several cities, early distributed quantum computing experiments have been demonstrated, and it is reasonable to expect that a network operator will soon need to serve multiple users whose jobs have varying connectivity needs on a fixed physical topology.

Critical weaknesses
  • The quantum modeling part is ambiguous, and the ML contribution is modest, leaving the paper caught between two audiences. The term “link” is used for the fiber, a memory slot, and an active Bell pair; “physical” simply means “between neighbors.” Experiments are never tied to an actual task, colors are not explained as hardware types, and swapping is treated as deterministic no matter the platform. The up-front reservation rule (requiring all pairs to be alive at once and consumed together after d_k steps) lacks justification; real circuits use pairs one gate at a time. On the ML front, this is a standard double DQN using an MPNN and a curriculum, applied to a single fixed instance per topology. There is no novel method or analysis presented.

  • There is no comparison to prior work, even to the work the paper explicitly cites. The introduction lists LP methods, a decision transformer, several RL papers, and the heuristics of Ni et al. (2026), which the authors state is the source of their problem. None of these are used as baselines. The three baselines are simply greedy rules written by the authors. Furthermore, the paper does not explain why these existing methods were left out.

Questions for authors
  • Why must all links of an experiment be reserved at once and consumed together? Which application needs this?

  • What is a link physically? It reads as a pair of memory slots, not a channel. How many memories per node, and are they dedicated to one neighbor?

  • Can you add a no-trajectory LLM control? We can't be sure that the policy helped at all.

  • What is the size of the action space, and how does it scale with N and experiment size?

  • Why there is no comparison with other published methods?

1 comment

Thank you for your comments and questions. We greatly appreciate the time taken to review our work.

The requirement that all requested Bell pairs be available before experiment execution is motivated by the work of Pompili et al. (2021, Section 3), cited in our paper. In their work, the authors prepare a three-party GHZ state by first generating and storing Alice–Bob entanglement while establishing Bob–Charlie entanglement, then combining both resources. Reserving all required Bell pairs throughout execution and consuming them together at completion is a coarse scheduling abstraction and a limitation of our model. Our request interface specifies the required entanglement topology and reservation duration, without exposing the internal circuit or its sequence of resource consumption. We therefore treat each experiment as an indivisible scheduling unit whose complete resource set must be available before execution and remains reserved until completion.

It is true that internal access to the user's circuit may enable the network to begin executing it earlier, potentially improving network efficiency. However, to support a simpler request interface, we assume no internal knowledge of the user's circuits that would otherwise make only a subset of required pairs necessary to begin the computation. This interface is similar to existing services like Amazon Braket Direct, in which a user can reserve a single QPU for a specified time interval during which they can submit circuits for execution.

In our evaluation, all experiments last one simulation time step, limiting the interval during which links are reserved after assigning an experiment. However, we have not quantified the effect of allowing intermediate resource release. Relaxing these requirements, while preserving the remaining dynamics and objective and retaining all original schedules as feasible options, could only improve or leave unchanged the optimal achievable objective. This observation does not establish a corresponding guarantee for our learned policy or its relative performance against the baselines.

A link exists if the two halves of a Bell pair are shared between two quantum memories of two distinct nodes. The number of memories per node depends on the network topology. Between two physically linked nodes, μ\mu independent links may be shared. Thus a node uu has μ×deg⁡(u)\mu \times \deg(u) total communication memories, where μ\mu memories are dedicated to each physical neighbor of uu. In our evaluation, μ=5\mu = 5.

We agree that we cannot be sure the LLM made use of the supplied trajectories. A controlled comparison with and without trajectories, using independent generation contexts and repeated generations, is left to future work, alongside an assessment of sensitivity to different prompts and LLMs.

The action space is composed of a wait operation, virtual link operations, and experiment placement operations. For a network size NN, the number of virtual link operations is the number of edges in a complete KNK_N graph with ∣E∣|E| physical edges removed. Thus the number of virtual link actions scales with N2N^2 and is larger for sparse networks. The number of experiment placement operations is the sum (for all experiments in X\mathcal{X}) of the number of mappings of an experiment's graph onto a subgraph of KNK_N. For placements of a complete experiment graph KkK_k in KNK_N, there are (Nk)\binom{N}{k} such mappings in the uncolored case. Thus ∣A∣=1+(N2)−∣E∣+∑x∈X(Nkx)|\mathcal{A}| = 1 + \binom{N}{2} - |E| + \sum_{x \in \mathcal{X}} \binom{N}{k_x} which scales as O(Nmax⁡(2,k))O(N^{\max(2, k)}) for a fixed number of complete-graph experiments of maximum size kk as in our evaluation.

Ni et al. (2026), cited in our introduction, address the related problem of assigning queued quantum jobs to available networked QPUs. Their formulation motivates our application-level scheduling setting but operates at a different level of abstraction. Specifically, they assume a fully connected QPU network with entanglement supplied as a network service (Section 3.2) and schedule jobs using resource requirements, EPR consumption, and estimated execution times. Our scheduler additionally controls virtual-link operations, choosing between these operations, experiment placement, and waiting according to the availability and age of entangled pairs. Applying their methods in our environment would require an additional mechanism to determine the intermediate virtual-link requests needed to enable experiment placement, making direct comparison difficult.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.