- Rating
- 3
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The paper presents a study on joint communication-route construction and intermediate-switch placement in a simplified, obstacle-aware interconnect model with fixed initiator and target locations. Its central contribution is a hierarchical graph construction that alternates switch expansion, placement on an extended Hanan grid, and route refinement. The construction process is formulated as a deterministic, sequential MDP whose state records the current route graph, switch positions, construction phase, and queues of switches and communication routes awaiting processing. Depending on the phase, the agent selects a switch to expand, assigns valid Hanan-grid positions to the affected switches, or chooses one of four local routing alternatives for each affected communication pair. Once the placement and refinement queues are emptied, the graph again represents a feasible routing configuration, and the process repeats until the prescribed switch budget is reached. The restricted construction space retains at least one globally optimal solution under their model.
A neural network jointly encodes the current route graph and the floorplan, including blockage geometry and legal switch locations. Its policy heads assign probabilities to the admissible expansion, placement, and route-refinement actions, while its value network estimates the return associated with the current state. Gumbel MCTS uses these action probabilities and value estimates to guide its simulations and improve action selection, using an upper-tail quantile-based search value and per-floorplan PopArt normalization. The terminal objective combines distinct physical-connection wirelength with one half of the summed communication-route lengths. Evaluation uses 24 synthetic training floorplans and four held-out transfer instances. The reported best-found objectives on the training instances favor Gumbel MCTS over the evaluated heuristic, random-search, genetic-algorithm, and PPO-EWMA baselines. Three-run fine-tuning experiments suggest benefits from pretraining, although Gumbel MCTS does not uniformly outperform PPO-EWMA on the held-out instances. The study does not model component footprints, detailed physical-design rules, congestion, or bandwidth constraints.
Critical strengths
-
A structured construction for a coupled optimization problem. The route-node representation and local expansion/refinement operations provide a concrete way to optimize intermediate-switch placement and shared communication routes together. Preserving assigned-route validity after each completed cycle removes the need to recover routing connectivity from arbitrary candidate graphs. The distinction between route-specific assignments and shared physical connections is also consequential to the objective. The main technical contribution is this construction and its representational properties.
-
Initial evidence for learned search and transferable initialization. The experiments compare multiple optimization paradigms on the same stated objective and use a common stated GPU/wall-clock budget for the four iterative methods. The separate held-out fine-tuning experiment compares pretrained and randomly initialized models over three runs and reports wall-time curves and variability. These are useful elements of the evaluation, even though the breadth and statistical support remain limited. The explicit limitations discussion and detailed solution visualizations help delimit and inspect what has been demonstrated.
Critical weaknesses
Critical Weaknesses
-
The evidence is insufficient to establish a robust advantage or isolate the contribution of Gumbel MCTS. Appendix C.3 explicitly states that only one pretrained model is trained for each learning method, while Table 1 reports best-found objectives on the 24 training instances without variability across independent repetitions of the complete training/search procedure. These results are informative for the reported runs, but do not establish the stability of the algorithm ranking. Figure 5 and Tables 7-8 appropriately report mean ± one standard deviation over three independent fine-tuning runs, thereby quantifying variability conditional on the selected initialization. However, Appendix C.3 states that only one pretrained checkpoint is trained for each learning. Consequently, the results do not quantify variability across independently pretrained checkpoints. The authors should either repeat pretraining with multiple seeds or limit their robustness claims to the variability of fine-tuning from the particular checkpoint used. With only three runs, showing the individual-run results would also improve transparency. The comparison between Gumbel MCTS and PPO-EWMA should use explicitly matched computational budgets. Matching the number of PPO rollouts to the number of MCTS simulations is not sufficient, because these operations may have different costs. Please report best-so-far objective versus wall-clock time, the number of complete trajectories, environment transitions, network evaluations, MCTS simulations, and PPO updates, as well as the heuristic runtime. As an additional control, evaluate a fixed PPO policy through repeated stochastic rollouts under the same measured budget as Gumbel MCTS and retain the best solution. Ablations using mean versus upper-tail value estimates, conventional MCTS, and different MCTS budgets would clarify whether the gains come from tree search itself, its value estimator, or simply additional computation.
-
The scope of validation is too narrow to establish broader generalization or practical relevance. Table 1 concerns the training instances, while transfer is evaluated on only four held-out synthetic floorplans. The switch budgets range from 2 to 5, and the instances contain 18-25 required communication pairs. A broader held-out evaluation, a reproducible instance-generation procedure, and scaling experiments are needed to establish how the method behaves across different communication structures and geometric difficulty. Reporting training-instance optimization results is legitimate, but those results should not be presented as evidence of unseen-instance generalization. The simplified abstraction is clearly acknowledged; nevertheless, its relationship to a realizable communication network remains unvalidated. Optimizing and evaluating the stated objective is appropriate, but an improvement in that objective alone does not establish improvements in chip area, congestion, timing, or bandwidth. Appendix D usefully separates wirelength and route length; additional analyses such as maximum route length or route stretch, sensitivity to the objective weight, and a modest experiment with a relevant physical or capacity constraint would better characterize the trade-offs. These measures should be chosen to match the intended application, they should not be replaced by fixed floorplan area or Appendix D's input free-space percentage, neither of which measures the quality of an optimized placement in this model.
Secondary Concerns:
-
The provenance and physical scope of the dataset should be clarified. The paper states that the experiments use synthetic floorplans, but it does not sufficiently describe how initiator and target locations, communication pairs, blockage geometries, circuit or communication topologies, and switch budgets are generated. It is also unclear whether the instances are intended to represent abstract NoC/interconnect problems or physical circuits from a particular technology. Because component dimensions, shapes, pin locations, and technology-specific constraints are ignored, the authors should explicitly state whether all components are modeled as zero-size points, whether component variants are supported, and how these assumptions limit the applicability of the method to physical EDA. A specific PDK is not necessarily required for this abstract study, but the intended problem scope and relationship between the synthetic instances and real design scenarios should be made explicit.
-
If accurately characterized, the following related work should be cited and discussed: Wang et al., "Exploring Search for FPGA Placement using RL", in RLC 2025 Workshop on Practical Insights into RL for Real Systems (RL4RS), 2025, available at https://openreview.net/pdf?id=nvge2HodpL. This paper explores the use of the Gumbel MCTS algorithm for FPGA placement. Although it addresses a different problem than the one studied here, both works fall within the broader area of RL for chip placement and, in particular, both employ the Gumbel MCTS technique. Prior work in this area should therefore be properly cited and discussed.
-
The novelty discussion relative to Cheng and Yan (2021), Chen et al. (2022), and Geng et al. (2024) should be sharpened. In particular, the proposed route-node construction and intermediate-switch topology/placement decisions should be distinguished from prior joint placement-and-routing learning and prior RL/tree-search formulations. The current brief description of Cheng and Yan understates its explicit joint-learning formulation.
-
The global-optimality guarantee in Proposition 4.8 is central to the contribution, but the induction in Appendix A.3 is difficult to follow as written. Lemma A.5 supplies a common-subpath property for a selected globally optimal routing configuration, the proof does not establish that this property, or another sufficient contractibility invariant, survives the successive contractions. It is not preserved by arbitrary contractions in a general route family: the switch sequences , , and satisfy the property, but contraction of yields , , and , which do not. Please supply the missing invariant or a contraction-selection argument guaranteeing that the construction can be completed.
-
The broad claim of "consistently outperforming direct policy optimization" requires further qualification, since Tables 7-8 show lower final mean objectives for pretrained PPO on two of the four held-out instances. The benefit of pretraining should be separated from the ranking of the two algorithms.
-
Section 5 says the value function is trained toward the top 25% of observed returns. Appendix B describes eight quantile outputs, quantile Huber training, interpolated targets, and use of the two largest predicted quantiles for the scalar search value. Training a quantile predictor and using an upper-tail summary at search time should be distinguished from discarding all but the top 25% of observed training returns. The main text description should be aligned with the actual loss and target construction.
-
The formal representation has one route node per communication use of an edge, whereas the neural implementation describes route-bundle nodes that can represent shared segments. The mapping between them, and how route-specific decisions are preserved, would benefit from a short explicit explanation.
-
Section 5's final implementation details pointer names Appendix C, although the architecture and learning hyperparameters are in Appendix B.
-
Reference "Yunqi Shi, Ke Xue, Song Lei, and Chao Qian. Macro placement by wire-mask-guided black-box optimization. Advances in Neural Information Processing Systems, 2023." requires an author name correction: Lei Song, not Song Lei.
-
It is recommended to protect words with capitalized letters in the LaTeX bibliography items with braces, otherwise they are not rendered properly. For example {RePlAce}, {Gumbel}, {Steiner}, {HubRouter}, {NeuralSteiner}, {REST}, {PopArt}, {NoC}, {VLSI}, and {NP}. The authors should check for other missing capitalized letters, for example in journal or conference titles.
-
It is also recommended to add volume, issue, pages/article number, and DOI or stable URL where the required style permits and the fields can be verified. The current entries are generally identifiable, but sparse. For example, volume, issue and page numbers are expected for journal articles.
Questions for authors
-
How were the 28 synthetic floorplans generated, and what real design setting are they intended to represent? Please specify the distributions or procedures used for component locations, communication topologies, blockage geometries, and switch budgets, clarify whether component variants, dimensions, shapes, or technology-specific constraints are outside the intended scope of the current framework, and how the four held-out instances were selected.
-
Can you provide a complete proof of Proposition 4.8? In particular, what invariant guarantees that the successive contractions in Appendix A.3 can always be continued until a single-switch configuration is obtained?
-
In Appendix E, are the reported and visualized solutions terminal solutions after reaching the maximum switch budget, or are they the best intermediate solutions found during optimization?
-
How many complete trajectories, environment transitions, neural-network evaluations, MCTS simulations, PPO updates, and genetic-algorithm generations are performed during the 48-hour experiments?
-
Can you provide best-objective-versus-wall-clock-time curves for all iterative methods, together with the measured runtime of the heuristic baseline?
-
Can you evaluate a frozen PPO-EWMA policy using repeated stochastic rollouts under a computational budget matched to Gumbel MCTS, retaining the best solution found? Please define the matching budget using measured computational cost rather than simply equating PPO rollouts with MCTS simulations.
-
How much of the observed improvement is due to tree search, upper-tail value estimation, and the specific Gumbel MCTS procedure? Please provide ablations using mean-value search, conventional MCTS, and different numbers of MCTS simulations.
-
Since only one pretrained checkpoint is produced for each learning algorithm, how sensitive are the transfer results to the pretraining seed? Could you report results from independently pretrained checkpoints, or limit the transfer claims to fine-tuning variability conditional on the selected checkpoint?
-
What is the exact mapping between the formal route-node representation in Definition 4.1 and the route-bundle representation used by the neural-network implementation?
-
Why was lambda = 1/2 selected in the objective? How sensitive are the results to this weight, and how do the individual wirelength and route-length components change across methods?
-
Given the simplified point-based physical model, which claims are intended to generalize beyond this abstract setting, and which omitted constraints are expected to be most important in future extensions?
5 comments
Ablations using mean versus upper-tail value estimates, conventional MCTS, and different MCTS budgets would clarify whether the gains come from tree search itself, its value estimator, or simply additional computation.
In the following table, we compare Gumbel MCTS with and without the risk-seeking value estimator, for a single seed run in 24h on a single RTX6000PRO, with 2048 trajectories per collection, which is a weaker setup compared to our paper’s pretraining experiments (which used 6 L40S during 48h) due to time and computational constraints.
| Instance | Scalar MCTS | Risk-seeking MCTS |
|---|---|---|
| 1 | 11.689 | 11.486 |
| 2 | 11.224 | 11.508 |
| 3 | 15.933 | 15.043 |
| 4 | 9.915 | 10.563 |
| 5 | 14.204 | 14.218 |
| 6 | 10.957 | 10.571 |
| 7 | 8.898 | 8.254 |
| 8 | 10.785 | 10.319 |
| 9 | 11.208 | 11.218 |
| 10 | 14.153 | 13.366 |
| 11 | 13.537 | 14.012 |
| 12 | 13.380 | 13.551 |
| 13 | 13.287 | 13.648 |
| 14 | 13.346 | 13.645 |
| 15 | 14.134 | 14.148 |
| 16 | 15.248 | 15.269 |
| 17 | 15.239 | 15.239 |
| 18 | 14.335 | 14.886 |
| 19 | 14.492 | 15.322 |
| 20 | 15.817 | 15.726 |
| 21 | 12.971 | 13.106 |
| 22 | 13.478 | 13.903 |
| 23 | 12.785 | 13.070 |
| 24 | 15.297 | 15.277 |
Risk-seeking value estimation doesn’t strongly affect the performance. Please also note that those values are not final, and that more than 24 h is required for the algorithm to converge.
We also provide a comparison between learned Gumbel MCTS and non-learned MCTS, where the prior actions are selected uniformly, and the values are estimated using 800 random rollouts per decision, on the fourth held-out floorplan. We use a single seed for the non-learned version, run on a single L40S for 24 h.
| Method | Wirelength | Routelength | Objective |
|---|---|---|---|
| Non-learned MCTS | 11.879 | 30.168 | 26.963 |
| Gumbel MCTS, pretrained | 3.907 ± 0.000 | 20.620 ± 0.000 | 14.217 ± 0.000 |
| Gumbel MCTS, scratch | 2.852 ± 0.009 | 23.134 ± 0.215 | 14.419 ± 0.116 |
We finally provide an evaluation of Gumbel MCTS using the pretrained checkpoint with a varying number of simulations, on instance 18, which is the largest floorplan of our pretraining set. Mean and standard deviation are computed over 1024 trajectories.
| Simulations/move | Trajectories | Mean objective | Std | Best objective | Median | Runtime (s) |
|---|---|---|---|---|---|---|
| 0 | 1024 | 14.622108 | 0.485391 | 14.178902 | 14.504927 | 111.766 |
| 8 | 1024 | 14.543460 | 0.350538 | 14.184241 | 14.454237 | 144.862 |
| 32 | 1024 | 14.558215 | 0.335296 | 14.166741 | 14.476794 | 228.776 |
| 128 | 1024 | 14.587810 | 0.456702 | 14.163643 | 14.479611 | 378.389 |
| 400 | 1024 | 14.540454 | 0.284910 | 14.159283 | 14.460787 | 669.971 |
| 800 | 1024 | 14.497172 | 0.218661 | 14.159283 | 14.443811 | 996.678 |
| 1600 | 1024 | 14.471442 | 0.263185 | 14.159281 | 14.418614 | 1543.999 |
| 3200 | 1024 | 14.461724 | 0.382405 | 14.152742 | 14.393384 | 2819.967 |
W2, W3, and Q1.
A broader held-out evaluation
Each held-out floorplan is costly, as it requires four additional experiments (one for both PPO-EWMA and Gumbel MCTS, for both pretrained initialization and fine-tuning). We believe four held-out floorplans to be enough to demonstrate that a pretrained initialization speeds up convergence.
a reproducible instance-generation procedure
Each instance consists of two components: (i) a communication graph describing the initiators and targets, their positions on the floorplan, and the required initiator-to-target communications; and (ii) a rectangular floorplan containing rectangular obstacles (blockages).
To generate diverse instances, we vary three main parameters:
-
Floorplan coverage: the fraction of free space remaining after placing rectangular obstacles, allowing us to generate floorplans with different levels of geometric obstruction. Number of initiators and targets: controlling the number of communicating components.
-
Connectivity percentage: the proportion of possible initiator-to-target communications included in the graph. A connectivity of 100% corresponds to full connectivity, where every initiator communicates with every target.
-
Blockages are positioned with varying sizes and aspect ratios to create diverse spatial layouts and routing constraints. Initiators and targets are placed at different locations across the floorplan, outside blockage interiors. Required communications are then selected as a subset of the possible initiator–target pairs, according to the desired connectivity percentage. We will clarify this generation procedure in the revised manuscript.
scaling experiments are needed to establish how the method behaves across different communication structures and geometric difficulty
Scaling to larger instances is difficult due to our limited computational availability. We evaluated our method to the best we could under our limited computing availability.
The simplified abstraction is clearly acknowledged; nevertheless, its relationship to a realizable communication network remains unvalidated.
For a discussion about the simplified abstraction, please refer to the answer to Reviewer bg2t.
Reporting training-instance optimization results is legitimate, but those results should not be presented as evidence of unseen-instance generalization.
We do not claim that our method generalizes to unseen floorplans in zero-shot. Our fine-tuning experiments show that a pretrained checkpoint provides a useful initialization for unseen floorplans.
These measures should be chosen to match the intended application, they should not be replaced by fixed floorplan area or Appendix D's input free-space percentage, neither of which measures the quality of an optimized placement in this model.
The input free-space percentage is specific to the floorplan instance and is not modified by our algorithm. It is only a characteristic of the problem instance.
W4. Gumbel MCTS is just a modern variant of MCTS and is not related to our core contribution, which is the hierarchical space construction. We will add a citation to the paper in the related work, along with all the other RL for EDA works already cited.
W5. Due to space constraints, we briefly discussed these citations. Besides, as clarified in Cheng et al. (2022), Cheng and Yan (2021) use a heuristic rip-up-and-reroute algorithm to provide routing feedback to the placement policy. Their codebase does not contain any RL method for routing eithertoo. In contrast, our method directly includes routing decisions in the learned search process, alongside switch placement and topology decisions.
Chen et al. (2022) also use reinforcement learning and MCTS but for global routing, at the wire level, while we represent each route explicitly. Moreover, our hierarchical switch-expansion and route-refinement operations are designed to preserve the validity of these routes throughout the construction, while they predict Steiner points on a fixed regular grid, not using the Hanan grid.
Finally, Geng et al. (2024) combine reinforcement learning and MCTS for macro placement, not routing. We also stress that RL-based search is not our core contribution.
W6 and Q2. We thank the reviewers for identifying the gap in Proposition 4.8. We agree there is a gap in the argument. We will promote Lemma A.5 to a standalone proposition and replace Proposition 4.8 with two complementary theoretical results.
1. Exact characterization of the hierarchical search space. We establish the following exact characterization:
A set of simple communication routes is expressible through switch expansions and route refinements if and only if its switches admit a sequence of pairwise contractions reducing them to a single switch, such that no contracted communication route contains the same switch twice at any step of the sequence.
2. Optimality coverage for ()-embeddable switch networks.
We further establish that our hierarchical search space contains a globally optimal solution whenever an optimum exists whose switch-network shortest-path metric is isometrically embeddable into . This sufficient condition covers a broad family of networks, including trees, forests, cycles, cactus graphs, four-vertex diamond graphs, weighted outerplanar graphs, and, more generally, -minor-free graphs.
Moreover, since every metric on at most four points is -embeddable (), we obtain the following corollary:
For every instance with , our hierarchical search space contains a globally optimal solution, independently of the floorplan geometry, obstacles, and communication topology.
W7. Gumbel MCTS outperforms or is equivalent to PPO on all of the pretraining floorplans, not fine-tuning. Fine-tuning experiments last only 24h, not 48h like the pretraining experiment, and MCTS takes longer than PPO to fully converge.
W8. This is just a minor textual shortcut, that is not substantial to the understanding of the method.
W9 and Q9. Conceptually, it is the same. It is simply that the architecture aggregates the embeddings of all route nodes before message passing, in order to save compute. We will clarify that in the paper.
W10, W11, W12 and W13. We will correct this.
Questions:
Q3. It corresponds to the best solutions after reaching the final number of switch expansions. Please note that switches can be discarded, so some solutions may not have the maximum amount of switches.
Q10. Lambda = ½ is arbitrary. As discussed in the paper, lines 152-154: “The wirelength encourages compact and shared wires, while the route length discourages long communication paths between individual initiator-target pairs”, and lambda controls which component is favored.
The wirelength and route length of all found solutions are already provided in Appendix D.
Weaknesses:
W1, Q4, Q5, Q6, Q7 and Q8.
The pretraining run is very costly (running on 6 L40S for 48 hours), so we report only one seed due to computational constraints. The clear gap in performance across the 24 floorplans between the different methods is too large to be solely attributed to seed randomness.
For transparency, we report here the individual runs for the four held-out floorplans.
We already match the computational budget. As stated in the paper, lines 408-410: “All methods except Heuristic are run for 48 hours on the 24 pretraining floorplans. Heuristic is instead run once as a deterministic constructive procedure”.
We can provide best-so-far objective versus wall-clock time curves if necessary. However, as we have a total of 28 floorplans and 4 methods needing a curve, this would add an additional 27 pages of figures in the Appendix, where we already have 27 pages of visualizations. Rather, we will make the Weight&Biases run public upon acceptance of the paper, which already contains those curves.
About the rest, for the experiments on the pretraining dataset, we estimate approximately 3.50 million complete trajectories for MCTS, 185 million for PPO, 1.36 billion for random search, and 1.18 billion for genetic search. The corresponding environment-transition counts are at most 141 billion for MCTS, and exactly 9.22 billion, 66.2 billion, and 50.3 billion for PPO, random search, and genetic search, respectively. The number for MCTS is based on 800 simulations per decision, as tree reuse allows us to keep part of the simulations done during the previous search. Collection/search network evaluations are estimated at most 141 billion for MCTS under the same upper-bound approximation and 9.40 billion for PPO; random and genetic search require no learned-network evaluations. PPO and MCTS perform approximately 377,000 and 71,700 optimizer updates, respectively.
We report here the runtime for the heuristic:
In our experiments, PPO and MCTS are already trained using the same machine and for the same duration.