- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper proposes COMPAS, a reinforcement learning framework for sequential macro placement. It addresses the problem that dense proxy signals available during placement, such as macro-placement HPWL, may not reliably reflect the final placement quality after full global placement (GP), while running GP evaluation after every placement decision is prohibitively expensive. In addition to standard PPO training, COMPAS periodically sends completed macro placements to a GP evaluator and uses the resulting feedback through two pathways: 1) Layouts with improved GP-HPWL are stored in a reference memory and subsequently used to guide the placement prior and wirelength optimization. 2) Selected evaluation trajectories are incorporated into Adaptive Self-Imitation (ASI) replay. The evaluator feedback is ultimately used to update the placement policy and adapt the coefficient controlling the placement prior.
Critical strengths
1.Practical engineering value. The paper develops a complete and effective reinforcement learning framework for macro placement, demonstrating meaningful potential for practical applications.
2.Detailed methodological and implementation descriptions. The paper provides substantial detail about its individual components and training procedure, including trajectory selection, replay objectives, and reference memory management. These descriptions are helpful for reproducibility.
3.Comprehensive reporting of physical metrics and computational costs. The experiments report detailed physical metrics as well as training overhead, which is valuable for evaluating an application-oriented placement method.
Critical weaknesses
1.The central motivation lacks quantitative validation. The paper is motivated by the claimed mismatch between intermediate macro-placement metrics and downstream placement quality after global placement. However, it does not provide sufficient quantitative evidence demonstrating the severity or prevalence of this mismatch. For example, the authors could report the rank correlation between macro-placement HPWL and post-GP metrics across training checkpoints, layouts, and benchmarks. Such an analysis would clarify when GP feedback is necessary and how much information is missed by the dense proxy reward.
2.The framework is highly engineered, but the necessity of its components is insufficiently established. COMPAS introduces reference memory, RPP, additional policy updates, adaptive constraints, and other interacting mechanisms. The current experiments do not sufficiently isolate the contribution of each component. As a result, it is difficult to identify the essential methodological contribution or determine whether the gains arise from the proposed feedback principle or simply from the accumulation of several engineering techniques.
3.GP feedback does not lead to consistent improvements across all physical-quality metrics. As shown in Table 2, COMPAS underperforms EXPlace on the rO-H and rO-V metrics. This suggests that COMPAS does not comprehensively improve physical placement quality and that GP-HPWL may still be an incomplete proxy for downstream physical objectives. The paper should analyze the trade-off between HPWL and routing-related metrics and explain why optimizing GP-derived feedback can degrade horizontal or vertical routing overflow.
4.The argument for introducing ASI is incomplete. The paper argues that GP evaluation is expensive and therefore introduces ASI to reuse sparse GP feedback. However, the high cost of GP evaluation alone does not establish that the relatively complex ASI mechanism is necessary. The authors should compare ASI against simpler methods for exploiting GP-improved layouts, such as periodically fine-tuning PPO on GP-selected trajectories, behavior cloning from the best layouts, reward relabeling using GP-HPWL, or maintaining a best-trajectory replay buffer. These comparisons are needed to demonstrate that ASI provides benefits beyond simpler alternatives with lower methodological complexity.
5.The ASI module lacks sufficiently fine-grained ablation studies. ASI contains multiple tightly coupled components, but their individual contributions are not clearly evaluated. The paper should separately ablate the trajectory-selection criterion, replay frequency, replay-buffer construction, imitation objective, adaptive constraint, and coefficient-update rule. It would also be useful to report how these choices affect sample efficiency, final GP quality, training stability, and computational overhead. Without such analyses, it remains unclear which parts of ASI are indispensable and whether the complete design is justified.
Questions for authors
See Weaknesses.
4 comments
Response to Reviewer aNVP (2/4) W3. Routing overflow relative to EXPlace
W3. Routing overflow relative to EXPlace
We thank the reviewer for raising this point and have included a more detailed analysis of routing overflow in the revised manuscript (Section 4.3 and Appendix D.8).
1. Per-design behavior. The rO-H difference is not uniform across designs. The rO-H is higher than that of EXPlace on six designs, equal on one, and lower on one (Appendix Table 12). Superblue1 and Superblue16 account for approximately 80% of the net gap, where rO-H is 1.19 and 0.47 percentage points higher. The wirelength outcome differs between these two designs, since GP-HPWL is 3.9% lower than that of EXPlace on Superblue1 and 9.1% higher on Superblue16 (Table 1). Vertical overflow is a more consistent limitation. It is higher than that of EXPlace on seven of eight designs, although no per-design increase exceeds 0.15 percentage points.
2. Relationship between GP-HPWL improvements and routing overflow. GP-HPWL is measured after global placement and legalization, but it does not explicitly penalize routing congestion. Since both feedback pathways are driven by HPWL-based scores, neither GP-selected reference-memory updates nor ASI replay selection directly evaluates or penalizes routing congestion. To examine the relationship empirically, Appendix Figure 13 compares GP-HPWL reductions with changes in routed metrics at 32 GP-incumbent updates from one COMPAS run per design. Routed wirelength decreases at 30 of 32 updates and broadly tracks the GP-HPWL reduction. In contrast, rO-H decreases at 18 updates and increases at 14, without a consistent relationship to the GP-HPWL reduction. Together with the per-design results, this analysis indicates that lower GP-HPWL neither guarantees lower overflow nor systematically increases it. The results suggest that routing overflow is left unconstrained by the objective, and they do not show a consistent wirelength–congestion trade-off across designs or incumbent updates.
3. Overall physical quality and limitation. The higher routing overflow should be considered alongside the other measured outcomes. Relative to EXPlace, COMPAS reduces mean routed wirelength from 121.19 m to 109.55 m (9.6%), NVP from 11.83 to 10.90 (×10³; 7.9%), and TNS magnitude from 8.59 to 4.36 (×10⁴ ns; 49%), while average WNS improves from −55.76 ns to −52.56 ns (Table 2). Thus, COMPAS improves several downstream quality measures, although routing overflow remains higher on average. Since the ASI replay score uses downstream quality as a scalar (Eqs. 3–4), congestion-aware feedback is a natural direction for future work, which we discuss in Section 4.3.
Response to Reviewer aNVP (3/4) W2 and W4. Component contributions and simpler GP-guided alternatives
W2 and W4. Component contributions and simpler GP-guided alternatives
We thank the reviewer for these constructive suggestions and agree that the contributions of individual components should be distinguished from the benefits of combining several mechanisms. Section 4.5 now organizes the component analysis around COMPAS's two GP-feedback pathways, and Appendix D.3 adds comparisons with simpler approaches that reuse GP-evaluated placements.
1. Identifying the central methodological contribution. The central contribution of COMPAS is to incorporate downstream GP evaluations into ongoing macro-placement optimization through two complementary mechanisms (Sections 3.2–3.3). GP-selected reference memory uses high-quality evaluated placements to guide subsequent placement decisions, while adaptive self-imitation (ASI) uses selected evaluated trajectories to update the policy directly. Thus, downstream feedback influences both subsequent search and policy learning. The ablations below examine whether these mechanisms contribute beyond the shared policy architecture and placement guidance.
2. Isolating the contribution of GP feedback. Figure 4 compares COMPAS against variants without GP-selected reference memory, without ASI, and without either mechanism on Superblue3, Superblue4, and Superblue18. Variants without reference memory retain the fixed initial reference for spatial guidance. Full COMPAS achieves the lowest final GP-HPWL on all three designs. The variant without either mechanism retains the policy architecture, both placement priors, and the dense PPO reward, but uses no GP feedback to guide search or policy learning. Thus, its gap to full COMPAS supports a contribution from GP-derived feedback beyond these shared design choices. Across all eight designs, full COMPAS also outperforms COMPAS w/o ASI on six designs (Section 4.4). Moreover, Appendix D.2 (Figure 6) reports ablations of the search-guidance components, including the reference-placement prior (RPP) and wirelength prior variants. These ablations provide additional evidence for the contribution of the search-guidance components.
3. Simpler GP-guided learning alternatives. Appendix D.3 (Figure 7 and Table 7) compares COMPAS with simpler GP-guided approaches on Superblue3 and Superblue4. All configurations use the same initial reference placement, and matched experimental settings within each design. Figure 7 shows the best-so-far GP-HPWL convergence curves and Table 7 reports the performance results normalized by EXPlace (lower is better).
| Method / ablation | Superblue3 | Superblue4 |
|---|---|---|
| COMPAS (full ASI) | 0.8939 | 0.9498 |
| GP-only SIL (fixed prior coefficients) | 0.9257 | 0.9573 |
| Top-20 GP-selected behavioral cloning | 0.8995 | 0.9599 |
| ASI with GP-only replay pool (no MP constraint) | 0.9080 | 0.9576 |
COMPAS achieves the lowest GP-HPWL on both designs. The GP-only replay-pool variant is an internal ASI ablation that tests the role of the MP-HPWL constraint, whereas GP-only SIL and behavioral cloning are simpler GP-guided learning alternatives. An additional PPO fine-tuning experiment on GP-selected trajectories yielded less favorable results than the approaches listed above, with GP-HPWL values of 6.45 × 10⁸ on Superblue3 and 4.01 × 10⁸ on Superblue4, corresponding to 1.4% below and 5.0% above EXPlace, respectively. In contrast, COMPAS reduces GP-HPWL relative to EXPlace by 9.9% on Superblue3 and 3.9% on Superblue4 (Table 1; Appendix D.3).
4. Interpretation and scope. Together, these tests indicate that GP feedback contributes through reference-guided search and policy supervision, and that the tested simpler GP-guided methods do not match full COMPAS on the tested designs. The internal ASI ablations, sensitivity, and computational overhead are addressed in W5 (4/4), with additional evidence in Section 4.6 and Appendix D.3–D.5.
Response to Reviewer aNVP (4/4) W5. Fine-grained ablations of ASI
W5. Fine-grained ablations of ASI
We thank the reviewer for requesting a finer-grained assessment of ASI's mechanisms and practical cost. The revised manuscript adds internal ablations and computational profiling in Appendix D.3 and a prior-coefficient analysis in Appendix D.5. These complement the adaptive-constraint comparison in Appendix D.4 and the sensitivity experiments in Section 4.6.
1. Imitation objective, trajectory selection, replay sampling, and frequency. Appendix D.3 (Figure 7 and Table 7) compares full ASI with changes to its imitation objective, prior-coefficient updates, trajectory selection, replay sampling, and update frequency. The comparisons performed under same experimental settings on Superblue3 and Superblue4. The table reports best-so-far GP-HPWL, normalized by EXPlace (lower is better).
| Method / ablation | Superblue3 | Superblue4 |
|---|---|---|
| COMPAS (full ASI) | 0.8939 | 0.9498 |
| Trajectory-weighted imitation | 0.9255 | 0.9607 |
| ASI with fixed prior coefficients (α, κ) | 0.9144 | 0.9666 |
| ASI with GP-only replay pool (no MP constraint) | 0.9080 | 0.9576 |
| ASI with uniform replay sampling | 0.9002 | 0.9652 |
| ASI with reduced update frequency | 0.8984 | 0.9701 |
Full ASI achieves the lowest GP-HPWL on both designs. Each ablation increases GP-HPWL, including the changes to replay sampling and update frequency, whose effects are relatively small on Superblue3 but among the largest on Superblue4.
2. Adaptive constraint and coefficient-update rule. Appendix D.4 compares adaptive and fixed constraint multiplier λ across eight designs. Adaptive λ achieves lower final mean GP-HPWL on seven of eight. ** Appendix D.3 and Appendix D.5** examines α and κ across eight designs. Their changes are generally modest and design-dependent, while adaptive coefficients yield about4.73% lower GP-HPWL than fixed coefficients on Superblue7. These results support the two adaptation mechanisms without implying uniform benefits on every design.
3. Replay-score and replay-admission sensitivity. Section 4.6 (Figure 5) varies macro-degradation tolerance δ over {0, 0.05, 0.1} and replay-admission margin ε_S over {0.005, 0.015, 0.03} on Superblue3 and Superblue18. The tolerance δ controls when the MP-HPWL penalty applies in the replay score (Eq. 4), while ε_S controls which evaluated trajectories enter the replay pool (Eq. 8). All tested settings outperform EXPlace on both designs, although the preferred setting differs by design.
4. Training stability and computational overhead. Both analyses are included in Appendix D.3. Under the matched experimental settings, the approximate KL divergence per PPO update remains well below the early-stopping threshold in all configurations, early stopping is rarely triggered, and both policy entropy and the PPO critic loss decline over training. Thus, none of the variants shows signs of unstable policy updates in these diagnostics. The within-run standard deviations of GP-HPWL across evaluated placements, also reported in Appendix D.3, describe variability among sampled layouts. We also profiled two matched 50-episode CPU runs per configuration on Superblue3 with cached GP feedback rather than DREAMPlace execution. Runs without ASI take 41.9–44.0 minutes. ASI adds 32 minibatches per PPO update cycle, which take 35.8–37.9 seconds per cycle, or 8.8–9.0% of non-ASI PPO update time. Peak memory rises from 14.59–16.26 GiB without ASI to 17.38–17.70 GiB with ASI.
Response to Reviewer aNVP (1/4) W1. Quantitative evidence for the MP–GP mismatch
We thank the reviewer for the thoughtful and constructive feedback. The revised manuscript quantifies the MP–GP mismatch, compares adaptive self-imitation (ASI) with simpler GP-guided alternatives, ablates its internal mechanisms, and extends the analysis of routing overflow and computational overhead. For clarity, we respond in four comments, with W1 in this comment, W3 in the second, W2 and W4 jointly in the third, and W5 in the fourth. Revisions are highlighted in red throughout the manuscript, including the appendix.
W1. Quantitative evidence for the MP–GP mismatch
We agree that the motivation for downstream feedback should rest on direct quantitative evidence rather than only on the conceptual difference between macro placement (MP) and global placement (GP). We have added this analysis in Section 4.2 and Appendix D.1 (Tables 4–6). It reports the rank correlation suggested by the reviewer, the downstream cost of selecting placements by MP-HPWL, and the behavior of both quantities across designs, training stages, and a GP-blind control.
1. Rank alignment and its downstream cost. We compare how MP-HPWL and GP-HPWL rank the same evaluated placements within each training run, across eight ICCAD 2015 designs. The median within-run Spearman correlation is −0.12, with negative design-level median correlations on six of eight designs. The median pairwise ranking-discordance rate is 54.3%, which is close to the 50% expected under independent rankings.
To quantify the practical consequence, we define GP regret as the relative GP-HPWL gap between the placement with the lowest MP-HPWL and the best GP-evaluated placement within the same run (Eq. 22). The median GP regret is 7.75% across 24 runs, and the design-level medians range from 3.46% on Superblue5 to 17.87% on Superblue10 (Table 4). This regret shows that the mismatch is not merely a difference in metric scales, since selecting by MP-HPWL can return a placement whose GP-HPWL is several percent higher than that of the best placement already evaluated in the same run.
2. Persistence across training stages. Appendix Table 5 repeats the analysis within the early (episodes 1–100), middle (101–200), and late (201–300) stages of COMPAS and COMPAS w/o ASI. The stage-level median correlation is negative in 34 of 48 design–approach–stage entries and never exceeds +0.42, with no consistent trend toward stronger positive alignment. GP regret decreases from early to late training in 11 of the 16 design–approach combinations, and the median across combinations falls from 5.51% to 4.84%. Nevertheless, the stage-level median regret remains at least 1.72% in every entry. The mismatch is thus not confined to early training and remains relevant during later optimization.
3. GP-blind control. Since the preceding experiments incorporate GP-selected reference memory, the observed mismatch could potentially be influenced by GP-guided search. Therefore, we analyze a control that uses neither GP-selected reference memory nor ASI (No reference memory w/o ASI, Table 6). GP-HPWL is recorded for evaluation but is not used to guide search or policy learning. On Superblue3, Superblue4, and Superblue18, the median correlations are −0.324, +0.259, and −0.053, and the median GP regrets are 3.32%, 3.08%, and 4.13%. Thus, the mismatch persists without GP-guided optimization. Since regret is defined relative to the best evaluated placement within each run, differences in regret across configurations should not be interpreted as direct comparisons of their placement quality.
When GP feedback is valuable and how much information the proxy misses. The analysis identifies a practical limitation of MP-only selection. The design-level median GP regret is at least 3.46% on all eight designs, the stage-level median is at least 1.72% in all 48 design–approach–stage entries, and the median regret in the GP-blind control is above 3% on each of its three designs. Therefore, direct GP evaluation provides useful downstream information beyond the initial search stage. The median GP regret of 7.75%, together with the variation across designs and training stages, quantifies the downstream selection cost associated with MP-HPWL rankings.