acceptodds
Under review as a conference paper at ICLR 2027

Learning from Good Placements: Memory and Adaptive Self-Imitation for Macro Placement

Abstract

Reinforcement learning for macro placement relies on intermediate placement information to guide hundreds of sequential decisions, whereas downstream quality can only be assessed after completing the full sequence and running a costly global-placement stage that includes standard-cell placement. Moreover, downstream placement quality is not necessarily well correlated with intermediate macro placement objectives, and these relationships can vary across benchmark designs, while the relative importance of these objectives is typically fixed throughout policy optimization. This limits both alignment with the downstream objective and the ability to adapt to the characteristics of individual placement problems. We address this with COMPAS, a Constrained Optimization framework for Macro Placement via Adaptive Self-Imitation, which incorporates sparse downstream placement feedback directly into policy learning. Downstream evaluation is incorporated into COMPAS through two pathways. Global Placement (GP) improving placements update a GP-selected reference memory that guides subsequent search, while selected evaluated trajectories directly supervise the policy through adaptive self-imitation (ASI). During standard policy optimization, PPO is trained using dense wirelength and reference-placement guidance rewards. Periodically, complete macro-placement trajectories are evaluated by the downstream global placer, and selected trajectories are retained for replay, allowing decisions associated with favorable downstream outcomes to directly shape subsequent policy updates. Experiments on eight ICCAD 2015 contest benchmarks show that COMPAS reduces mean GP-HPWL by approximately 8.7% relative to EXPlace, the strongest RL-based baseline in our evaluation, improving seven of eight benchmarks with reductions of up to 22%. Additionally, COMPAS discovers 44% more new GP-HPWL incumbents than the non-ASI variant, with the cumulative gap widening over the optimization horizon. This trend suggests that ASI continues to improve downstream search later in training by replaying trajectories associated with favorable global-placement outcomes. In downstream physical-quality evaluation, COMPAS further achieves 9.6% lower average rWL, 7.9% lower NVP, and a 49% reduction in TNS magnitude relative to EXPlace. Together, these results show that sparse downstream evaluation can both guide subsequent search and provide direct policy supervision through adaptive self-imitation during sequential macro-placement optimization.

Primary area:
other topics in machine learning (i.e., none of the above)
Keywords:
Macro placement, reinforcement learning, adaptive self imitation, downstream aware optimization, constrained optimization, reference guided policy learning

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_aNVPOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes COMPAS, a reinforcement learning framework for sequential macro placement. It addresses the problem that dense proxy signals available during placement, such as macro-placement HPWL, may not reliably reflect the final placement quality after full global placement (GP), while running GP evaluation after every placement decision is prohibitively expensive. In addition to standard PPO training, COMPAS periodically sends completed macro placements to a GP evaluator and uses the resulting feedback through two pathways: 1) Layouts with improved GP-HPWL are stored in a reference memory and subsequently used to guide the placement prior and wirelength optimization. 2) Selected evaluation trajectories are incorporated into Adaptive Self-Imitation (ASI) replay. The evaluator feedback is ultimately used to update the placement policy and adapt the coefficient controlling the placement prior.

Critical strengths

1.Practical engineering value. The paper develops a complete and effective reinforcement learning framework for macro placement, demonstrating meaningful potential for practical applications.

2.Detailed methodological and implementation descriptions. The paper provides substantial detail about its individual components and training procedure, including trajectory selection, replay objectives, and reference memory management. These descriptions are helpful for reproducibility.

3.Comprehensive reporting of physical metrics and computational costs. The experiments report detailed physical metrics as well as training overhead, which is valuable for evaluating an application-oriented placement method.

Critical weaknesses

1.The central motivation lacks quantitative validation. The paper is motivated by the claimed mismatch between intermediate macro-placement metrics and downstream placement quality after global placement. However, it does not provide sufficient quantitative evidence demonstrating the severity or prevalence of this mismatch. For example, the authors could report the rank correlation between macro-placement HPWL and post-GP metrics across training checkpoints, layouts, and benchmarks. Such an analysis would clarify when GP feedback is necessary and how much information is missed by the dense proxy reward.

2.The framework is highly engineered, but the necessity of its components is insufficiently established. COMPAS introduces reference memory, RPP, additional policy updates, adaptive constraints, and other interacting mechanisms. The current experiments do not sufficiently isolate the contribution of each component. As a result, it is difficult to identify the essential methodological contribution or determine whether the gains arise from the proposed feedback principle or simply from the accumulation of several engineering techniques.

3.GP feedback does not lead to consistent improvements across all physical-quality metrics. As shown in Table 2, COMPAS underperforms EXPlace on the rO-H and rO-V metrics. This suggests that COMPAS does not comprehensively improve physical placement quality and that GP-HPWL may still be an incomplete proxy for downstream physical objectives. The paper should analyze the trade-off between HPWL and routing-related metrics and explain why optimizing GP-derived feedback can degrade horizontal or vertical routing overflow.

4.The argument for introducing ASI is incomplete. The paper argues that GP evaluation is expensive and therefore introduces ASI to reuse sparse GP feedback. However, the high cost of GP evaluation alone does not establish that the relatively complex ASI mechanism is necessary. The authors should compare ASI against simpler methods for exploiting GP-improved layouts, such as periodically fine-tuning PPO on GP-selected trajectories, behavior cloning from the best layouts, reward relabeling using GP-HPWL, or maintaining a best-trajectory replay buffer. These comparisons are needed to demonstrate that ASI provides benefits beyond simpler alternatives with lower methodological complexity.

5.The ASI module lacks sufficiently fine-grained ablation studies. ASI contains multiple tightly coupled components, but their individual contributions are not clearly evaluated. The paper should separately ablate the trajectory-selection criterion, replay frequency, replay-buffer construction, imitation objective, adaptive constraint, and coefficient-update rule. It would also be useful to report how these choices affect sample efficiency, final GP quality, training stability, and computational overhead. Without such analyses, it remains unclear which parts of ASI are indispensable and whether the complete design is justified.

Questions for authors

See Weaknesses.

4 comments
AuthorsOpenReview ↗

Response to Reviewer aNVP (1/4) W1. Quantitative evidence for the MP–GP mismatch

We thank the reviewer for the thoughtful and constructive feedback. The revised manuscript quantifies the MP–GP mismatch, compares adaptive self-imitation (ASI) with simpler GP-guided alternatives, ablates its internal mechanisms, and extends the analysis of routing overflow and computational overhead. For clarity, we respond in four comments, with W1 in this comment, W3 in the second, W2 and W4 jointly in the third, and W5 in the fourth. Revisions are highlighted in red throughout the manuscript, including the appendix.

W1. Quantitative evidence for the MP–GP mismatch

We agree that the motivation for downstream feedback should rest on direct quantitative evidence rather than only on the conceptual difference between macro placement (MP) and global placement (GP). We have added this analysis in Section 4.2 and Appendix D.1 (Tables 4–6). It reports the rank correlation suggested by the reviewer, the downstream cost of selecting placements by MP-HPWL, and the behavior of both quantities across designs, training stages, and a GP-blind control.

1. Rank alignment and its downstream cost. We compare how MP-HPWL and GP-HPWL rank the same evaluated placements within each training run, across eight ICCAD 2015 designs. The median within-run Spearman correlation is −0.12, with negative design-level median correlations on six of eight designs. The median pairwise ranking-discordance rate is 54.3%, which is close to the 50% expected under independent rankings.

To quantify the practical consequence, we define GP regret as the relative GP-HPWL gap between the placement with the lowest MP-HPWL and the best GP-evaluated placement within the same run (Eq. 22). The median GP regret is 7.75% across 24 runs, and the design-level medians range from 3.46% on Superblue5 to 17.87% on Superblue10 (Table 4). This regret shows that the mismatch is not merely a difference in metric scales, since selecting by MP-HPWL can return a placement whose GP-HPWL is several percent higher than that of the best placement already evaluated in the same run.

2. Persistence across training stages. Appendix Table 5 repeats the analysis within the early (episodes 1–100), middle (101–200), and late (201–300) stages of COMPAS and COMPAS w/o ASI. The stage-level median correlation is negative in 34 of 48 design–approach–stage entries and never exceeds +0.42, with no consistent trend toward stronger positive alignment. GP regret decreases from early to late training in 11 of the 16 design–approach combinations, and the median across combinations falls from 5.51% to 4.84%. Nevertheless, the stage-level median regret remains at least 1.72% in every entry. The mismatch is thus not confined to early training and remains relevant during later optimization.

3. GP-blind control. Since the preceding experiments incorporate GP-selected reference memory, the observed mismatch could potentially be influenced by GP-guided search. Therefore, we analyze a control that uses neither GP-selected reference memory nor ASI (No reference memory w/o ASI, Table 6). GP-HPWL is recorded for evaluation but is not used to guide search or policy learning. On Superblue3, Superblue4, and Superblue18, the median correlations are −0.324, +0.259, and −0.053, and the median GP regrets are 3.32%, 3.08%, and 4.13%. Thus, the mismatch persists without GP-guided optimization. Since regret is defined relative to the best evaluated placement within each run, differences in regret across configurations should not be interpreted as direct comparisons of their placement quality.

When GP feedback is valuable and how much information the proxy misses. The analysis identifies a practical limitation of MP-only selection. The design-level median GP regret is at least 3.46% on all eight designs, the stage-level median is at least 1.72% in all 48 design–approach–stage entries, and the median regret in the GP-blind control is above 3% on each of its three designs. Therefore, direct GP evaluation provides useful downstream information beyond the initial search stage. The median GP regret of 7.75%, together with the variation across designs and training stages, quantifies the downstream selection cost associated with MP-HPWL rankings.

AuthorsOpenReview ↗

Response to Reviewer aNVP (2/4) W3. Routing overflow relative to EXPlace

W3. Routing overflow relative to EXPlace

We thank the reviewer for raising this point and have included a more detailed analysis of routing overflow in the revised manuscript (Section 4.3 and Appendix D.8).

1. Per-design behavior. The rO-H difference is not uniform across designs. The rO-H is higher than that of EXPlace on six designs, equal on one, and lower on one (Appendix Table 12). Superblue1 and Superblue16 account for approximately 80% of the net gap, where rO-H is 1.19 and 0.47 percentage points higher. The wirelength outcome differs between these two designs, since GP-HPWL is 3.9% lower than that of EXPlace on Superblue1 and 9.1% higher on Superblue16 (Table 1). Vertical overflow is a more consistent limitation. It is higher than that of EXPlace on seven of eight designs, although no per-design increase exceeds 0.15 percentage points.

2. Relationship between GP-HPWL improvements and routing overflow. GP-HPWL is measured after global placement and legalization, but it does not explicitly penalize routing congestion. Since both feedback pathways are driven by HPWL-based scores, neither GP-selected reference-memory updates nor ASI replay selection directly evaluates or penalizes routing congestion. To examine the relationship empirically, Appendix Figure 13 compares GP-HPWL reductions with changes in routed metrics at 32 GP-incumbent updates from one COMPAS run per design. Routed wirelength decreases at 30 of 32 updates and broadly tracks the GP-HPWL reduction. In contrast, rO-H decreases at 18 updates and increases at 14, without a consistent relationship to the GP-HPWL reduction. Together with the per-design results, this analysis indicates that lower GP-HPWL neither guarantees lower overflow nor systematically increases it. The results suggest that routing overflow is left unconstrained by the objective, and they do not show a consistent wirelength–congestion trade-off across designs or incumbent updates.

3. Overall physical quality and limitation. The higher routing overflow should be considered alongside the other measured outcomes. Relative to EXPlace, COMPAS reduces mean routed wirelength from 121.19 m to 109.55 m (9.6%), NVP from 11.83 to 10.90 (×10³; 7.9%), and TNS magnitude from 8.59 to 4.36 (×10⁴ ns; 49%), while average WNS improves from −55.76 ns to −52.56 ns (Table 2). Thus, COMPAS improves several downstream quality measures, although routing overflow remains higher on average. Since the ASI replay score uses downstream quality as a scalar (Eqs. 3–4), congestion-aware feedback is a natural direction for future work, which we discuss in Section 4.3.

AuthorsOpenReview ↗

Response to Reviewer aNVP (3/4) W2 and W4. Component contributions and simpler GP-guided alternatives

W2 and W4. Component contributions and simpler GP-guided alternatives

We thank the reviewer for these constructive suggestions and agree that the contributions of individual components should be distinguished from the benefits of combining several mechanisms. Section 4.5 now organizes the component analysis around COMPAS's two GP-feedback pathways, and Appendix D.3 adds comparisons with simpler approaches that reuse GP-evaluated placements.

1. Identifying the central methodological contribution. The central contribution of COMPAS is to incorporate downstream GP evaluations into ongoing macro-placement optimization through two complementary mechanisms (Sections 3.2–3.3). GP-selected reference memory uses high-quality evaluated placements to guide subsequent placement decisions, while adaptive self-imitation (ASI) uses selected evaluated trajectories to update the policy directly. Thus, downstream feedback influences both subsequent search and policy learning. The ablations below examine whether these mechanisms contribute beyond the shared policy architecture and placement guidance.

2. Isolating the contribution of GP feedback. Figure 4 compares COMPAS against variants without GP-selected reference memory, without ASI, and without either mechanism on Superblue3, Superblue4, and Superblue18. Variants without reference memory retain the fixed initial reference for spatial guidance. Full COMPAS achieves the lowest final GP-HPWL on all three designs. The variant without either mechanism retains the policy architecture, both placement priors, and the dense PPO reward, but uses no GP feedback to guide search or policy learning. Thus, its gap to full COMPAS supports a contribution from GP-derived feedback beyond these shared design choices. Across all eight designs, full COMPAS also outperforms COMPAS w/o ASI on six designs (Section 4.4). Moreover, Appendix D.2 (Figure 6) reports ablations of the search-guidance components, including the reference-placement prior (RPP) and wirelength prior variants. These ablations provide additional evidence for the contribution of the search-guidance components.

3. Simpler GP-guided learning alternatives. Appendix D.3 (Figure 7 and Table 7) compares COMPAS with simpler GP-guided approaches on Superblue3 and Superblue4. All configurations use the same initial reference placement, and matched experimental settings within each design. Figure 7 shows the best-so-far GP-HPWL convergence curves and Table 7 reports the performance results normalized by EXPlace (lower is better).

Method / ablationSuperblue3Superblue4
COMPAS (full ASI)0.89390.9498
GP-only SIL (fixed prior coefficients)0.92570.9573
Top-20 GP-selected behavioral cloning0.89950.9599
ASI with GP-only replay pool (no MP constraint)0.90800.9576

COMPAS achieves the lowest GP-HPWL on both designs. The GP-only replay-pool variant is an internal ASI ablation that tests the role of the MP-HPWL constraint, whereas GP-only SIL and behavioral cloning are simpler GP-guided learning alternatives. An additional PPO fine-tuning experiment on GP-selected trajectories yielded less favorable results than the approaches listed above, with GP-HPWL values of 6.45 × 10⁸ on Superblue3 and 4.01 × 10⁸ on Superblue4, corresponding to 1.4% below and 5.0% above EXPlace, respectively. In contrast, COMPAS reduces GP-HPWL relative to EXPlace by 9.9% on Superblue3 and 3.9% on Superblue4 (Table 1; Appendix D.3).

4. Interpretation and scope. Together, these tests indicate that GP feedback contributes through reference-guided search and policy supervision, and that the tested simpler GP-guided methods do not match full COMPAS on the tested designs. The internal ASI ablations, sensitivity, and computational overhead are addressed in W5 (4/4), with additional evidence in Section 4.6 and Appendix D.3–D.5.

AuthorsOpenReview ↗

Response to Reviewer aNVP (4/4) W5. Fine-grained ablations of ASI

W5. Fine-grained ablations of ASI

We thank the reviewer for requesting a finer-grained assessment of ASI's mechanisms and practical cost. The revised manuscript adds internal ablations and computational profiling in Appendix D.3 and a prior-coefficient analysis in Appendix D.5. These complement the adaptive-constraint comparison in Appendix D.4 and the sensitivity experiments in Section 4.6.

1. Imitation objective, trajectory selection, replay sampling, and frequency. Appendix D.3 (Figure 7 and Table 7) compares full ASI with changes to its imitation objective, prior-coefficient updates, trajectory selection, replay sampling, and update frequency. The comparisons performed under same experimental settings on Superblue3 and Superblue4. The table reports best-so-far GP-HPWL, normalized by EXPlace (lower is better).

Method / ablationSuperblue3Superblue4
COMPAS (full ASI)0.89390.9498
Trajectory-weighted imitation0.92550.9607
ASI with fixed prior coefficients (α, κ)0.91440.9666
ASI with GP-only replay pool (no MP constraint)0.90800.9576
ASI with uniform replay sampling0.90020.9652
ASI with reduced update frequency0.89840.9701

Full ASI achieves the lowest GP-HPWL on both designs. Each ablation increases GP-HPWL, including the changes to replay sampling and update frequency, whose effects are relatively small on Superblue3 but among the largest on Superblue4.

2. Adaptive constraint and coefficient-update rule. Appendix D.4 compares adaptive and fixed constraint multiplier λ across eight designs. Adaptive λ achieves lower final mean GP-HPWL on seven of eight. ** Appendix D.3 and Appendix D.5** examines α and κ across eight designs. Their changes are generally modest and design-dependent, while adaptive coefficients yield about4.73% lower GP-HPWL than fixed coefficients on Superblue7. These results support the two adaptation mechanisms without implying uniform benefits on every design.

3. Replay-score and replay-admission sensitivity. Section 4.6 (Figure 5) varies macro-degradation tolerance δ over {0, 0.05, 0.1} and replay-admission margin ε_S over {0.005, 0.015, 0.03} on Superblue3 and Superblue18. The tolerance δ controls when the MP-HPWL penalty applies in the replay score (Eq. 4), while ε_S controls which evaluated trajectories enter the replay pool (Eq. 8). All tested settings outperform EXPlace on both designs, although the preferred setting differs by design.

4. Training stability and computational overhead. Both analyses are included in Appendix D.3. Under the matched experimental settings, the approximate KL divergence per PPO update remains well below the early-stopping threshold in all configurations, early stopping is rarely triggered, and both policy entropy and the PPO critic loss decline over training. Thus, none of the variants shows signs of unstable policy updates in these diagnostics. The within-run standard deviations of GP-HPWL across evaluated placements, also reported in Appendix D.3, describe variability among sampled layouts. We also profiled two matched 50-episode CPU runs per configuration on Superblue3 with cached GP feedback rather than DREAMPlace execution. Runs without ASI take 41.9–44.0 minutes. ASI adds 32 minibatches per PPO update cycle, which take 35.8–37.9 seconds per cycle, or 8.8–9.0% of non-ASI PPO update time. Peak memory rises from 14.59–16.26 GiB without ASI to 17.38–17.70 GiB with ASI.

Reviewer_Tb52OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes COMPAS, a reinforcement learning framework for macro placement that aims to address the discrepancy between intermediate placement optimization metrics and final global placement quality.

The method consists of two main components. First, GP-selected Reference Memory leverages historical placements with favorable downstream quality to guide subsequent placement search. Second, Adaptive Self-Imitation (ASI) uses GP-HPWL feedback obtained from periodic downstream evaluations to select and imitate high-quality placement trajectories. ASI further introduces an adaptive MP-HPWL constraint to limit excessive degradation in intermediate placement quality and dynamically adjusts the weights of placement priors based on downstream feedback.

Critical strengths
  1. Existing reinforcement learning-based macro placement methods typically rely on computationally inexpensive intermediate placement metrics for training. However, improvements in these metrics do not necessarily translate into better final global placement quality. This paper incorporates GP-HPWL feedback into placement search and policy optimization through periodic downstream evaluations, thereby improving the alignment between the training process and the final optimization objective. In particular, the combination of GP-selected Reference Memory and Adaptive Self-Imitation allows downstream evaluation results to guide subsequent search and directly update the policy, providing a reasonable approach to addressing the mismatch between optimization objectives across placement stages.
  2. The paper conducts experiments on eight ICCAD 2015 benchmarks and compares COMPAS with representative methods, including MaskPlace, MaskRegulate, and EXPlace. COMPAS achieves lower GP-HPWL on seven of the eight benchmarks, reducing the average GP-HPWL by approximately 8.7% compared with EXPlace.
Critical weaknesses
  1. ASI combines several existing techniques, including self-imitation learning, prioritized replay, and adaptive constraints. Although the paper provides ablations removing ASI and fixing the constraint multiplier, it lacks comparisons with simpler alternatives. For example, would standard self-imitation using GP-HPWL alone achieve similar improvements? How much does adapting the placement-prior coefficients contribute? Additional ablations are needed to establish the necessity of these designs.
  2. In Section 3.3.2, all decisions within a trajectory receive the same supervision target derived from the final placement quality. However, a good trajectory may still contain suboptimal individual decisions. Although the evaluator critic provides state-dependent advantages, it is unclear whether this mechanism can effectively distinguish beneficial actions from unfavorable ones. Further analysis or comparison with simple trajectory-weighted imitation learning would strengthen the justification for this design.
Questions for authors

Same as Critical Weaknesses

1 comment
AuthorsOpenReview ↗

Response to Reviewer Tb52

We thank the reviewer for their valuable and constructive feedback, particularly the questions of whether simpler GP-guided imitation with fixed placement priors can match ASI and whether a shared trajectory-level target can support selective supervision of individual placement decisions. We have added matched comparisons and further analyses in Sec. 4.5 and Appendices D.3–D.5. All revisions are highlighted in red throughout the revised manuscript, including the appendix.

1. Simpler GP-guided alternatives and fixed priors do not match full COMPAS on the tested designs.

We compare methods on Superblue3 and Superblue4 under matched experimental settings, with the same initial reference placement and GP evaluation schedule within each design. The table reports best-so-far GP-HPWL normalized by EXPlace (lower is better; Appendix D.3, Table 7 and Fig. 7 convergence plots).

MethodSB3 ↓SB4 ↓
Full COMPAS0.89390.9498
GP-only SIL, fixed priors0.92570.9573
ASI, fixed α,κ\alpha,\kappa0.91440.9666
Trajectory-weighted imitation0.92550.9607

GP-only SIL with fixed priors, the closest analogue to standard self-imitation using GP-HPWL alone, yields higher GP-HPWL on both designs; it tests a simpler combined alternative rather than isolating the replay objective.

Since circuit designs differ in connectivity and spatial constraints, the appropriate balance between wirelength and reference-placement priors may vary across designs and training stages. The prior mixture α\alpha and strength κ\kappa (Eq. 2) remain fixed during PPO but are adapted through evaluator-guided ASI using downstream-evaluated experience (Sec. 3.3.3). The fixed-α,κ\alpha,\kappa ablation retains the other ASI mechanisms and yields higher GP-HPWL on both designs. Appendix D.5 (Figs. 10–11) shows modest, design-dependent coefficient changes across eight designs. On Superblue7, where these changes are largest, full COMPAS achieves 4.73% lower final GP-HPWL than its fixed-prior variant and continues improving after the latter plateaus.

The adaptive MP-HPWL constraint serves a complementary purpose. GP-HPWL remains the primary replay criterion, while the constraint penalizes excessive macro-stage wirelength degradation beyond a tolerance (Eq. 4). This provides a soft safeguard for macro-stage quality without requiring both metrics to improve simultaneously. The multiplier λ\lambda responds to constraint residuals from GP-competitive trajectories (Eqs. 5–7). Adaptive λ\lambda achieves lower final mean GP-HPWL than fixed λ\lambda on seven of eight designs (Appendix D.4), and removing the constraint yields higher GP-HPWL on both tested designs (Table 7). Through replay scores, λ\lambda also affects replay admission and evaluator targets (Eqs. 8–12).

Further Appendix D.3 comparisons show higher GP-HPWL for top-20 GP-selected behavior cloning, uniform replay sampling, and reduced ASI update frequency on both designs. GP-selected PPO fine-tuning likewise produces 12.2% and 9.3% higher GP-HPWL than COMPAS on Superblue3 and Superblue4, respectively.

2. State-dependent evaluator advantages distinguish ASI from trajectory-weighted imitation.

In sequential macro placement, each action changes the spatial configuration and feasible positions for subsequent macros. Thus, a shared trajectory-level target need not provide equally informative supervision at every partial-placement state.

The evaluator advantage has a return-minus-value structure analogous to Monte Carlo advantage estimation. Its target is derived from the GP-primary constrained score of the complete placement and shared across its decisions, analogous to undiscounted terminal feedback. A separate evaluator critic predicts this target from each partial-placement state. ASI computes the capped nonnegative advantage max⁡(0,yi−Vψ(si,t))\max(0, y_i - V_\psi(s_{i,t})), clipped at amax⁡a_{\max} (Eq. 10), which can differ across decisions in the same trajectory. Actions whose targets do not exceed the state-dependent prediction receive no ASI actor update (Eq. 11).

Unlike trajectory-weighted imitation, which assigns supervision based on trajectory quality alone, ASI uses the state-dependent comparison to emphasize stored actions whose trajectory outcomes exceed the critic's expectations for their partial-placement states. The critic neither ranks candidate grid locations nor identifies individual actions' causal contributions; it provides a state-dependent baseline for selective imitation. The matched comparison supports this distinction, since full COMPAS achieves lower GP-HPWL than trajectory-weighted imitation on both designs (Table 7; Appendix Fig. 7).

Reviewer_r3HWOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
2
Read review
Paper summary

This paper proposes COMPAS, which utilizes sparse global layout feedback to update reference memory and guide adaptive constraint self imitation learning.

Critical strengths

1.The problem sloved is of vlaue;

Critical weaknesses
  1. The tested benchmarks is a little out of time
Questions for authors

no questions

2 comments
AuthorsOpenReview ↗

Response to Reviewer r3HW

Thank you for recognizing the value of the problem and for drawing attention to our choice of benchmark designs.

We agree that the ICCAD 2015 contest suite is not recent. We use this suite primarily to enable direct comparison with recent leading RL macro-placement methods, including MaskRegulate (Xue et al., 2024) and EXPlace (Gao et al., 2026), the strongest recent RL baseline in our evaluation. Both methods report results on ICCAD 2015. We use their published results and follow the same DREAMPlace global-placement evaluation flow (Sec. 4.1, Sec. 4.3). This provides a consistent basis for assessing COMPAS against recent competitive RL methods.

The eight benchmark designs also provide distinct placement challenges on which to assess the framework under one consistent configuration. COMPAS uses the same framework and hyperparameters across all designs, without design-specific macro clustering, manually constructed macro groups, or per-design architectural changes (Sec. 4.1, App. A.2). This allows the evaluation to focus on the method's consistency across different designs rather than improvements that depend on case-specific engineering. Under this setup, COMPAS achieves approximately 8.7% lower mean GP-HPWL than EXPlace and lower GP-HPWL on seven of the eight designs (Table 1). These results support the effectiveness of COMPAS across the evaluated designs without reliance on design-specific configurations or manual tailoring.

Additionally, the revision strengthened the empirical support for the method through an analysis of the macro-global placement mismatch (Sec. 4.2) and targeted ablations that compare ASI with simpler GP-guided alternatives and isolate its internal mechanisms (Sec. 4.5, App. D.3), all marked in red.

Thanks for your additional experiments, the rating will be kept.

Comments on the paper

2 comments
AuthorsOpenReview ↗

General Response to Reviewers

We sincerely thank all reviewers for their time, constructive feedback, and valuable suggestions. We appreciate their recognition of the importance of downstream-aware macro placement and the opportunity to strengthen our work. We have substantially revised the manuscript and conducted additional experiments and analyses to address the reviewers' concerns. All revisions are highlighted in red throughout the manuscript, including the appendix. The main improvements are summarized below.

1. Quantitative validation of the MP–GP mismatch (Sec. 4.2, App. D.1). We added a systematic analysis across designs, random seeds, and training stages, together with a GP-blind control. The results reveal weak alignment between MP-HPWL and GP-HPWL rankings (median Spearman correlation of −0.12). Moreover, the GP-HPWL of the placement with the lowest MP-HPWL exceeds that of the best GP-evaluated placement in the same run by a median of 7.75% (GP regret). Both results provide direct empirical support for our motivation.

2. Additional comparisons with simpler GP-guided alternatives (Sec. 4.5, Apps. D.2–D.3). In addition to the existing component ablations of GP-selected reference memory, ASI, and placement priors, we added matched comparisons with GP-only self-imitation and behavior cloning, and a PPO fine-tuning experiment on GP-selected trajectories. None of these simpler approaches matches full COMPAS on the two tested designs, which suggests that the gains of ASI do not come from access to GP feedback alone.

3. Fine-grained ASI ablations and computational overhead (Sec. 4.5, Apps. D.3–D.5). We added targeted experiments that examine evaluator-guided versus trajectory-weighted imitation, adaptive versus fixed prior coefficients, GP-only versus constrained replay, replay sampling strategies, and ASI update frequency. Under the matched experimental settings, each tested ablation yields higher GP-HPWL than full COMPAS on both designs. We also added training-stability diagnostics and measurements of the runtime and memory overhead of ASI. These experiments complement the existing adaptive-constraint ablation and hyperparameter-sensitivity analysis, and together they provide a more detailed assessment of ASI's individual mechanisms and practical costs.

4. Further analysis of downstream physical quality (Sec. 4.3, App. D.8). We expanded the routing-overflow analysis with per-design comparisons and GP-incumbent tracking. The analysis shows no consistent relationship between GP-HPWL reductions and changes in routing overflow, since rO-H decreases at 18 of the 32 incumbent updates and increases at 14. Overflow is not constrained by the current HPWL-based objective and is higher than that of EXPlace on average, whereas routed wirelength and timing metrics improve. We discuss congestion-aware feedback as a natural direction for future work.

5. Clarification of benchmark selection and experimental consistency (Sec. 4.1). We clarify the rationale for the ICCAD 2015 benchmarks, which enable direct comparison with recent competitive RL-based methods. The eight designs pose distinct placement challenges, and COMPAS applies the same framework and hyperparameters to all of them. This allows the evaluation to focus on the approach itself rather than on gains that depend on design-specific engineering.

Overall, these additions provide stronger empirical motivation for downstream feedback, distinguish the contributions of COMPAS more clearly from simpler alternatives, and offer a more comprehensive assessment of its effectiveness, computational cost, and limitations. We are grateful to the reviewers for helping us improve the rigor and clarity of the manuscript. We hope that the revised paper and our detailed responses address their concerns, and we remain available to clarify any remaining questions.

Dear Reviewers,

The author-reviewer discussion period will end soon. Please review and acknowledge the authors’ responses.

Your AC

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.