acceptodds
Under review as a conference paper at ICLR 2027

Climbing the Design Ladder: Sequential Knowledge Distillation for Early-Stage Circuit Timing Prediction

Abstract

Integrated circuit design involves multiple design stages: logic synthesis, floorplanning, placement, and routing, with each stage taking hours to weeks to complete. Discovering timing violations late in this flow forces costly iterations back to earlier stages, wasting computational resources and delaying product launches. While predicting post-routing timing from early-stage data could prevent these failures, existing machine learning approaches struggle with the massive abstraction gap between post-synthesis logical descriptions and post-routing physical layouts. We propose STEP-KD (Sequential Timing Evaluation via Progressive Knowledge Distillation), which leverages intermediate design stages as “stepping stones” for progressive knowledge transfer rather than attempting direct prediction. STEP-KD trains teacher models at the post-routing, post-placement, and post-floorplan stages, then sequentially distills their knowledge to a post-synthesis student model through representation alignment. Experiments on diverse circuits demonstrate that STEP-KD reduces timing prediction error compared to direct distillation and supervised baselines, and in most settings compared to the industry-standard Static Timing Analysis (STA) tool. STEP-KD reduces the weighted mean absolute percentage error of Total Negative Slack prediction to 19.78%, compared with 74.84% for STA. Our proposed method is step forward to identify timing problems earlier, avoiding expensive late-stage redesigns.

Primary area:
infrastructure, software libraries, hardware, systems, etc.
Keywords:
Knowledge Distillation, Progressive Learning, Graph Neural Networks, Multi-Stage Learning, Integrated Circuit Design

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_GuVBOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes STEP-KD for predicting post-routing endpoint slack, and hence circuit-level TNS and WNS, using post-synthesis inputs. Models are trained in reverse design order, from routing through placement and floorplanning to synthesis. Each earlier-stage model aligns node-, subgraph-, and graph-level representations with frozen later-stage models through cascaded projections. Only the synthesis-stage model and its projection cascade are needed at inference. The authors construct approximately 12.2k implementations of 922 RTL designs using OpenROAD and ASAP7, split by source design, and additionally evaluate on OpenLLM-RTL. Experiments compare against synthesis-stage STA, supervised learning without distillation, direct distillation from different stages, and several ablations. The main test-set results favor STEP-KD; the external benchmark shows stronger improvements for aggregate TNS errors than for WNS.

Critical strengths
  1. The design-stage hierarchy is a well-motivated structure for privileged-information transfer, and the ablations (independently trained multi-teacher, projection depth) genuinely test that structure rather than just adding capacity.

  2. The aligned multi-stage dataset with design-disjoint splits, plus the external OpenLLM-RTL evaluation and multiple error aggregations, makes the evaluation considerably more convincing than a single aggregate score.

Critical weaknesses
  1. The reported error bars come from resampling test implementations rather than independent training runs, and the external benchmark excludes five circuits precisely where the baseline is exact. So the robustness and external validity of relatively small gains are not fully established (STEP-KD is already worse than STA on external WNS, 36.55% vs 33.71%).

  2. It remains unclear how much of the benefit requires the full graph/distillation pipeline: the Appendix C slack-only ablation nearly matches Direct-KD, there is no comparison to a lightweight endpoint-level correction of early STA, and no failure-screening (false-pass) analysis to support the intended use case.

Questions for authors
  1. Were the models retrained across random seeds, or do all standard deviations reflect only the 20 test-mplementation draws? Training-seed variability for STEP-KD vs direct PR distillation would be particularly informative.

  2. What do the results look like with the five excluded circuits from Appendix B.2 restored, and is the count of 38 designs / 189 implementations before or after this exclusion?

  3. How does STEP-KD compare to a simple supervised endpoint-slack correction model using only synthesis-time information (early slack, cell features, planned density/utilization)? The Appendix C ablation still uses graph representations, so it doesn't answer this.

  4. At a stated timing-violation threshold, how often does STEP-KD predict a failing circuit as passing, compared with STA and Direct-KD? Reporting inference time including feature extraction and early STA would also help judge practical utility.

1 comment

We thank the reviewer for their feedback.

Q1. All standard deviations in Table 1 reflect only the 20 test-implementation draws. We agree that independent training runs would characterize optimization variance more fully. In the revised paper, we have added results on three additional random train/validation/test splits, with all models (including all STEP-KD teachers) retrained on each split (new Appendix H). STEP-KD has the lowest TNS MedAE and TNS WMAPE on all three splits, and the lowest error on 11 of the 18 split–metric pairs. It is also more accurate than direct distillation from the PR teacher on every metric of every split.

Q2. The five excluded implementations are cases where the post-synthesis and post-routing STA values are identical, so the prediction task is trivial. These circuits are "precisely where the baseline is exact", but STEP-KD is also nearly exact on them, because it takes the current-stage STA slack as an input feature (Section 4.2, Appendix C).

We removed these few circuits because they also have extremely large TNS and WNS, so they dominate WMAPE and RSE and inflate the scores of both STA and STEP-KD. The exclusion follows a single criterion, applied uniformly to all methods, that removes trivial, extreme-valued circuits; it is not a hand-picked selection. The table below reports results with and without the excluded circuits. Because both methods are nearly exact on the restored circuits, restoring them lowers WMAPE and RSE for both methods and leaves MedAE essentially unchanged. The counts of 38 designs and 189 implementations are after the exclusion.

OpenLLM-RTLMethodMedAE (retained)MedAE (full)WMAPE (retained)WMAPE (full)RSE (retained)RSE (full)
TNSPost-synthesis STA50.7650.7141.8421.3543.428.54
STEP-KD62.4362.3827.4314.206.491.82
WNSPost-synthesis STA13.2413.1833.7113.1416.880.67
STEP-KD15.9816.1036.5515.4016.520.67

Q3. We trained a simple endpoint-level MLP that corrects the post-synthesis timing estimate using only synthesis-time information (the early slack, cell features, and the placement density, core utilization, and clock period), and we have added this new baseline to the paper (Table 1, Section 5.3). It is less accurate than STEP-KD on every metric of both datasets except OpenLLM-RTL WNS MedAE, and on the held-out set it is less accurate than No KD on every metric. This highlights the importance of the graph structure.

Q4. Inference consists of a forward pass through a GCN, which is very fast. Graph parsing is also fast, taking less than 10 seconds on the largest graphs. Compared with the PS → PR flow, which involves search and optimization problems, STEP-KD gives an instant (but approximate) estimate of the timing.

We produce a false-pass table with threshold set to zero. Values below are percentages, mean ± std over the 20 evaluation runs ( lower is better).

MethodHeld-out test set (original split)OpenLLM-RTL benchmark
STA0.0 ± 0.00.0 ± 0.0
No KD0.9 ± 0.56.7 ± 1.8
Direct KD (PR teacher)2.5 ± 0.819.6 ± 2.3
STEP-KD1.4 ± 0.614.6 ± 3.7

The false-pass rate is the fraction of circuits that actually fail timing but are predicted to pass. A circuit fails when its post-routing WNS is negative, meaning that at least one endpoint has negative slack, and it is predicted to pass when its predicted WNS is non-negative (here, exactly 0). On the original held-out set (about 160 failing designs per evaluation run), STEP-KD falsely passes 1.4% of failing circuits. On OpenLLM-RTL, where all 38 designs fail, STEP-KD's rate rises to 14.6%, higher than that of No KD (6.7%) but lower than those of Direct KD (19.6%).

One caveat is that this metric rewards pessimism. STA flags about 99% of circuits as failing and therefore has a 0% false-pass rate, and the models with the lowest rates also predict nearly every circuit as failing (97–98%).

Reviewer_yYQrOpenReview ↗
Rating
2
Confidence
4
Informativeness
1
Presentation
3
Soundness
2
Read review
Paper summary

STEP-KD predicts post-routing endpoint slack from post-synthesis netlists and aggregates it into TNS and WNS. Routing, placement, floorplan, and synthesis models are trained in reverse design order. Each earlier model aligns multi-granularity representations with all frozen later-stage teachers through cascaded projections; the synthesis model alone is used at inference. The paper constructs approximately 12,000 implementations of 922 RTL designs and separates them by source design. It reports improvements on 178 held-out designs over supervised learning, direct distillation, and independently trained multiple teachers, with additional evaluation on a filtered OpenLLM-RTL subset.

Critical strengths
  1. A meaningful information hierarchy is supported by appropriate controls. RTLDistil, ICML 2025 already supplies multi-granularity cross-stage distillation, and teacher-assistant distillation, AAAI 2020 already allows multiple intermediate models. Both are acknowledged. STEP-KD's useful increment is organizing those intermediates by available physical information. Tables 1–2 compare the same synthesis student with independently trained versus sequentially distilled teachers; Figure 4 and Tables 7/10 control projection depth. These comparisons substantiate a benefit beyond simply adding teachers or layers.

  2. The principal evaluation demonstrates a worthwhile predictive contribution. The source-design split keeps parameter variants together, and all learned baselines receive the same early-stage features. On the held-out set, STEP-KD lowers TNS WMAPE from 23.65% for the best single-teacher variant to 19.78%, and WNS WMAPE from 28.64% to 21.39%. Against independently trained teachers, the corresponding values improve from 30.18% and 26.91%; all six main metrics improve. Multiple error measures and the outlier discussion make the evidence informative. The PR teacher's access to PR slack is explicitly disclosed and confined to privileged training, not the deployed synthesis model.

Critical weaknesses
  1. The external comparison is conditional on a consequential selection rule. Appendix B.2 explicitly excludes five valid circuits on which post-synthesis STA is exact because their large targets would dominate aggregate metrics. The rationale explains the choice but does not establish performance on the complete valid benchmark. Full-set results alongside equal-design or stratified reporting are needed; the direction and size of the effect cannot be quantified from the manuscript. Even on the retained subset, STA has better WNS MedAE and WMAPE than STEP-KD (13.24 versus 15.98; 33.71% versus 36.55%), while STEP-KD has slightly better RSE (16.52% versus 16.88%). Thus the external result supports strong TNS aggregate-error reduction, not uniform superiority to STA. This concern does not invalidate the separately constructed primary test set.

  2. The evidence supports a bounded empirical contribution rather than universal or fully characterized robustness. The 20 evaluation runs resample implementations; they do not measure independent training-seed variation. Some advantages are modest: the 10-layer direct projection reaches 20.96% held-out TNS WMAPE versus 19.78%. Intermediate teachers also do not improve monotonically: Table 2's PP WNS MedAE worsens from 12.11 to 14.46 after distillation, with mixed behavior acknowledged by the authors. Appendix C shows that explicit early STA slack is particularly informative, but its “slack embedding” variant retains GNN subgraph/global representations. The precise additional value of graph processing over simple timing calibration remains open. Loss weights, slack units, and flow details also need specification for reproduction.

  3. This work makes only a marginal contribution beyond existing distillation algorithms in the EDA field; its level of innovation is very weak.

Questions for authors
  1. Please report unfiltered OpenLLM-RTL results with the frozen models and retain the five excluded circuits in a complementary robust aggregation. Were exclusions at design or implementation level, and are the stated 38 designs/189 implementations counted before or after exclusion?

  2. How many independently trained seeds underlie each row, and are implementation samples paired across methods? Please separate training variation from test-resampling variation and provide existing seed results if available.

  3. Please give the KD/granularity weights, their validation-selection procedure, slack units/sign convention, and exact OpenROAD/PDK configuration.

  4. Is a compact STA-plus-design-parameter calibration result available? It would sharpen deployment interpretation; the existing same-input KD controls already support the narrower sequential-distillation contribution.

9 comments

We thank the reviewer for their detailed feedback.

W1 and Q1.

The five excluded implementations are cases where the post-synthesis and post-routing STA values are identical, so the prediction task is trivial. These circuits are "precisely where the baseline is exact", but STEP-KD is also nearly exact on them, because it takes the current-stage STA slack as an input feature (Section 4.2, Appendix C).

We removed these few circuits because they also have extremely large TNS and WNS, so they dominate WMAPE and RSE and inflate the scores of both STA and STEP-KD. The exclusion follows a single criterion, applied uniformly to all methods, that removes trivial, extreme-valued circuits; it is not a hand-picked selection. The table below reports results with and without the excluded circuits. Because both methods are nearly exact on the restored circuits, restoring them lowers WMAPE and RSE for both methods and leaves MedAE essentially unchanged. The counts of 38 designs and 189 implementations are after the exclusion.

OpenLLM-RTLMethodMedAE (retained)MedAE (full)WMAPE (retained)WMAPE (full)RSE (retained)RSE (full)
TNSPost-synthesis STA50.7650.7141.8421.3543.428.54
STEP-KD62.4362.3827.4314.206.491.82
WNSPost-synthesis STA13.2413.1833.7113.1416.880.67
STEP-KD15.9816.1036.5515.4016.520.67

W2 and Q2.

Each row in Table 1 corresponds to one trained model; the reported mean and standard deviation measure variation across the 20 implementation-resampling runs, not training variation. In each run, one implementation is sampled per source design, and the same sampled implementations are used for every method, so the comparisons are paired.

"The 20 evaluation runs resample implementations; they do not measure independent training-seed variation"

We agree that independent training runs would characterize optimization variance more fully. In the revised paper, we have added results on three additional random train/validation/test splits, with all models (including all STEP-KD teachers) retrained on each split (new Appendix H). STEP-KD has the lowest TNS MedAE and TNS WMAPE on all three splits, and the lowest error on 11 of the 18 split–metric pairs. It is also more accurate than direct distillation from the PR teacher on every metric of every split.

We note that STEP-KD requires sequentially training and freezing four stage-specific models, so repeating the complete study many times is computationally substantial (especially during the rebuttal period), which is why we report three additional random splits.

Q3.

We have added all loss weights, as well as additional implementation details, to the revised manuscript (Section 5.2 and Appendices B.3 and C). We did not tune the hyperparameters through a hyperparameter search. We chose them so that the node-, subgraph- and graph-level alignment terms have comparable magnitudes, and so that the timing and distillation terms are of the same order. All units are now included in Appendix C.

Sign convention: slack = required arrival time − actual arrival time, so a negative slack is a timing violation, and TNS and WNS are computed from the negative slacks (Equation 1).

Flow: OpenROAD with the ASAP7 PDK (typical corner); placement density ∈ {0.70, 0.75, 0.80} × core utilization ∈ {40, 45, 50, 55, 60}%, and OpenROAD's default clock period for each design.

Q4.

Yes. We have added a Residual MLP baseline that corrects the post-synthesis STA slack of each endpoint from its cell features and the design parameters (placement density, core utilization, clock period), without the netlist graph or distillation (Table 1, Section 5.3). It is less accurate than STEP-KD on every metric of both datasets except OpenLLM-RTL WNS MedAE (14.79 vs. 15.98).

Note about novelty:

STEP-KD differs fundamentally from Teacher Assistant Knowledge Distillation (TAKD). TAKD addresses a capacity gap: an intermediate-capacity network helps transfer knowledge from a large teacher to a smaller student operating on the same task and input domain. In STEP-KD, all stage models share the same architecture and capacity; the challenge is instead an information and representation gap created by the progressive availability of physical-design information. Our mapping modules (which to be best of our knowledge do not exist in prior work) are meant to address this challenge.

Note about W2:

"Some advantages are modest"

We agree that the size of the gains varies across metrics and settings; some are substantial and some are more modest, and we have aimed to report this transparently rather than claim uniform improvements. We believe the consistent gains over direct distillation, independently trained teachers and deeper projections support our main claim.

"Intermediate teachers also do not improve monotonically: Table 2's PP WNS MedAE worsens from 12.11 to 14.46 after distillation, with mixed behavior acknowledged by the authors."

We acknowledge this behavior in the paper. We would note, however, that the intermediate teachers serve to transfer knowledge to the post-synthesis student, which is the only model used at inference, and the student improves on all six held-out metrics when its teachers are distilled along the ladder (Table 2).

"Appendix C shows that explicit early STA slack is particularly informative, but its "slack embedding" variant retains GNN subgraph/global representations"

Could the reviewer kindly elaborate on the weakness or point of confusion here? We note that the new Residual MLP baseline uses the early STA slack without any graph representation, which may address this point.

Reviewer_yYQrOpenReview ↗

Human-written assessment of the paper

My primary concern with this paper is its methodological novelty, rather than simply whether its experimental evaluation is sufficiently comprehensive.

The paper addresses post-synthesis to post-routing timing prediction, which is a practically relevant problem. I also appreciate the authors' idea of progressively transferring knowledge through intermediate physical design stages.

However, I find the underlying research idea quite similar to RTLDistil. RTLDistil has already investigated cross-stage knowledge distillation from layout-aware teachers to RTL-level students and has also adopted multi-granularity representation alignment. STEP-KD further introduces a sequential teacher structure involving PR, PP, PF, and PS stages, but I question whether this extension constitutes a sufficiently substantial methodological innovation.

What I am particularly concerned about is the following: given that existing work has already demonstrated the transfer of physical information across design stages, how much genuinely indispensable additional value does STEP-KD provide by introducing intermediate teachers? If the performance improvements are relatively limited while the training complexity increases considerably, the practical significance of this design becomes less convincing.

I do not mean that the experimental results are entirely ineffective. The main test-set results do demonstrate improvements in both TNS and WNS WMAPE. However, the authors exclude five valid circuits from the OpenLLM-RTL evaluation, and STEP-KD does not outperform STA across all WNS metrics in the external evaluation. Furthermore, the original 20 evaluation runs measure test-set resampling variation rather than independent training-seed variation.

These issues further limit my confidence in the methodological contribution and experimental robustness of the proposed framework.

Compared with existing approaches such as RTLDistil and related knowledge distillation methods, I believe the methodological novelty is insufficient to support a strong claim of originality. I therefore give this paper a Weak Reject recommendation.

Reviewer_2tMzOpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies the interesting topic of predicting post-routing circuit timing from early-stage post-synthesis data. The authors propose STEP-KD, a sequential knowledge distillation framework that leverages intermediate design stages to facilitate knowledge transfer across the integrated circuit design flow.

The framework first represents circuits at different design stages as graphs and uses graph neural networks to extract timing-related representations. It then trains models sequentially from post-routing to post-synthesis, transferring knowledge through cascaded projection modules. The method further adopts multi-granularity knowledge distillation at the node, subgraph, and graph levels to capture timing-related information at different levels of abstraction.

The experimental results show that STEP-KD outperforms direct knowledge distillation and supervised learning baselines in predicting post-routing timing metrics. These results support the effectiveness of the proposed framework.

Critical strengths

S1: The paper addresses an interesting and practically relevant problem of predicting post-routing timing from early-stage circuit information. The idea of leveraging intermediate design stages through sequential knowledge distillation is intuitive and interesting, as it follows the natural progression of the circuit design flow.

S2: The paper constructs a task-specific dataset containing circuit representations from four design stages. This data construction process supports the proposed sequential distillation framework and provides a useful basis for studying timing prediction across different design stages.

Critical weaknesses

W1: The experimental evaluation could be more comprehensive. Although the paper includes several baselines and ablation studies, further comparisons with existing methods and additional analyses of key design choices would help better establish the effectiveness and robustness of the proposed framework.

W2: The presentation could be further improved. Some descriptions are not sufficiently clear or fluent, and the formatting could be polished for better readability. Additionally, several statements in the Introduction would benefit from more explicit references to relevant literature, providing stronger context for the research problem.

Questions for authors

Q1: The experimental evaluation mainly compares STEP-KD with STA and variants of the proposed framework. Could the authors clarify how STEP-KD compares with existing learning-based timing prediction methods? Since STEP-KD trains models at multiple design stages, would it be feasible to compare its intermediate-stage models with existing methods operating at the corresponding stages? Similarly, for methods such as NetTAG that use similar post-synthesis inputs but different prediction targets, could their prediction modules be adapted to the same post-routing timing prediction task? If such comparisons are not feasible, what are the main technical limitations?

Q2: The proposed framework introduces several loss weights. How were these hyperparameters selected, and how sensitive is STEP-KD's performance to their values? Could the authors provide a sensitivity analysis to clarify whether the reported improvements are robust to different hyperparameter settings?

Q3: In Table 1, Direct KD with the PR and PP teachers achieves similar performance. This raises the question of whether every stage in the sequential distillation process is necessary. How would STEP-KD perform with a shorter distillation path, such as PP→PF→PS instead of PR→PP→PF→PS? Could the authors clarify the contribution of each stage and whether the additional stages provide sufficient performance gains to justify their computational cost?

1 comment

We thank the reviewer for their feedback.

Q1. Existing timing-prediction methods differ from STEP-KD in their input stage, prediction target, and training paradigm. RTL-Timer and CircuitFusion operate on RTL text, before technology mapping, and therefore do not use the library-specific cells and timing information available in our post-synthesis netlists. RTL-Timer also predicts post-synthesis rather than post-routing timing. Methods operating after placement use physical information that is unavailable to our final post-synthesis model, so comparing them with STEP-KD would not evaluate the same prediction setting.

NetTAG is the closest related method because it can consume post-synthesis netlists. However, a direct comparison is still not practical. NetTAG is a foundation-model framework based on large-scale self-supervised pretraining and cross-stage contrastive alignment, whereas STEP-KD is trained specifically to predict post-routing endpoint slack and derive TNS/WNS. Adapting NetTAG would require reproducing its pretraining corpus and procedure, redefining its downstream labels as post-routing endpoint slack, and training a new task-specific prediction head on our multi-stage dataset. This would be a substantial new training pipeline, not simply a replacement of the output layer, and the resulting model would no longer represent NetTAG's published experimental setting. Training NetTAG from scratch without its full pretraining would also produce an unfairly weakened baseline.

Q2. We did not tune β1, β2, β3 through a hyperparameter search. We selected them so that the node-, subgraph-, and graph-level loss terms have comparable magnitudes, preventing any single granularity from dominating training. We use β1 = β2 = 10^3 and β3 = 10^4 (with γ_timing = 0.02 and γ_KD = 1), and we have added this procedure and the selected values to the revised manuscript (Section 5.2).

Although our ablations are not a full sensitivity analysis, Figure 3 and Tables 6 and 9 show that the improvements do not depend on one specific granularity. Except for subgraph-only KD on the held-out set, each single-granularity variant improves over No KD. Combining all three levels further reduces TNS WMAPE from 24.1% with graph-level KD alone to 19.8%, supporting the value and robustness of multi-granularity supervision.

Q3.

"Do additional stages provide sufficient performance gains to justify their computational cost?"

Going all the way to PR (rather than stopping at PP→PF→PS) adds almost no computational cost at inference: the only extra cost is a forward pass through one more projection, which is negligible. The extra training cost is that of distilling from PR to the other stages. The main cost of adding another stage is in dataset generation, which is done once, offline.

"Is every stage in the sequential distillation process necessary?"

This is a good question. To address it, we have added a shorter-ladder ablation to the revised paper (new Appendix I). Rather than the exact PP→PF→PS ladder suggested by the reviewer, we remove one intermediate teacher (PP or PF) from the ladder, giving PR→PF→PS and PR→PP→PS, and keep everything else as in STEP-KD. This directly tests whether each intermediate stage is needed. On the held-out test set, both shorter ladders are less accurate than the full ladder on all six metrics (for example, +5.2 and +7.5 percentage points of TNS WMAPE, and +3.9 and +4.1 points of WNS WMAPE). On OpenLLM-RTL, they are less accurate on MedAE, WMAPE and WNS RSE, and slightly more accurate only on TNS RSE (by 1.7 and 3.4 points). Each intermediate stage therefore contributes to the accuracy of the final model.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Message from Authors to All Reviewers

We thank the reviewers for their time and reviews. We have posted individual responses to each reviewer's questions and concerns. Several reviewers asked for additional experiments to further support our results, and we have added the following three to the revised paper (all changes are highlighted in blue):

  • A new Residual MLP baseline (added to Table 1 and Section 5.3)
  • Results on three additional random train/validation/test splits, with all models retrained (new Appendix H)
  • A shorter-ladder ablation (new Appendix I)

We have also added the loss weights, optimizer settings, units and other implementation details requested by the reviewers (Section 5.2 and Appendices B.3 and C).

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.