acceptodds
Withdrawn from ICLR 2027

TREAD: A Transferable RTL Optimizer via Rewrite Trajectories and Synthesis-Grounded Supervision

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

Improving RTL before synthesis is an important way to reduce hardware cost, but effective rewrites depend on both the design structure and the downstream synthesis flow. Manually exploring this space or rediscovering useful transformations for each design is expensive, motivating methods that can learn and transfer optimization experience across RTL designs. We develop TREAD, a transferable RTL optimizer that learns such experience from rewrite trajectories. Imitation learning first provides an executable policy over legal rule–location actions, while offline reinforcement learning captures long-horizon rewrite effects from complete trajectories and delayed returns. To overcome the limited action coverage of observed trajectories, we generate alternative legal rewrites from the same source states and evaluate these unselected actions under a common Design Compiler flow, providing synthesis-grounded preferences for value and policy learning. At inference, TREAD performs bounded residual search beyond the imitation endpoint and accepts a learned residual only when its predicted utility exceeds a positive margin. Across seven RTL benchmark categories under a common technology library and synthesis flow, TREAD achieves the best QoR among the evaluated RTL optimizers under the reported synthesis flow, including representative e-graph-based and LLM-based systems. These results suggest that combining executable behavior, long-horizon returns, and same-state synthesis feedback enables optimization experience to transfer across RTL designs and improve RTL optimization.

Primary area:
reinforcement learning
Keywords:
RTL optimization, RTL rewriting, reinforcement learning, imitation learning, hardware design automation

Reviews

Among:
Shape:
1234this paper, Bottom 9%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.50 over 4 reviews, above 9% of the others.
Reviewer_J2wvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes TREAD, a transferable RTL optimizer that learns rewrite policies reusable across designs through imitation learning, offline reinforcement learning, same-state synthesis supervision, and residual search. TREAD constructs trajectories using e-graphs, encodes states with graph neural networks, selects rule-position pairs via a factorized policy, assigns long-term credit using trajectory returns, and generates counterfactual synthesis data for local preference signals.

Critical strengths

RTL optimization is a practical bottleneck in the hardware design flow. TREAD combines offline RL with synthesis feedback to reduce online synthesis calls, which is a sound idea with a well-equipped set of methodological components. It also includes thorough system design and ablation studies, demonstrating tangible efficiency improvements.

Critical weaknesses
  1. In my view, the main issue with this paper is that the core methodology consists entirely of a combination of known techniques, including imitation learning for policy initialization, offline TD learning, double Q-networks, reward predictors, and residual search. No new learning methods or theories are introduced specifically for RTL optimization, resulting in limited novelty.
  2. The experimental scale is relatively small. The number of valid designs mentioned in Section 4.2.3 is limited. For MIPS, only 10–13 designs are used. Furthermore, the experiments lack statistical significance testing, and the RL component lacks stability testing. Additionally, a direct comparison with ABC-RL is missing.
Questions for authors
  1. Please provide a detailed comparison with existing methods, including whether new ideas or observations are introduced, as well as comparisons with key baselines, including an adapted comparison with ABC-RL and variants of ROVER combined with synthesis feedback.
  2. Could the authors expand the experimental scale? For example, use more random seeds, include statistical significance tests, and report means and standard deviations.
  3. Please report the total number of DC invocations and the runtime for trajectory collection and counterfactual synthesis. Currently, I cannot determine whether this computational cost would be prohibitive.
  4. How were the hyperparameters in Equation (18) determined? There seem to be quite a few hyperparameters to tune, it might be worth investigating whether different settings could yield better results.
Reviewer_6E5EOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
1
Soundness
1
Read review
Paper summary

The paper learns a transferable RTL optimization policy from rewrite trajectories. It combines imitation learning, reinforcement learning and synthesis measurements. At inference, it extends the imitation policy output through residual search, accepting a replacement only when its predicted utility exceeds a fixed margin.

Critical strengths

Same state synthesis supervision addresses limited action coverage by directly evaluating alternatives absent from observed trajectories. Its separation of immediate synthesis feedback from long horizon value learning is motivated.

Critical weaknesses
  1. Several Table 4 rules are not generally sound under integer semantics. For example, (A/B -> A(1/B)) changes (2) to (0).
  2. Table 3’s aggregate results are inconsistent with the stated calculation. The displayed TREAD values yield a geometric mean of 0.7648, whereas the table reports 0.7750. Discrepancies also occur in the other columns.
  3. Results are aggregated over each method’s valid outputs rather than a common design set. For MIPS, IL includes 13 designs, while the other variants include 10. Complete failure counts and exclusion reasons are needed to assess whether these differences affect the reported gains.
  4. Acceptance depends on a predicted score, but the paper reports neither the frequency of accepted residuals that worsen measured ADP nor sensitivity to the acceptance threshold. Its effectiveness in preventing regressions relative to IL remains unclear.
  5. Results use one reported training seed and one TSMC 28 nm synthesis setup, without uncertainty estimates. The reliability of small baseline margins cannot be assessed, and transfer beyond the evaluated library and timing constraints is untested.
  6. Table 3’s reporting precision needs clarification. In the intermediate ablation columns, 18 of 21 values end in zero at four decimal places, compared with 3 of 14 for IL and TREAD.
  7. Several Table 1 entries exactly match the original design or another method at the reported precision.
Questions for authors
  1. What guards make the division, shifted multiplication, and modular exponentiation rules in Table 4 sound? Are final inference outputs formally verified, and what are the held-out equivalence pass rates?
  2. How were Table 3’s averages computed?
  3. Why do the valid MIPS design counts differ? Please report valid, failed, and excluded counts by method and category, and repeat the comparisons on a common design set.
  4. How often do accepted residuals worsen ADP relative to IL? How were the score coefficients and threshold selected, and how sensitive are results to the threshold?
  5. Can you report multiple seed results with uncertainty estimates and paired comparisons for the small baseline margins?
  6. Were the intermediate ablation values rounded differently? Please provide reported results and the unrounded measurements.
  7. Which matching Table 1 entries represent unchanged designs, and which arise from independent optimization?
Reviewer_gwfrOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

This paper proposes an e-graph rewriting approach for logic synthesis optimization. Egraphs are intermediate representations of boolean logic that can be rewritten in a range of different functionally equivalent formats, and can then be synthesized by commercial tools into actual logic. The paper adopts a foundation learning and RL approach to this problem.

Critical strengths

The major trends of this paper are the results achieved that outperform existing approaches, including RTL writers and other Egraphs approaches. Ablation studies demonstrate that these benefits come largely from the RL approach adopted.

Critical weaknesses

One of the major weaknesses of the paper is that the results are still illustrated on relatively small graphs and circuits. While there is a generally accepted dearth of data on the internet since RTL is largely proprietary, yet results on large open designs like OpenTitan and other RISC-V processors would be far more compelling. This is a large part because synthesis run times and design complexity can explode easily therefore in addition to absolute improvements, the scalability of an approach to large designs is crucial to establish. Second, if I'm not mistaken it appears that most optimizations here are combinational, but do not involve sequential rewrites that may not be purely logically equivalent but sequentially equivalent to the original design.

Questions for authors
  1. Can you handle rewrites that would produce the same input-output behavior on a cycle-by-cycle basis? In other words if the only requirement was sequential equivalence, would you be able to cover all such transformations? Retiming would be one example.
  2. Can you comment on scalability to large designs and provide some evidence of this approach on large open timing blocks?
Reviewer_XiuKOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper proposes TREAD, a transferable RTL optimization framework that learns from synthesis-aware rewrite trajectories. The method first uses imitation learning to establish an executable rewrite policy, followed by offline reinforcement learning to capture long-term optimization effects. Additional Design Compiler evaluations of alternative rewrites from the same source state provide synthesis-grounded supervision. At inference time, bounded residual search further improves the imitation endpoint when the predicted utility exceeds a positive margin. Experiments across seven held-out RTL benchmark categories report improvements in post-synthesis area-delay product (ADP) and reduced test-time runtime compared with bounded e-graph search.

Critical strengths

1.The problem is practically relevant.

Learning transferable RTL optimization strategies to reduce repeated synthesis and design exploration is a meaningful research direction in hardware design automation.

2.The framework is comprehensive and reports encouraging results.

The integration of imitation learning, offline reinforcement learning, and synthesis feedback provides a structured approach to RTL optimization, supported by evaluations across several design categories.

Critical weaknesses
  1. Insufficient guarantees of functional equivalence raise fundamental soundness concerns.

Appendix Table 4 includes rewrite rules that are not generally valid under integer or bit-vector semantics. For example, a/b → a*(1/b) does not preserve ordinary integer division. The paper does not clearly specify the additional arithmetic, width, and signedness constraints under which such transformations become valid.

Although equivalence checking is described during offline data construction, the final evaluation primarily requires successful synthesis and positive ADP. It remains unclear whether every final optimized RTL design is formally verified against the original implementation. Since functional equivalence is a fundamental requirement of RTL optimization, this uncertainty directly undermines confidence in the reported QoR improvements.

  1. Excluding invalid designs and using inconsistent evaluation subsets weakens the experimental conclusions.

Section 4.2.3 and Appendix I exclude failed or invalid synthesis results from the reported ADP statistics. Moreover, the MIPS CPU ablation includes 13 valid designs for IL but only 10 for the other configurations.

Computing geometric means over different valid subsets prevents a strictly paired comparison and may introduce selection bias. Without complete validity rates and per-design paired results, the reported improvements cannot be reliably interpreted as consistent gains across the full benchmark.

  1. The runtime comparison does not adequately control synthesis costs.

Table 2 reports runtime reductions of 66.45%–87.77%. However, the bounded e-graph baseline synthesizes 20 candidate designs, whereas TREAD performs only one final synthesis. Much of the reported speedup may therefore result from differences in the number of synthesis calls rather than the intrinsic efficiency of the learned optimization strategy.

Furthermore, offline trajectory collection, counterfactual synthesis labeling, and model training are excluded from the runtime accounting. Without matched synthesis budgets or an amortized cost analysis, the paper does not sufficiently establish a superior overall optimization-efficiency trade-off.

  1. The incremental contribution and statistical robustness of the learning components remain insufficiently established.

TREAD combines established techniques, including imitation learning, offline reinforcement learning, synthesis-based supervision, and bounded search. Table 3 shows that IL+RL already achieves a 21.09% average ADP reduction, compared with 22.50% for the complete method. The additional average benefit from same-state synthesis supervision is therefore relatively modest.

The evaluation also relies on a single training seed and does not provide uncertainty estimates for the main QoR comparisons. Consequently, the necessity and reproducibility of the complete framework over simpler alternatives remain insufficiently demonstrated.

Questions for authors

1.Functional correctness: What exact validity conditions apply to the potentially non-equivalent rewrite rules in Appendix Table 4? Are all final optimized RTL designs formally verified against their original implementations? Please provide the equivalence-verification results for the complete test set.

2.Evaluation validity: Why do different component configurations contain different numbers of valid MIPS CPU designs? Can the authors provide paired per-design ADP results and report synthesis and equivalence-verification failures separately?

3.Efficiency: How does TREAD compare with e-graph optimization under matched synthesis-call or wall-clock budgets? What are the total offline data-collection, synthesis-labeling, and training costs, and how many deployments are needed to amortize them?

4.Incremental contribution: Does the improvement from same-state synthesis supervision over IL+RL remain significant across multiple training seeds and held-out designs?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.