- Rating
- 2
- Confidence
- 4
- Informativeness
- 1
- Presentation
- 2
- Soundness
- 1
Read review
Paper summary
TurboRTL is a 288-design Verilog benchmark that keeps only designs where e-graph rewriting finds an equivalent variant with lower ADP after DC compile_ultra. It evaluates 19 LLMs single-shot and compares evolutionary search with an agentic workflow.
Critical strengths
The paper targets an important gap, optimization that holds up after real synthesis rather than just functional correctness, and the automated pipeline scales beyond hand-curated benchmarks.
Critical weaknesses
-
E-graph RTL optimization (ROVER, SEER, E-Syn) and LLM-guided e-graph rewriting with real PPA feedback (ASPEN, MLCAD 2025) already exist, so using a fixed-rule e-graph as a filter is a small step. How does your approach differ from ASPEN, and how does d_egg compare with ASPEN-optimized designs on the same inputs?
-
The rewrite rules are a black box. The 76 rules come from an unpublished submission and are never listed, yet they determine every target in the benchmark. Will you release and categorize the rule set?
-
The cost model is undefined. ADP is a post-synthesis metric, and delay is a max-over-paths quantity, so neither can be an additive per-e-node cost before extraction. How is each e-node's cost, especially delay, assigned during extraction, and how well does that estimate agree with post-synthesis ADP?
-
Extraction is heuristic. You use egg's greedy extraction plus DC selection among a few iteration snapshots, not ILP, and report no e-graph sizes or saturation limits. How much does d_egg improve with ILP or PPA-guided extraction, and how many designs actually reach saturation?
-
Two toolchains, one metric. Targets are validated with DC, but evaluation uses Yosys+ABC, and eff@k shifts by up to 15 points between the two. Which flow defines ADP(d_egg) in eff@k, and is eff clipped when a model beats d_egg?
-
"Complex" designs are mostly simple. The average complexity is 0.91 out of 5 and designs average 129 lines, so most are trivial or simple sequential. How many designs fall at each complexity level, and are any at C≥4? Could you try NVDLA or some OpenTitan block level designs?
-
The designs come from RTLCoder, MG-Verilog, and DeepRTL, the same corpora used to train several of the evaluated models. Could memorization explain the identity-copying behavior of the Verilog-specialized models? Could the author provide the results on an unseen testing set designs?
Questions for authors
-
E-graph RTL optimization (ROVER, SEER, E-Syn) and LLM-guided e-graph rewriting with real PPA feedback (ASPEN, MLCAD 2025) already exist, so using a fixed-rule e-graph as a filter is a small step. How does your approach differ from ASPEN, and how does d_egg compare with ASPEN-optimized designs on the same inputs?
-
The rewrite rules are a black box. The 76 rules come from an unpublished submission and are never listed, yet they determine every target in the benchmark. Will you release and categorize the rule set?
-
The cost model is undefined. ADP is a post-synthesis metric, and delay is a max-over-paths quantity, so neither can be an additive per-e-node cost before extraction. How is each e-node's cost, especially delay, assigned during extraction, and how well does that estimate agree with post-synthesis ADP?
-
Extraction is heuristic. You use egg's greedy extraction plus DC selection among a few iteration snapshots, not ILP, and report no e-graph sizes or saturation limits. How much does d_egg improve with ILP or PPA-guided extraction, and how many designs actually reach saturation?
-
Two toolchains, one metric. Targets are validated with DC, but evaluation uses Yosys+ABC, and eff@k shifts by up to 15 points between the two. Which flow defines ADP(d_egg) in eff@k, and is eff clipped when a model beats d_egg?
-
"Complex" designs are mostly simple. The average complexity is 0.91 out of 5 and designs average 129 lines, so most are trivial or simple sequential. How many designs fall at each complexity level, and are any at C≥4? Could you try NVDLA or some OpenTitan block level designs?
-
The designs come from RTLCoder, MG-Verilog, and DeepRTL, the same corpora used to train several of the evaluated models. Could memorization explain the identity-copying behavior of the Verilog-specialized models? Could the author provide the results on an unseen testing set designs?