- Rating
- 3
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces CktFormalizer, an agent framework that translates natural-language hardware descriptions into typed circuit representations in Lean and exports them to SystemVerilog. The framework combines compiler-guided generation and repair with separate experiments on equivalence proof generation and hardware optimization. Evaluation spans VerilogEval, RTLLM, ResBench, and a CVDP subset, using RTL and gate-level simulation, synthesis, and physical-design checks. Reported compilation rates range from 91.1% to 99.4%. A 30-problem proof pilot improves kernel-accepted proof generation from 16 to 19 cases through stepwise feedback, while optimization experiments report synthesis-area reductions of up to 58.6%.
Critical strengths
- Clear engineering contributions. These include CKTLean, a typed DSL for characterizing RTL semantics, as well as a complete pipeline for hardware design, verification, and PPA optimization.
- Solid experiments. They cover 4 mainstream RTL generation benchmarks, and beyond compilation and simulation, they also evaluate metrics required in real-world hardware design scenarios, such as synthesis, place-and-route, DRC/LVS.
Critical weaknesses
-
There is a mismatch between the "Lean-to-RTL" paradigm and real-world hardware design scenarios. The method proposed in this paper uses Lean to describe the behavior of a circuit and then converts it into RTL. For a hardware designer without a background in functional programming and theorem proving, the cost of learning Lean and its toolchain is prohibitively high. A more common technical route in the field is to convert the RTL code written by hardware designers into Lean ("RTL-to-Lean") [1-2] and prove properties over it, thereby verifying the correctness of an existing design so that design engineers need not concern themselves with the specific details of formal verification. The authors should provide additional evidence to demonstrate that the proposed method can indeed be used in real-world IC design scenarios.
-
Missing comparison with other verification methods. In the PPA Optimization Loop, this paper guarantees the correctness of the implementation during optimization by generating Lean proofs of the spec. However, in the traditional hardware design flow, one could also use model checking (i.e., having the model generate SystemVerilog assertions [3]) or fuzzing testing [4-5] to verify the correctness of the optimization. The authors need to explain the advantages of theorem proving over these methods. Moreover, compared with assertion or test-case generation, generating formal proofs may incur a larger token overhead.
[1] Rtl2lean: Automated RTL-to-Lean Translation with Hierarchical Theorem Generation and Lemma Reuse. https://arxiv.org/abs/2607.16855
[2] CircuitProver: Agentic Lean 4 Theorem Proving with Reusable Circuit Proof Library for Hardware Verification. https://arxiv.org/abs/2607.27259
[3] QiMeng-CodeV-SVA: Training Specialized LLMs for Hardware Assertion Generation via RTL-Grounded Bidirectional Data Synthesis. https://arxiv.org/abs/2603.14239
[4] AutoPPA: Automated Circuit PPA Optimization via Contrastive Code-based Rule Library Learning. https://arxiv.org/abs/2604.18445
[5] QiMeng-CodeV-R1: Reasoning-Enhanced Verilog Generation. https://arxiv.org/abs/2505.24183
Questions for authors
- Can the conversion from CKTLean to Verilog guarantee semantic consistency? If not, is there a human-evaluated accuracy rate?
- Can CKTLean support all Verilog features? The semantics of the same Verilog code may differ when synthesized by different EDA tools (e.g., Yosys)—how does CKTLean handle this?
6 comments
Response to the Reviewer KHhE Part II
Q2: Missing comparison with other verification methods. In the PPA Optimization Loop, this paper guarantees the correctness of the implementation during optimization by generating Lean proofs of the spec. However, in the traditional hardware design flow, one could also use model checking (i.e., having the model generate SystemVerilog assertions [3]) or fuzzing testing [4-5] to verify the correctness of the optimization. The authors need to explain the advantages of theorem proving over these methods. Moreover, compared with assertion or test-case generation, generating formal proofs may incur a larger token overhead.
A2: Lean supports reusable libraries of machine-checked circuit lemmas, allowing component properties and optimization rules to be proved once and reused under their stated assumptions. Effective reuse requires semantic retrieval: identifying lemmas whose properties and assumptions match the current verification goal. Lean lemmas can preserve meaningful names, typed interfaces, and component-level abstractions that support such retrieval. Semantically relevant results are often harder to retrieve from low-level SMT formulas, where the connection to circuit components and design intent is less explicit.
Such reuse is critical for industrial-scale verification, where recurring IP blocks and frequent revisions make repeated verification costly. Existing proofs can help focus effort on modified logic and component interactions. We acknowledge that proof generation may require more tokens than generating assertions or tests. Reusable lemmas could amortize this cost across designs and optimization iterations.
Response to the Reviewer KHhE Part III
Q3: Can the conversion from CKTLean to Verilog guarantee semantic consistency? If not, is there a human-evaluated accuracy rate?
A3: Thanks for your valuable suggestion!
We formulate equivalence between CktLean and a semantic model of the emitted SystemVerilog as a proof obligation in Lean 4. We performed per-design translation validation on all 152 compiled single-clock synchronous VerilogEval designs:
| Design set | Evaluated | Equivalence proved | Inequivalence proved | Equivalence proof rate |
|---|---|---|---|---|
| All eligible designs | 152 | 150 | 2 | 98.7% |
| RTL-simulation-passing subset | 129 | 129 | 0 | 100% |
Each equivalence proof establishes agreement on every output port at every cycle, for every input sequence under the modeled semantics. Failed equivalence proofs can also help reveal semantic modeling issues. In the two cases reported here, inequivalence was additionally proved in Lean. Further analysis exposed a registerWithEnable inconsistency: its Lean simulation semantics failed to preserve state when the enable signal was low, while the emitted RTL behaved correctly. These cases demonstrate the diagnostic value of translation validation for identifying inconsistencies between the source semantics and generated RTL.
We additionally cross-checked the SystemVerilog-side model against Icarus Verilog and tested the validation procedure using injected mutations:
| Additional validation | Result |
|---|---|
| Cycles cross-checked against Icarus Verilog | 28,000 |
| Combinational designs exhaustively checked | 63 |
| Injected SystemVerilog mutations rejected | 8/8 (100%) |
We resolve the source-to-RTL verification gap for all 152 evaluated designs under the modeled semantics through per-design translation validation in Lean 4. Each design receives a machine-checked proof of either equivalence or inequivalence, supported by additional validation through Icarus Verilog cross-checks and mutation testing.
When proof reveals a behavioral mismatch, we can feed the diagnostic information back into the design loop, correct the responsible modeling or translation component, and repeat validation until equivalence between the Lean and emitted SystemVerilog semantics is proved. A Lean-checked equivalence proof thus serves as the acceptance criterion for the revised design.
Response to the Reviewer KHhE Part I
Q1: There is a mismatch between the "Lean-to-RTL" paradigm and real-world hardware design scenarios. The method proposed in this paper uses Lean to describe the behavior of a circuit and then converts it into RTL. For a hardware designer without a background in functional programming and theorem proving, the cost of learning Lean and its toolchain is prohibitively high. A more common technical route in the field is to convert the RTL code written by hardware designers into Lean ("RTL-to-Lean") [1-2] and prove properties over it, thereby verifying the correctness of an existing design so that design engineers need not concern themselves with the specific details of formal verification. The authors should provide additional evidence to demonstrate that the proposed method can indeed be used in real-world IC design scenarios.
A1: We thank the reviewer for highlighting this usability concern. CktLean serves as an intermediate representation (IR) within our agentic workflow. Designers provide natural-language specifications and interface requirements, while the agent generates and manipulates the Lean representation. This IR makes the workflow more general and flexible by decoupling circuit generation and formal reasoning from any particular backend. The same representation can support RTL generation, C++ simulation, and JIT execution, extending its use beyond RTL- based implementation.