acceptodds
Under review as a conference paper at ICLR 2027

Search-Guided Reasoning Trace Curation for Verilog Generation

Abstract

Recent advances in reasoning-capable large language models (LLMs) have improved Verilog generation. However, most training pipelines distill natural-language reasoning traces whose intermediate design decisions are weakly structured and are primarily selected according to the correctness of the final generated RTL. We present MCTS-SFT, a verifier-guided framework for curating reasoning traces for supervised fine-tuning (SFT) of Verilog generation models. MCTS-SFT formulates trace construction as search over intermediate reasoning states and uses Monte Carlo Tree Search (MCTS) to propagate functional feedback from complete Verilog rollouts to earlier design decisions. Within this framework, we introduce Plan-MCTS, which searches over compact hardware-design plans that explicitly represent interfaces, timing, state, behavior, and update rules. We further study plan-preserving supervision, in which the searched structured state is retained as part of the downstream SFT target. Our experiments find that Plan-MCTS achieves the highest functional trace yield, with 84.2% correct traces compared with 81.9% for one-shot structured planning and 82.0% for unguided tree exploration. For identical Plan-MCTS trajectories and Verilog outputs, retaining the explicit searched plan during SFT raises pass@10 from 52.56% to 57.7%, demonstrating that the representation of intermediate design decisions materially affects downstream learning. In a separate 6,000-trace study, MCTS-curated supervision improves pass@10 over full-response chain-of-thought distillation by up to 3.7 points while reducing downstream generation length by approximately 50% on VerilogEval-v2. These results demonstrate that structured hardware planning, verifier-guided search, and explicit intermediate-state supervision provide complementary mechanisms for constructing more effective and token-efficient training data for Verilog generation.

Primary area:
unsupervised, self-supervised, semi-supervised, and supervised representation learning
Keywords:
LLM, reasoning, Verilog, fine-tuning, MCTS

Reviews

Among:
Shape:
1234this paper, Bottom 27%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.00 over 4 reviews, above 27% of the others.
Reviewer_n6cvOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

This paper introduces MCTS-SFT, a reasoning-trace curation framework for Verilog code generation. Motivated by the limited control over intermediate design decisions and their supervisory representation in full-response CoT distillation, the framework uses functional verification results from generated code to guide search over intermediate states. Its Plan-MCTS variant organizes interface, timing, state, and behavioral decisions into structured hardware plans and examines the effect of retaining these plans in supervised fine-tuning targets. With search trajectories and final Verilog implementations held fixed, explicit-plan supervision increases pass@10 from 52.56% to 57.70%. The 6k-trace study uses rollout-only targets and reports accuracy improvements across several VerilogEval tasks while approximately halving the average response length on VerilogEval-v2. On 172 multi-module CVDP tasks, Plan-MCTS and the CoT baseline achieve success rates of 12.21% and 9.30%, respectively, providing preliminary evidence of cross-benchmark transfer, although absolute success rates remain low.

Critical strengths
  1. The paper provides a task-specific approach to reasoning-data curation for Verilog generation by incorporating hardware design decisions into both the search state and the supervision target. Structured plans explicitly represent interfaces, timing, state, and update rules, allowing intermediate decisions to be progressively refined and evaluated through functional verification of the generated code. Using executable feedback for trajectory selection without training an additional process reward model offers a practical approach to constructing supervision for hardware design reasoning.

  2. The experiments examine how the representation of structured hardware plans affects supervised fine-tuning of Qwen2.5-Coder-7B-Instruct for Verilog generation. With search trajectories, final implementations, and verifier rewards held fixed, explicit structured plans outperform both plan omission and natural-language plan rendering, indicating that the organization of hardware design information affects its value as supervision. Comparisons with one-shot planning, random tree exploration, and reasoning compression further examine the main design choices. Reporting functional accuracy alongside response length also characterizes generation quality and output efficiency.

Critical weaknesses
  1. The baseline comparisons do not adequately establish the method’s advantages over existing training approaches. The experiments primarily compare CoT, planning, and search variants constructed within this study, without directly evaluating existing Verilog generation models. This leaves the method’s relative performance unclear. Moreover, the answer-only baseline is derived from conventional CoT data rather than the same Plan-MCTS samples. The absence of a matched code-only comparison leaves the contributions of the search-generated Verilog implementations and intermediate reasoning supervision insufficiently separated.

  2. The performance–cost trade-off of MCTS relative to one-shot planning remains unclear. Plan-CoT achieves 57.05% pass@10, close to the 57.70% obtained by Plan-MCTS with explicit-plan supervision, and attains a higher pass@1. The paper does not report variability in these small differences across training runs or quantify the actual number of model calls, cumulative token usage, and data-curation time. Consequently, neither the stability of the search gains nor their cost-effectiveness relative to one-shot planning is established.

  3. Explicit-plan supervision lacks further validation across tasks and training datasets. Its representation-level comparisons are limited to the 1k-trace study on VerilogEval-v1-Human, whereas the broader 6k-trace evaluation uses rollout-only targets. The specifications in the main studies come from VeriThoughts, and the scaling experiment in the appendix does not provide a comparison using an independent training dataset. Evaluations across VerilogEval tasks and CVDP provide cross-benchmark evidence, but do not establish whether the benefits of explicit-plan supervision depend on the task or training-data source.

  4. The method’s effectiveness across student models has not been evaluated. The main experiments use Qwen2.5-Coder-7B-Instruct throughout, without comparisons across model families or parameter scales. The experiment using GPT-5.4-mini as the teacher retains the same student and lacks CoT and Plan-CoT baselines under that teacher. The current results therefore do not establish consistent improvements across base models or characterize the dependence of the gains on the teacher configuration.

Questions for authors
  1. Could you compare the method with representative existing Verilog generation models under a consistent evaluation protocol and document differences in training data, model size, and inference budget? For comparisons between training pipelines, using the same student model and comparable training budgets where feasible would help establish the gains over existing approaches.

  2. Could you construct code-only targets from the same Plan-MCTS samples and compare them with rollout-only and explicit-plan supervision? How does a plan-and-code target perform when the subsequent free-form reasoning is omitted? These comparisons would help separate the contributions of Verilog implementation quality, plan supervision, and reasoning supervision.

  3. Could you report the actual data-curation costs of MCTS-SFT and compare them with CoT and Plan-CoT? The number of model calls, cumulative input and output tokens, and wall-clock time would be informative. Please also clarify which resources are matched by the stated generation budget and report variability across random initializations to characterize the stability of the search gains and their computational cost.

  4. What motivated the omission of explicit plans from the 6k-trace training targets? Could you provide matched comparisons between explicit-plan and rollout-only supervision on the existing VerilogEval-v2 tasks to examine the effectiveness of plan supervision across tasks and at a larger data scale?

  5. Could you reproduce the main comparisons on an independent training dataset with a different specification distribution or design complexity, using the same student and comparable training budgets? This would help assess whether the gains depend on the characteristics of VeriThoughts and distinguish transfer across evaluation benchmarks from applicability across training-data sources.

  6. Could you evaluate the relative gains of Plan-MCTS, Plan-CoT, and conventional CoT training on students from different model families or parameter scales? Non-search baselines under the second teacher would also help characterize how teacher configurations and student models affect the results.

2 comments
AuthorsOpenReview ↗

Rebuttal - Reviewer n6cv (Part 1)

We thank the reviewer for the constructive suggestions on baselines, supervision, and generalization. We seek to address each question above through testing what the plan and rollout reasoning contribute, alongside comparisons across students and teachers.

Questions and answers

  • Q1. How does MCTS-curated training compare with existing Verilog pipelines?
    A1. With the same Qwen student and roughly 11,000 SFT examples per model, the student trained on MCTS-curated traces scored 56.66% at pass@10 versus 44.65% for the student trained on a CodeV-R1 subset.
  • Q2. What do plans and rollout reasoning add when the final RTL is fixed?
    A2. Adding rollout reasoning to the same RTL raised pass@10 by 5.13 percentage points, and including the plan alongside that reasoning shortened outputs by 604 tokens on average.
  • Q3. How much does curation cost, and how stable are the results?
    A3. Plan-MCTS recorded 15.58M teacher tokens versus 3.03M for CoT and 4.73M for Plan-CoT, and both paired task intervals in the two-seed comparison include zero.
  • Q4. Do explicit plans help when restored to the 6k targets?
    A4. Explicit plans raised pass@1 by 1.06 percentage points on Code Completion and 0.23 on Spec-to-RTL, while rollout-only led at pass@5 and pass@10.
  • Q6. How do the rankings change with another student or teacher?
    A6. Plan-MCTS led at pass@10 in both the Llama student and GPT-5.4-mini teacher comparisons, while CoT led at pass@1 in both.

Detailed responses

1. Comparison with existing Verilog models

We fine-tuned Qwen2.5-7B-Instruct on roughly 11,000 examples per model. One dataset contained MCTS-curated traces from VeriThoughts specifications. The other contained a subset of CodeV-R1 training examples. We evaluated both on VerilogEval-v1 Human (156 tasks, 20 samples each).

SFT dataOptimizer stepsPass@1Pass@5Pass@10
MCTS-curated VeriThoughts10k38.14%51.84%56.66%
CodeV-R1 subset11k28.78%40.55%44.65%

The MCTS-trained student led at all three k values. Its 12.01-percentage-point lead at pass@10 has a paired 95% task interval of [6.04, 17.99]. This comparison holds the base model and approximate SFT dataset size fixed. The optimizer step counts differ as shown, and the exact decoding limits for those evaluations are unverified.

Published VeriThoughts-7B scores 37.2% at pass@1 and 56.8% at pass@10 with a 1,024-token output cap. Our 11k model leads its pass@1 by 0.94 points and is within 0.14 points at pass@10. Our 15k model leads at all three k values, reaching 41.73%, 54.97%, and 59.60% at pass@1, pass@5, and pass@10. Full CodeV-R1 reports 81.7% pass@10 with 87,000 SFT examples, reinforcement learning, and a 16,384-token context. VeriThoughts · CodeV-R1

2. Isolating RTL, plans, and rollout reasoning

Of the 1,000 Plan-MCTS trajectories in the controlled study, 956 had terminal RTL available for all four target formats. We trained four Qwen2.5-7B-Instruct models on those same 956 examples, holding the prompts and final RTL fixed and adding the selected plan, rollout reasoning, both, or neither. Each model generated 10 samples per task on the 156 VerilogEval-v1 Human tasks.

SFT targetPass@1Pass@5Pass@10Mean tokens
RTL only30.32%45.12%50.64%2,282
Plan + RTL26.99%46.17%51.92%2,367
Rollout + RTL29.62%49.89%55.77%3,523
Plan + rollout + RTL31.09%48.67%55.13%2,919

Rollout reasoning gave the largest observed pass@10 gain over RTL alone, +5.13 percentage points (paired 95% task interval [−0.64, +10.90]). Adding just the plan to RTL gave +1.28 points [−4.49, +6.43]. Adding the plan to the rollout reduced mean response length from 3,523 to 2,919 tokens, with pass@1 up 1.47 points and pass@10 down 0.64 points [−6.41, +5.13]. All three pass@10 intervals include zero.

AuthorsOpenReview ↗

Rebuttal - Reviewer n6cv (Part 2)

3. Curation cost and variability

We matched 1,000 retained traces per method and the per-specification quotas. The table includes all ten logged attempts per method on each of the 142 shared specifications, including attempts discarded before SFT.

MethodLogged teacher activityInput / output tokensYosys starts / testbench returnsMedian min/spec
CoT1,417 completion replies0.647M / 2.384M1,185 / 1,1408.47
Plan-CoT1,420 planner starts, 1,122 rollout replies1.386M / 3.341M1,063 / 1,00420.08
Plan-MCTS4,232 candidate replies, 1,416 rollout replies4.525M / 11.054M1,346 / 1,25526.03

Reply counts include responses with token records. Plan-CoT token totals cover accepted plans and their rollouts. Testbench returns come from the compile-and-simulate procedure, with simulation conditional on successful compilation. Exact retry and simulator-launch counts are unavailable.

We refit Plan-CoT and Plan-MCTS at SFT seeds 42 and 43 using each method's fixed 1,000-example dataset, a common split and LoRA recipe, and targets containing a structured plan, rollout reasoning, and RTL. On VerilogEval-v1 Human at 10 samples per task, Plan-CoT reached 55.13% pass@10 at both seeds, and Plan-MCTS reached 51.92% and 54.49%. The Plan-MCTS minus Plan-CoT gaps were −3.21 and −0.64 percentage points, with paired 95% task intervals of [−8.33, +2.56] and [−5.13, +4.49], respectively. The intervals resample tasks for fixed checkpoint pairs. The two fits per method probe training variation while holding curation fixed.

4. Explicit-plan supervision at 6k

The original targets came from the DeepSeek-R1 rollouts, and the DeepSeek-V3 plans were recorded separately in the search traces. We prepended the selected plan to each target, keeping the same 6,000 trajectories from 600 specifications and holding prompts, reasoning, and RTL fixed. Each VerilogEval-v2 evaluation used 20 samples per task across 156 tasks.

VerilogEval-v2SFT targetPass@1Pass@5Pass@10Pass@20
Code CompletionRollout only44.10%59.00%63.57%67.31%
Code CompletionExplicit plan45.16%58.16%62.03%64.10%
Spec-to-RTLRollout only41.73%58.16%63.20%66.67%
Spec-to-RTLExplicit plan41.96%57.33%62.78%68.59%

Explicit plans raised pass@1 by 1.06 percentage points on Code Completion and 0.23 on Spec-to-RTL. Rollout-only led at pass@5 and pass@10 on both tasks. At pass@20, rollout-only led by 3.21 points on Code Completion and explicit plans led by 1.92 points on Spec-to-RTL. The comparison therefore shows a trade-off across metrics. Paired task intervals for the 6k differences are pending.

6. Alternate student and second-teacher controls

We evaluated Llama 3.1 8B on VerilogEval-v1 Human (156 tasks, 10 samples each). The three reasoning-based models use 1,000 examples per method. The RTL-only model uses the 956 Plan-MCTS examples with available terminal RTL.

Llama SFT armRowsPass@1Pass@5Pass@10
CoT1,00025.51%42.88%46.79%
Plan-CoT rollout1,00023.97%40.76%48.08%
Plan-MCTS rollout1,00024.04%44.11%50.64%
Plan-MCTS RTL-only95622.50%37.54%41.67%

CoT led at pass@1. Plan-MCTS led at pass@5 and pass@10, by 3.85 points over CoT and 2.56 over Plan-CoT at pass@10. Isolating the effect of RTL-only supervision requires training a rollout model on the same 956 examples.

With GPT-5.4-mini as teacher, we matched 5,390 sample IDs across 600 specifications for CoT, Thought-MCTS, and Plan-MCTS. All three Qwen students trained on terminal RTL only. We evaluated with 10 samples per task on VerilogEval-v1 Human.

Teacher strategyPass@1Pass@5Pass@10
CoT37.88%48.15%50.64%
Thought-MCTS36.35%46.34%48.08%
Plan-MCTS37.44%48.84%51.92%

Plan-MCTS led at pass@5 and pass@10, including a 1.28-point lead over CoT at pass@10. CoT led at pass@1 by 0.44 points.

Conclusion

The student trained on MCTS-curated data led the student trained on a CodeV-R1 subset by 12.01 points at pass@10. With identical RTL, rollout reasoning gave the largest observed pass@10 gain. We thank the reviewer again for proposing controls that helped sharpen these conclusions.

Reviewer_BHn3OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper proposes MCTS-SFT, which uses executable Verilog verification at completed rollouts and Monte Carlo Tree Search to curate reasoning traces for supervised fine-tuning. Its main variant, Plan-MCTS, searches over structured hardware-design plans covering interface, clock/reset, state, combinational logic, FSM, edge cases, and implementation constraints. In a controlled 1,000-trace study, Plan-MCTS has an 84.2% functional trace yield, versus 81.9% for one-shot planning and 82.0% for random-tree exploration. With the same Plan-MCTS trajectories and Verilog outputs, retaining the explicit structured plan in the SFT target improves VerilogEval-v1-Human pass@10 from 52.56% to 57.70%. In a larger 6,000-trace study, Plan-MCTS improves pass@10 over full-response CoT by 3.2 points on VerilogEval-v1-Human and 3.7 points on VerilogEval-v2 Code Completion, while reducing response length on the two VerilogEval-v2 tasks by about half. A preliminary 172-task multi-module evaluation is also reported.

Critical strengths
  1. Good controlled decomposition. The 1k study compares CoT, Thought-MCTS, Plan-CoT, Plan-RandomTree, and Plan-MCTS under a common teacher, verifier, budget, student, and SFT configuration. The fixed-trajectory SFT-target comparison is particularly informative because it holds terminal Verilog and rewards constant.

  2. Practical efficiency result. Plan-MCTS improves functional correctness while reducing generated response length substantially on VerilogEval-v2, which is relevant for training and deployment cost.

Critical weaknesses
  1. Limited statistical and experimental breadth. The controlled study is relatively small and the experiments use one 7B student and one primary teacher configuration, limiting generality. The paper also leaves the treatment of failed rollouts and unsuccessful traces insufficiently specified, which makes the data curation and MCTS-learning pipeline harder to assess.

  2. Verifier/data dependence is under-characterized. Training examples and testbenches come from the VeriThoughts pipeline and automatically constructed testbenches. The paper does not quantify testbench coverage, false-positive/false-negative rates, overlap or contamination with evaluation prompts, or whether search over repeated verifier calls introduces selection bias. These issues matter because the method optimizes exactly the same type of executable feedback used for curation.

Questions for authors
  1. Could the authors disclose how failed cases are handled throughout curation and training? In particular, how are non-compiling rollouts and compilable-but-testbench-failing implementations represented in the datasets, how do they affect MCTS value propagation and the reported functional-yield denominators, and are any failed traces discarded or repaired before SFT?

  2. How are the training specifications and automatically generated testbenches separated from VerilogEval-v1/v2 and CVDP, and what measurements support that the testbenches provide adequate coverage rather than reward overfitting?

  3. In the 6k study, why is DeepSeek-V3 used for Plan-MCTS state construction while DeepSeek-R1 is used for rollouts, and how much of the gain remains when all intermediate states and rollouts are produced by the same teacher?

2 comments
AuthorsOpenReview ↗

Rebuttal - Reviewer BGn3 (Part 1)

We thank the reviewer for the thoughtful feedback. The questions about failure handling, benchmark independence, and teacher assignment helped us clarify how the verifier shapes the training set and where the evidence supports generalization. We address each question through the experiments below:

Questions and answers

  • Q1. How do failed rollouts affect search and the SFT cohort?
    A1. Their terminal rewards update search values, and the 1,000 retained traces per method include partial and failed examples under shared quotas.
  • Q2. How independent are the benchmarks, and what do the results say about verifier coverage?
    A2. The screen flagged no duplicate or near-duplicate prompts, and the Table 2 gain from adding the selected plan remained 4.70 percentage points at pass@10 after excluding seven VerilogEval tasks.
  • Q3. Why split planning and rollout between two teachers, and what happens with a single teacher?
    A3. V3 handles repeated, compact plan updates and R1 writes full rollouts at 6k, while the all-R1 1k comparison shows a higher retained full-pass yield for Plan-MCTS than for Plan-CoT.

Detailed responses

1. Failed rollouts, reward propagation, and SFT

For RTL that compiles and runs, the reward in Eq. 4 is the fraction of tests passed, with reward 1 for a full pass and 0 if all tests fail. A compilation failure ordinarily receives −1. During MCTS, we increment visit counts along the selected path and use the terminal reward to update running mean action values (Eq. 6). This lets failed rollouts inform subsequent choices. In the implementation, −1 can also indicate RTL extraction failure or an exception. Zero can also indicate a timeout or unavailable verifier component.

We reproduced the Table 1 counts from the traces used to train the five methods. For each specification usable by all methods, we selected traces with nonempty reasoning in file order up to a common quota. The denominator is 1,000 retained traces per method across 142 unique specifications, which we will clarify in the manuscript. Failures with usable training targets could therefore enter SFT.

Curation methodFull passPartialZeroLogged −1
CoT7315414201
Thought-MCTS8364210112
Plan-CoT8195915107
Plan-RandomTree8204517118
Plan-MCTS842491297

For Plan-MCTS, this retains 158 traces below reward 1, including 97 assigned reward −1. These are counts by reward. A complete breakdown of extraction failures, compilation failures, timeouts, and any upstream repairs would require the corresponding failure records.

To test whether keeping failed traces mattered for the student, we compared full-pass-only and mixed-quality SFT using the same student, original target format within each method, and 632 examples per arm across 116 specifications with matched quotas. On VerilogEval-v1 Human (156 tasks, 10 samples each), filtering raised Plan-CoT pass@10 from 54.49% to 55.77% and lowered Plan-MCTS pass@10 from 53.85% to 52.56%. The result varies by curation method at fixed dataset size.

AuthorsOpenReview ↗

Rebuttal - Reviewer BGn3 (Part 2)

2. Benchmark independence and verifier coverage

We screened the 142 controlled and 600 extended VeriThoughts training specifications against the 156 VerilogEval designs in v1 Human and both v2 prompt formats, plus 172 CVDP prompts. We checked exact normalized text, overlap of consecutive five-word sequences, and TF-IDF cosine similarity. For near matches, we flagged pairs sharing at least 10 five-word sequences with either Jaccard overlap ≥0.20 or containment ≥0.50, and pairs with TF-IDF cosine ≥0.65. No prompt pair met any of the criteria. In the 600-specification comparison, the maximum Jaccard overlap was 0.081 and the maximum TF-IDF cosine was 0.403 across the evaluation suites.

We also compared the VerilogEval reference RTL and RTL visible in 86 CVDP prompts or input files. No exact RTL match was found before consistently renaming identifiers. After renaming, four VerilogEval tasks shared a module with training RTL and one additional task had a similar multiplexer with different default behavior. The only CVDP flag involved an adder helper supplied in the task input.

We rescored the Table 2 Plan-MCTS models on VerilogEval-v1 Human using their original ten samples per task. The two models were trained on the same 1,000 trajectories and final RTL. Only one target included the selected plan. The plan-supervision gain persisted both after excluding the five flagged tasks and after also excluding two related tasks from the same exam family.

Evaluation subsetTasksRollout-only pass@10Explicit-plan pass@10Difference
All v1 Human tasks15652.56%57.69%+5.13 pp
Excluding five RTL flags15150.99%56.29%+5.30 pp
Also excluding two related tasks14951.68%56.38%+4.70 pp

The original VeriThoughts study used v1 Human and a separate 291-question holdout scored by formal equivalence. Our full 6,000-trace study uses the same source corpus for training and evaluates on v1 Human and both v2 tasks using benchmark-supplied tests. This supports transfer across evaluation tasks. During training-data curation, we check compilation with Yosys and compare against the training reference RTL using CodeV-R1-derived testbenches. The original training testbenches and completed CVDP targets were unavailable for the overlap screen. To directly measure training-testbench coverage, we would rescore the same rollouts with independent stimuli or formal equivalence and report how often they disagree with the original verifier. VeriThoughts, §4.1

3. DeepSeek-V3 for planning and DeepSeek-R1 for rollout

As described in Appendix A.1.1, we used DeepSeek-V3 to reduce the cost of the repeated expansion stage, which retains only a compact plan update. DeepSeek-R1 produced the full reasoning and RTL from the selected plan. For the 600-specification extended study, the teacher logs report the following token counts.

Extended-study stageReplies with token countsInput tokensOutput tokens
V3 planning17,82611,804,497915,675
R1 rollout5,9953,272,27416,463,734

Planning accounted for about three times as many recorded replies. Rollouts accounted for about 95% of the recorded output tokens.

The controlled 1k study used DeepSeek-R1 for both stages. Plan-MCTS had a higher retained full-pass yield than Plan-CoT (84.2% vs. 81.9%) on the shared specifications. For the downstream comparison we trained Qwen2.5-7B-Instruct with the same structured plan, rollout, and RTL target format for both methods. The table gives pass@10 on v1 Human with 10 samples per task.

MethodPass@10, seed 42Pass@10, seed 43
Plan-CoT55.13%55.13%
Plan-MCTS51.92%54.49%

The paired 95% task intervals for Plan-MCTS minus Plan-CoT were [−8.33, +2.56] percentage points at seed 42 and [−5.13, +4.49] at seed 43. With a single teacher, Plan-MCTS retained a greater share of fully passing traces, and its downstream advantage under the same SFT target format remains uncertain. Measuring how much of the extended result depends on teacher assignment requires a paired 6,000-trace all-R1 run with the same specifications, target format, and student recipe.

Conclusion

The dataset audit establishes how partial and failed rollouts enter the denominator, and the filtering experiment shows that keeping them can help, depending on the method. Removing seven flagged or related benchmark tasks leaves a 4.70-point pass@10 benefit for the selected plan. Independent checks of the training testbenches would address verifier coverage directly. With R1 as the sole teacher at 1k, Plan-MCTS retained more fully passing traces. We appreciate the reviewer’s guidance in clarifying what these verifier and teacher controls establish.

Reviewer_wZKcOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MCTS-SFT, which uses Monte Carlo tree search guided by functional testbench feedback to construct Verilog training data. Plan-MCTS searches structured hardware plans and studies how to retain them in supervised fine-tuning (SFT) targets. It achieves an 84.2% functional trace yield, compared with 81.9% for one-shot Plan-CoT. With the same Plan-MCTS trajectories and final RTL, retaining explicit plans raises student pass@10 from 52.56% to 57.70%. A separate 6,000-trace experiment evaluates downstream accuracy and response length using a different teacher configuration and rollout-only supervision.

Critical strengths
  1. The paper provides useful comparisons with Plan-CoT and Plan-RandomTree, which help distinguish the effects of structured planning, tree exploration, and verifier-guided selection. These controlled comparisons make it easier to assess the contribution of MCTS rather than attributing all improvements to the search method.
Critical weaknesses
  1. The additional value of MCTS over one-shot structured planning is not sufficiently established. In Table 2, Plan-MCTS improves pass@10 over Plan-CoT by only 0.65 percentage points (57.70% vs. 57.05%), while pass@1 is 0.87 points lower (31.50% vs. 32.37%). The higher trace yield in Table 1 is positive evidence, but does not by itself establish a worthwhile downstream benefit. The reported generation settings and dataset sizes do not account for all expansion, rollout, and verification costs. Without actual cost measurements and a comparison at comparable total budgets, the case for the more complex search procedure remains incomplete.
  2. The reliability and scope of the learned benefits remain uncertain. Independent training repeats and sensitivity analyses are not reported for the small Plan-MCTS–Plan-CoT differences, so it is unclear whether their ranking persists under reasonable training and decoding variations. The 6,000-trace experiment changes both teacher configuration and supervision format and omits a same-scale Plan-CoT baseline; it therefore does not establish that the MCTS increment or explicit-plan benefit persists at larger scale. The evaluation also provides limited evidence about which hardware decisions the student learns more reliably or which task types benefit most from search.
Questions for authors
  1. What are the actual data-curation costs of Plan-CoT, Plan-RandomTree, and Plan-MCTS, including teacher calls and tokens, rollouts, testbench executions, and runtime? At comparable total budgets, does Plan-MCTS retain an advantage? A simpler verifier-guided greedy selection control would also help assess the value of the tree-selection rule.
  2. What changes in student behavior distinguish Plan-MCTS-SFT from Plan-CoT-SFT? Can the authors analyze plan accuracy and plan-to-RTL consistency, and compare benefits across combinational, sequential, FSM, and multi-module tasks? The existing multi-module results do not include Plan-CoT, leaving the search-specific benefit unclear.
  3. Have the authors conducted ablations or sensitivity analyses covering independent SFT runs, data sampling, decoding settings, prompt paraphrases, or student configurations? Please report available uncertainty estimates and representative checks showing whether the Plan-CoT–Plan-MCTS ranking persists. Training variability and decoding-sampling uncertainty should be distinguished; a second-teacher experiment does not resolve either.
  4. Why does the 6,000-trace experiment use rollout-only supervision when explicit plans perform best in the controlled study? Are the full SFT target formats matched across Plan-CoT, Plan-RandomTree, and Plan-MCTS? A controlled comparison at the same scale would help separate the effects of planning, search, and target representation.
  5. How are the 1,000 generated trajectories in Table 1 mapped to the 1,000 SFT examples in Table 2, given that only 842 Plan-MCTS trajectories pass all tests? Please clarify filtering, replacement, and failed-trace handling across methods. Separately, the CoT response means in Table 4 approach the 8,192-token cap: what are the cap-hit and incomplete-RTL rates, and does a higher cap change the accuracy–length comparison?
2 comments
AuthorsOpenReview ↗

Rebuttal - Reviewer wZKc

We appreciate the reviewer’s detailed feedback. Your questions prompted us to quantify curation costs and test how the downstream results vary by circuit type and evaluation setting. We seek to address each question through the experiments below:

Questions and answers

  • Q1. What does each strategy cost, and how much do the budget and selection rule explain?
    A1. We report logged teacher usage, verification activity, and time for each strategy, and the token-matched CoT and verifier-guided greedy comparisons remain open.
  • Q2. Where does Plan-MCTS help across circuit types, and how well do students follow their plans?
    A2. Keeping the searched plan gave the largest pass@10 gain on FSM-related tasks, while student plan fidelity and a matched multi-module comparison with Plan-CoT remain open.
  • Q3. Does the ranking persist across training runs and student configurations?
    A3. Plan-CoT led at both seeds with matched Qwen targets, while Plan-MCTS led at pass@10 in the separate Llama reasoning-target comparison.
  • Q4. Which methods share a target format, and what changes when we restore the plans at 6k?
    A4. At 1k we matched the structured targets for Plan-CoT and Plan-MCTS, and at 6k explicit plans slightly raised v2 pass@1 while rollout-only led at pass@10.
  • Q5. Which traces entered SFT, and how often did generations approach the token cap?
    A5. Each method used 1,000 retained traces including failures, and 87.8% of full-response CoT completions on Code Completion and 88.4% on Spec-to-RTL fell within 16 tokens of the 8,192-token context limit.

Detailed responses

1. Curation cost and comparable budgets

We matched 1,000 retained examples per method and per-specification quotas. Costs cover 1,420 logged attempts per method across 142 shared specifications, including traces excluded from SFT.

MethodInput / output tokensYosys starts / testbench returnsMedian wall time (min/spec)
CoT0.647M / 2.384M1,185 / 1,1408.47
Plan-CoT1.386M / 3.341M1,063 / 1,00420.08
Plan-RandomTree2.845M / 7.451M1,329 / 1,22718.70
Plan-MCTS4.525M / 11.054M1,346 / 1,25526.03

Plan-CoT logged 1,420 planner starts and 1,122 rollouts with token counts. Its tokens cover accepted plans and rollouts. At the same configured branching width, Plan-RandomTree and Plan-MCTS logged 815 and 1,420 expansions, 2,440 and 4,232 candidate replies with token counts, and 1,416 rollouts each. Testbench returns record the compile-and-simulate procedure, with simulation conditional on compilation. Exact provider-request and simulator-launch counts are unavailable.

CoT logged 1,417 replies and 3.03M tokens, versus 5,648 candidate and rollout replies and 15.58M tokens for Plan-MCTS. The CoT pool covers 19.5% of Plan-MCTS's token budget, so equal-budget filtering requires more CoT samples. Plan-RandomTree tests unguided exploration.

2. Plan accuracy, plan-to-RTL consistency, and task type

We grouped 156 VerilogEval-v1 Human tasks using specifications and reference RTL. The finite-state machine (FSM) group includes combinational next-state and output logic for FSMs. The Table 2 rollout-only and explicit-plan Plan-MCTS models share 1,000 trajectories and their final RTL. Only the explicit-plan target includes the selected plan. At ten samples per task, pass@10 is the share of tasks with at least one pass.

Circuit typeTasksPlan-CoT rolloutPlan-MCTS rolloutPlan-MCTS with planGain from adding plan
Combinational7475.68%71.62%74.32%+2.70 pp
Other sequential4461.36%50.00%54.55%+4.55 pp
FSM-related3815.79%18.42%28.95%+10.53 pp

Adding the plan increased the number of FSM-related tasks solved from 7 to 11 out of 38 (exploratory paired 95% task interval [2.63, 21.05] percentage points). With rollout-only targets, Plan-MCTS led Plan-CoT by 2.63 points on FSM-related tasks, and Plan-CoT led in the other two groups. The endpoint is functional RTL. Directly assessing whether the student writes a correct plan and follows it in RTL requires the full student responses. Table 5 reports multi-module CVDP transfer for CoT, Thought-MCTS, and Plan-MCTS.

AuthorsOpenReview ↗

Rebuttal - Reviewer wZKc (Part 2)

3. Training repeats and sensitivity

We trained Qwen2.5-7B-Instruct twice per method with fixed 1,000-example datasets, split, and LoRA recipe. Both targets included a structured plan, rollout reasoning, and RTL.

SFT seedPlan-CoT pass@10Plan-MCTS pass@10MCTS minus Plan-CoT, pp [95% CI]
4255.13%51.92%−3.21 [−8.33, +2.56]
4355.13%54.49%−0.64 [−5.13, +4.49]

Plan-CoT led at both seeds, with a narrower gap at seed 43. Both intervals include zero. Each interval uses 10,000 resamples of the 156 paired VerilogEval-v1 Human task scores for one fixed checkpoint pair, with ten samples per task. The two fits per method probe training variation.

In a separate trace-quality check on the same benchmark, we matched 632 examples and quotas across 116 specifications. Filtering for full passes moved pass@10 from 54.49% to 55.77% for Plan-CoT and from 53.85% to 52.56% for Plan-MCTS.

We also trained Llama 3.1 8B on 1,000 reasoning targets per method, with rollout-only targets for the two plan-based methods. CoT led at pass@1, while Plan-MCTS led at pass@5 and pass@10 on VerilogEval-v1 Human.

Llama SFT methodPass@1Pass@5Pass@10
CoT25.51%42.88%46.79%
Plan-CoT23.97%40.76%48.08%
Plan-MCTS24.04%44.11%50.64%

4. Target format and the 6k study

Originally, Plan-CoT and Plan-RandomTree used rollout-only targets at 1k. Plan-MCTS also had rendered and structured plans. Q3 matches the structured format across Plan-CoT and Plan-MCTS.

The original 6k dataset used DeepSeek-R1 rollout reasoning and RTL as targets. The DeepSeek-V3 plans were recorded separately in the search traces. We recovered the selected plan for each of the same 6,000 trajectories across 600 specifications and prepended it to the student target. This holds the teachers, prompts, rollout reasoning, and RTL fixed.

Benchmark, 20 samples per taskRollout-only pass@1 / 10Explicit-plan pass@1 / 10
VerilogEval-v1 Human35.29% / 57.94%36.54% / 58.76%
VerilogEval-v2 Code Completion44.10% / 63.57%45.16% / 62.03%
VerilogEval-v2 Spec-to-RTL41.73% / 63.20%41.96% / 62.78%

Explicit plans raised pass@1 by 1.06 percentage points on Code Completion and 0.23 on Spec-to-RTL. At pass@10, explicit plans led by 0.82 points on v1 Human, while rollout-only led by 1.54 points on Code Completion and 0.41 on Spec-to-RTL. This comparison isolates target format within Plan-MCTS.

5. Trace mapping and the 8,192-token cap

Table 1 counts rewards on the same 1,000 traces per method used in Table 2. We filled common quotas in file order across 142 shared specifications, skipping traces with empty reasoning. Plan-MCTS's 1,000 traces consisted of 842 full passes, 49 partial passes, 12 zero-reward traces, and 97 traces with reward −1. The 158 traces below full pass all contributed to SFT, and the three Plan-MCTS variants in Table 2 use the same terminal RTL.

For the context-limit analysis, we used previously generated VerilogEval-v2 responses from full-response CoT, which trains on the teacher's reasoning and Verilog, and rollout-only Plan-MCTS. The CoT evaluation results reproduce the scores and mean output lengths in the corresponding Table 4 row. The exact training checkpoint for that row is unverified. Each method generated 20 responses per task across 156 tasks in each v2 suite, giving 3,120 responses per method per suite.

We counted prompt and response tokens together with the Qwen2.5-7B-Instruct tokenizer. “Near the limit” means within 16 tokens of the 8,192-token context window. We used the CodeV-R1 RTL extractor to determine whether a complete top module could be recovered.

VerilogEval-v2 taskMethodMean response tokensNear the limitNo complete top module
Code CompletionFull-response CoT7,316.62,739/3,120 (87.8%)2,044/3,120 (65.5%)
Code CompletionRollout-only Plan-MCTS3,622.4829/3,120 (26.6%)678/3,120 (21.7%)
Spec-to-RTLFull-response CoT7,312.72,757/3,120 (88.4%)1,761/3,120 (56.4%)
Spec-to-RTLRollout-only Plan-MCTS3,710.4892/3,120 (28.6%)769/3,120 (24.6%)

The saved responses lack recorded stop reasons, so proximity is a proxy for hitting the context limit. Full-response CoT was closer to the cap and more often lacked a complete extracted module on both tasks.

Conclusion

The audit quantifies the extra curation work Plan-MCTS used to meet shared training quotas and shows which failed traces entered SFT. The largest Table 2 gain from including plans was on FSM-related tasks. The Qwen and Llama comparisons show that rankings vary across settings, and the 6k comparison shows a trade-off between pass@1 and pass@10. On both v2 tasks, full-response CoT more often approached the context limit and lacked a complete module. We thank the reviewer again for their detailed feedback.

Reviewer_4retOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper proposes MCTS-SFT, a framework for building better reasoning traces to fine-tune Verilog generation models. A teacher LLM explores intermediate reasoning steps with Monte Carlo Tree Search. Each partial reasoning state is completed into full Verilog and scored with a testbench, and that score is propagated back to rate the earlier decisions. The main variant, Plan-MCTS, searches over a structured hardware plan with seven fields: interface, clock/reset, state elements, combinational logic, FSM, edge cases, and constraints. It also keeps this plan in the fine-tuning target.

In a controlled study (100 specs, 1,000 traces per method, DeepSeek-R1 as teacher, Qwen2.5-Coder-7B as student), Plan-MCTS has the highest share of correct traces (84.2% vs. 81.9% for one-shot planning). Keeping the explicit plan in the training target raises pass@10 on VerilogEval-v1-Human from 52.56% to 57.7%. In a larger 6k-trace study, Plan-MCTS beats full chain-of-thought distillation by up to 3.7 pass@10 points while roughly halving output length on VerilogEval-v2. It also shows a small gain on 172 multi-module CVDP tasks.

Critical strengths
  1. Well-designed controls. The paper includes one-shot planning without search (Plan-CoT), random tree search (Plan-RandomTree), summarized free-form reasoning (Summary-MCTS), and three training targets built from the same search trajectories. The fixed-trajectory comparison is the most convincing result. It shows that how intermediate reasoning is presented to the student matters, even when the final Verilog is identical.
  2. Practical efficiency gain. Cutting generated length by about half while keeping or improving accuracy is useful, given how verbose distilled reasoning models are for RTL.
Critical weaknesses
  1. The search itself doesn’t clearly help; the plan does. One-shot planning without any tree search (Plan-CoT) scores 57.05% pass@10, almost the same as full Plan-MCTS (57.70%), and it is higher at pass@1. Free-form Thought-MCTS does worse than plain CoT (51.28 vs. 52.56). With GPT-5.4-mini as teacher, Plan-MCTS doesn’t beat Thought-MCTS either. So the gains seem to come from the structured plan format, not from MCTS, even though MCTS is the title and core contribution.

  2. The larger study doesn’t test the proposed method. The 6k experiment uses a different teacher setup (DeepSeek-V3 for planning) and a training target without the plan, which is the version the controlled study showed works worst. So the headline 6k gains and the 50% token reduction come from a different recipe than the one the paper recommends. The two studies also don’t support each other cleanly. Compute is not matched either: MCTS makes many more teacher calls and verifier runs than sampling CoT, so it’s unclear whether the gain beats simply sampling more CoT traces and filtering for correct ones.

Questions for authors
  1. Do the SFT datasets include traces that failed the testbench? If so, what happens if every method is trained only on correct traces?
  2. Does Plan-CoT’s training target include the plan? If so, what does MCTS add beyond Plan-CoT, given their nearly equal results?
  3. How many teacher calls and verifier runs does each method use? Can you compare against CoT given the same budget (e.g., sampling more and filtering)?
  4. Can you report results over multiple seeds, or with confidence intervals?
  5. Why does the 6k study use the rollout-only target instead of the explicit plan you recommend? Do you have 6k results with the explicit plan?
2 comments
AuthorsOpenReview ↗

Rebuttal - Reviewer 4ret (Part 1)

We thank the reviewer for the careful reading and constructive feedback. These questions helped us distinguish the quality of the retained traces, the cost of acquiring them, and what the student learns from them. We seek to address these questions through the experiments below:

Questions and answers (Overview)

  • Q1. Did SFT include failed traces, and does filtering them out help?
    A1. The original SFT cohorts included failed traces, and at matched size, filtering for full passes helped three methods at pass@10 while Thought-MCTS and Plan-MCTS did better with mixed data.
  • Q2. What is in the Plan-CoT target, and what does search add?
    A2. The original Plan-CoT target contained only rollout reasoning and RTL, and Plan-MCTS retained a higher share of fully passing traces than Plan-CoT (84.2% vs. 81.9%).
  • Q3. What does curation cost, and can CoT be compared at the same budget?
    A3. Plan-MCTS used 15.58M logged teacher tokens versus 3.03M for CoT, and a token-matched filtering comparison requires more CoT samples.
  • Q4. How does the matched search comparison vary across training seeds?
    A4. Plan-CoT led at both SFT seeds, the pass@10 gap narrowed at seed 43, and both paired task intervals include zero.
  • Q5. What changes when explicit plans are added to the 6k targets?
    A5. Prepending the recovered plans to the original rollout-only targets slightly improved pass@1 on both VerilogEval-v2 tasks, while rollout-only led at pass@10.

Detailed responses

1. Failed traces and correct-only training

The original SFT data include failed traces. We recovered 1,000 retained traces per method across 142 shared specifications and will correct the manuscript's cohort description. The exporter applied common per-specification quotas and skipped empty targets. For Plan-MCTS, it retained 842 full passes, 49 partial passes, 12 zero-reward traces, and 97 traces with reward −1. All 158 traces below full pass remained in the SFT corpus.

To test the effect of filtering, we matched 632 full-pass traces with 632 mixed-quality traces per method, using the same quotas across 116 specifications. We evaluated on VerilogEval-v1 Human with 10 samples for each of 156 tasks.

MethodCorrect-only pass@10Mixed pass@10
CoT55.13%53.85%
Thought-MCTS51.92%54.49%
Plan-CoT55.77%54.49%
Plan-RandomTree56.41%55.77%
Plan-MCTS52.56%53.85%

Filtering helped three methods at pass@10, while Thought-MCTS and Plan-MCTS did better with mixed data. Because the training set size and quotas were fixed, the comparison measures the effect of changing which traces were retained.

2. Plan-CoT targets and the value of search

The original Plan-CoT teacher used a one-shot plan to guide the rollout. Its SFT target contained only the rollout reasoning and Verilog. On the same specifications, Plan-MCTS had a higher retained full-pass yield (84.2% vs. 81.9%) at higher curation cost. In a separate comparison using the same Plan-MCTS trajectories and final RTL, including the searched plan raised archived v1 Human pass@10 from 52.56% to 57.69%.

The comparison in Q4 gives both students the same target format to test whether the search yield carries through to student performance.

3. Teacher and verifier cost

We matched retained size and quotas. These costs cover ten logged attempts per method on each of 142 shared specifications, including work discarded before SFT.

MethodPlanning activityCompletion or rollout usage recordsTestbench returns
CoTNone1,4171,140
Plan-CoT1,420 planner starts1,1221,004
Plan-RandomTree2,440 candidate usage records1,4161,227
Thought-MCTS1,827 candidate usage records3721,261
Plan-MCTS4,232 candidate usage records1,4161,255

Thought-MCTS also reused 240 rollouts. CoT and Plan-MCTS recorded 3.03M and 15.58M teacher tokens, 1,185 and 1,346 verifier entries, and median times of 8.47 and 26.03 min/spec. Exact provider requests and simulator launches are unavailable. The 1,417 usable CoT traces give 417 extra candidates for filtering. A token-matched comparison requires more CoT samples.

AuthorsOpenReview ↗

Rebuttal- Reviewer 4ret (Part 2)

4. Training-seed repeats for the matched search comparison

We trained four Qwen2.5-7B-Instruct adapters, one per method at each of two SFT seeds. Each method used the same 1,000 traces across seeds. Both methods shared the 142 specifications, train/validation split, and LoRA recipe. Both targets included a structured plan, rollout reasoning, and RTL. We scored 10 samples on each of 156 VerilogEval-v1 Human tasks, so pass@10 is the share of tasks with at least one passing sample.

SFT seedPlan-CoT pass@1 / pass@10Plan-MCTS pass@1 / pass@10MCTS minus Plan-CoT at pass@10 [95% CI]
4232.12% / 55.13%31.22% / 51.92%−3.21 pp [−8.33, +2.56]
4331.60% / 55.13%30.77% / 54.49%−0.64 pp [−5.13, +4.49]

Each interval uses 10,000 paired resamples of benchmark tasks for one fixed checkpoint pair. The two fits per method probe sensitivity to the training seed. Plan-CoT led at both seeds, with a smaller gap at seed 43. Both intervals include zero. The higher trace yield in Q2 is clear for the retained cohort, while the downstream benefit of search under matched supervision remains uncertain in this case.

5. Explicit-plan supervision at 6k

The extended study used the pipeline's rollout-only export. DeepSeek-V3 built the plans and DeepSeek-R1 wrote the rollouts. We recovered the selected plans and prepended them to the same 6,000 targets, holding the reasoning and RTL fixed.

Benchmark, 20 samples per taskRollout-only pass@10Explicit-plan pass@10
VerilogEval-v1 Human57.94%58.76%
VerilogEval-v2 Code Completion63.57%62.03%
VerilogEval-v2 Spec-to-RTL63.20%62.78%

On v1 Human, explicit plans led at all four k values, including a 0.82-percentage-point gain at pass@10 (58.76% vs. 57.94%). At pass@10, rollout-only led on both v2 tasks.

Conclusion

The filtering ablation demonstrates that the best trace selection policy can differ by method between fully correct and mixed datasets. Plan-MCTS retained more fully passing traces than Plan-CoT, and our curation audit quantifies the extra work behind that result. With the same SFT target format, the two seed repeats mitigate the benefit of downstream search. Keeping the searched plan improved pass@10 on the archived 1k trajectories, and the 6k comparison shows that the effect depends on the evaluation metric. We thank the reviewer again for their feedback.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.