- Rating
- 2(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper presents SchGen, a framework for generating editable PCB schematics from natural-language requirements. Its main contribution is Code-L1, an executable representation combining anchor-relative component placement with pin-name-based electrical connections. The authors construct a dataset of 2,105 schematics from 1,390 unique designs through an LLM-assisted reconstruction and human-correction pipeline, and fine-tune two language-model backbones on synthesized request–program pairs. Experiments compare alternative representations, general-purpose LLMs, and performance on an independently collected GitHub dataset, reporting improvements in netlist accuracy and expert-assessed functional correctness.
Critical strengths
-
Practical representation design with meaningful empirical gains. Code-L1 abstracts electrical connectivity from low-level wire geometry while preserving relative component placement. The controlled ablations show substantial improvements over absolute-coordinate and explicit-wire representations on both fine-tuned backbones, supporting the value of the proposed abstraction.
-
Useful dataset construction and evaluation effort. The human–agent reconstruction pipeline addresses the scarcity of paired natural-language requests and editable PCB schematics. The evaluation covers execution validity, netlist accuracy, spatial quality, and expert-assessed functionality, providing a more informative assessment than syntactic validity alone.
-
Encouraging transfer across data sources. The additional evaluation on 988 schematics from GitHub projects shows that the fine-tuned models retain advantages on designs collected outside the original SparkFun source. The component-overlap analysis further helps characterize the distribution shift.
Critical weaknesses
- Limited algorithmic novelty and generalizability. SchGen primarily relies on existing LLMs and standard supervised fine-tuning, with its contributions centered on task-specific engineering rather than novel learning algorithms or reasoning mechanisms. Despite promising empirical results, the paper provides limited exploration of broader applicability within the PCB design domain or transferable methodological insights.
- Limited realism of the evaluation. The in-domain test requests are synthesized from existing schematics, potentially making the task closer to reconstructing known designs than generating circuits from real user requirements. Performance on independently written, underspecified engineering requests remains unclear.
- Insufficient comparison with competitive design pipelines. The experiments mainly compare simplified representation variants or fine-tuned models against general-purpose LLMs. Stronger baselines using existing netlist-first generation and automated schematic construction under matched training conditions are missing, making it difficult to establish the advantage of the proposed approach.
Questions for authors
-
Beyond task-specific engineering and fine-tuning existing LLMs, what generalizable methodological insights does SchGen introduce?
-
How does SchGen perform on independently written engineering requests and larger, more complex PCB schematics?
-
Can the authors compare against a netlist-first or SKiDL-based generation pipeline under matched training and inference conditions?
-
How are electrical constraints beyond KiCad ERC checks validated? Can the authors provide more detailed evidence of expert agreement and failure analysis?
5 comments
[Response to Reviewer ua6o, cont']
W3 / Q3. Matched netlist-first or SKiDL pipeline
A related netlist representation is SKiDL, which directly compiles a netlist without spatial or editable schematic design. Although this is different from the problem we want to solve, we agree that this representation comparison is helpful.
A fine-tuned netlist-first baseline. We converted the original dataset to SKiDL programs and fine-tuned the same Qwen3.8-27B backbone using the original training/test split. This adds a learned netlist-first baseline, rather than comparing Code-L1 only with prompted general-purpose LLMs. The following results are from tests over the same test set of 500 samples.
| Method (SFT setup; checkpoint) | Pass Rate (%) | Netlist Jaccard (%) | Netlist precision (%) | Netlist recall (%) |
|---|---|---|---|---|
| SchGen (Qwen3.8-27B) | 95.2 | 70.7 | 76.3 | 74.5 |
| SKiDL (Qwen3.8-27B) | 97.2 | 58.97 | 61.25 | 61.26 |
Both methods achieve high execution/export pass rates (95.2% for SchGen and 97.2% for SKiDL), while SchGen's reported netlist Jaccard, precision, and recall are higher by 11.73, 15.05, and 13.24 percentage points, respectively, which highlights that even compared to netlist-based representation, SchGen can still maintain both readability and higher performance of netlist accuracy.
Q4. Electrical constraints, expert agreement, and failure analysis
Beyond KiCad’s Electrical Rules Check (ERC), we use expert review to assess whether generated schematics can meet the user’s design request. As described in Appendix §6.6 (p. 19), six experts evaluated the generated schematics using the user request, the reference schematic, and relevant component datasheets when needed. Electrically equivalent implementations that satisfy the request are accepted.
We use the following rubrics:
- Symbol Error. Count missing, incorrect, or duplicated components. Electrically suitable substitutes are accepted when the request does not specify an exact part.
- Connection Error. Count required functional pins that are left unconnected, connected to the wrong net, or involved in an unintended short. Electrically equivalent connections are accepted regardless of wire routing or net-label naming.
- Functional Correctness. Judge whether the complete schematic can reasonably perform the requested functions. A positive judgment requires zero symbol and connection errors and a separate assessment of end-to-end functionality.
We report symbol and connection errors as mean counts per schematic across all experts, and functional correctness as the percentage of evaluated schematics that receive a positive judgment among all experts. We separately reconcile disagreement for a small amount of samples not reaching full agreement. Below is a table of the raw distribution of expert agreement, which shows the consistent agreement within experts.
| Number of Disagreements | Number of schematics |
|---|---|
| 0 (No disagreement) | 478 |
| 1 | 11 |
| 2 | 8 |
| 3 | 3 |
| Total | 500 |
Failure-case study. We inspected retained Code-L1 evaluation records for the two SchGen backbones in the main results. The examples below represent the main failure types across all samples.
| Failure type | SchGen backbone and example | Observed evidence |
|---|---|---|
| Incomplete program | Qwen3.8-27B, ZED-F9P GPS | Generated code ends inside an add_schematic_symbol call; a SyntaxError prevents netlist export. |
| Invalid pin reference | Qwen3.8-27B, indoor air-quality sensor | The code requests LX on the AP3012 symbol, whose library pin is named SW; pin lookup fails before export. |
| Ambiguous pin selection | Qwen3.8-27B, mosaic-T UART protection | All four UART signals use the same D1.K selector; pins 1, 3, and 4 remain unconnected. Only 2/5 reference nets match: precision 100%, recall 40%. |
| Connectivity mismatch after export | GPT-OSS-20B, XBee development board | The saved evaluation reports successful execution/export, but the single evaluated net matches neither of the two reference nets: precision and recall 0%. |
[Kind Reminder of Feedback to Comments]
Dear Reviewer,
Thank you again for your thoughtful review.
As the author–reviewer discussion period is drawing to a close and the deadline is only a few days away, we would greatly appreciate any final feedback regarding whether our rebuttal has adequately addressed your concerns.
If there are still any remaining issues or questions, we would be more than happy to clarify them before the discussion period ends.
Thanks a lot for your time and hard work.
[Response to Reviewer ua6o]
We sincerely thank the reviewer for recognizing the empirical representation gains, the dataset-construction effort, and transfer across data sources.
We agree that the following aspects need clearer explanation or more analysis. We will update our paper accordingly to address them.
W1 / Q1. Methodological insight beyond standard fine-tuning
SchGen’s methodological contribution, recognized by the other two reviewers, Vd1x and NWZR, is the code representation named Code-L1, an executable representation derived from how engineers design schematics. Engineers organize components around functional anchors and connect pins according to their roles in the circuit. Code-L1 encodes these decisions as anchor-relative placement and named-pin connections. The model specifies the intended circuit relationships, and the API constructs the wire geometry and writes the KiCad schematic. The representation benefits both prompting and supervised fine-tuning. With API specifications and examples supplied in the prompt, Code-L1 improves expert-assessed functional correctness over Code-L2 by 22.4–24.0 percentage points across GPT-5.5, GPT-5.2, and Grok-4 (Table 3). Under supervised fine-tuning, the ablation shows the contribution of each design choice. For Qwen3.8-27B, netlist Jaccard increases from 19.6% with explicit wire geometry to 50.9% with named-pin wiring, then to 70.7% with anchor-relative placement. GPT-OSS-20B shows the same ordering (Table 2).
Besides, SchGen also made contributions to the dataset and corresponding construction pipeline for the community. Frontier AI labs, Anthropic reports results on internal PCB benchmark, while OpenAI reports results on its internal design tasks. Restricted access to the underlying tasks and data limits what independent researchers can reproduce and build on from these evaluations. We have already released the model, dataset, and the whole pipeline but have not disclosed them here due to anonymity. Researchers can use this shared dataset to train new models, construct new datasets, and evaluate schematic generation.
Methodological Insights: Applying LLMs to hardware engineering has attracted wide interest, including Anthropic [https://www.anthropic.com/research/claude-electronics-design] and OpenAI [https://openai.com/index/accelerating-engineering-for-the-physical-world/]. However, it still raises a common representation problem`. Many hardware designs can be understood as spatial graphs whose components have both functional connections and geometric relationships. Our proposed SchGen uses PCB schematics to study how these designs can be represented in a form that LLMs can learn and generate more efficiently. The general method is to identify how engineers express design intent and encode those decisions as executable operations. In Code-L1, the model places components relative to functional anchors and connects pins by name, and the backend constructs the wires and writes the schematic. The same principle could guide representation design for other engineering artifacts, such as mechanical CAD, where a model could specify how named features mate, and for chip design, where it could connect block ports and specify relative placement constraints.
W2 / Q2. Independently authored requirements and larger designs
From independently written requirements to working hardware. To test this, we add two requirement-driven hardware design case studies involving three users.
Each case starts from a natural-language device specification. SchGen decomposes the request into functional blocks and calls the model with written requirements, then generates and merges the block-level schematics. The resulting boards were then tested separately for correctness. We also asked the 3 experts to perform a similar task for a complete PCB board and record the time consumed.
These cases demonstrate the practical role of automating repetitive design operations to reduce manual labor so that human effort can focus on requirements, inspection, correction, and functional verification.