- Rating
- 3(+1)
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper proposes GRODE-RTL, a method for providing structured context to an LLM performing RTL repair. Given a suspicious signal, it extracts the signal’s fan-in cone from a synthesized netlist, maps the relevant logic back to source statements, and presents a compact dependency view to the agent. The evaluation covers 30 injected faults and three LLMs. The results show improvements over source-level exploration and the SeqFeed baseline, although most of the improvement in the ablation comes from selecting relevant source code.
Critical strengths
-
The paper addresses a practical problem: code responsible for an incorrect signal can be difficult to find through source-level search, especially when related statements are far apart.
-
Mapping circuit dependencies back to editable source locations makes the representation useful for repair. The approach can also be used with existing LLMs without additional training.
-
The improvement over source-level exploration appears across all three evaluated models. The representation ablations and per-case outcomes provide useful information for understanding the results.
Critical weaknesses
I will summarize the main weaknesses and explain what I meant below.
W1. The technical contribution beyond established backward slicing and cone-of-influence analysis is not sufficiently clear.
W2. The evidence for the added value of dependency organization and compression is limited.
W3. The evaluation uses a small, restricted set of injected faults and assumes a useful anchor is already available.
W4. Some descriptions of the experimental protocol need clarification, particularly regarding the information and exploration budget available to each baseline.
Detailed comments:
-
My main concern is what the method adds beyond using established program-analysis techniques to select context for an LLM. Starting from a suspicious signal and tracing backward through its dependencies is a familiar approach to narrowing the search space for debugging, error propagation analysis, taint analysis, etc. The paper also acknowledges that cone-of-influence reduction is standard.
-
My understanding of the "representation", which was the main contribution of the paper, is the way the selected code and its dependencies are presented to the LLM. To demonstrate that such representation actually achieves better performance, it is not clear to me whether the dependency order, relationships, and other factors make sense in the first place. Why does providing such information to LLMs help?
-
In my opinion, the method’s usefulness depends on obtaining an anchor whose dependency region contains the actual fault. The paper describes the anchor as the first upstream divergent sequential register, but provides limited detail on how this is determined. I would like to see a clearer description of anchor selection and some assessment of sensitivity to imperfect anchors.
-
Table 3 seems to suggest that the benefits of the compression are quite limited.
Questions for authors
-
Can you provide evidence (at least some intuition) that the proposed representation could affect the debugging success rate
-
The evidence for the benefit of the compact representation is currently limited to an ablation on one model and 30 faults, where it improves over the cone-only configuration by one successful repair. Could the authors extend this comparison to the other two evaluated models and report results across multiple runs?
Dear Reviewer,
Thank you again for your time and constructive feedback on our manuscript. As the ICLR rebuttal period is approaching its end in 3 days, we wanted to kindly follow up to ensure our previous responses have adequately addressed your concerns.
If you have any further questions or require any additional experiments, please feel free to let us know at your earliest convenience so that we have sufficient time to conduct them before the deadline. We truly appreciate your time and consideration.
Best regards, The Authors