acceptodds
Withdrawn from ICLR 2027

SpecterRTL: LLM-Orchestrated Hardware Trojan Evolution for Automated RTL Vulnerability Injection

Withdrawn from ICLR 2027, so left out of the paper lists and the map.

Abstract

LLM-assisted hardware design creates a new attack surface for automated RTL red teaming. Prior approaches are constrained by limited autonomy, weak adversarial evolution, and hardware security data scarcity. We present **SpecterRTL**, an end-to-end framework for autonomous Hardware Trojan construction and evolution. SpecterRTL establishes a self-sustaining closed loop where design-aware vulnerability mining extracts targeted attack seeds that fuel adversarial evolution to create previously unseen threat variants. Continuously absorbing newly evolved attacks into foundational threat knowledge mitigates data scarcity and drives increasingly sophisticated exploration. Across 137 RTL modules spanning 24 categories, SpecterRTL produces an average of 5.27 verified Trojan seeds per module and raises the proportion of high-risk instances from 12.5% to 92.3% on representative evolution cases. On the clean designs shared with the current SOTA RTL red-teaming benchmark, SpecterRTL produces **42.4** more verified Hardware Trojan instances and a **35** higher successful evasion count across three representative detectors, advancing RTL security evaluation from template-constrained generation to self-sustaining threat evolution.

Primary area:
applications to computer vision, audio, language, and other modalities
Keywords:
Hardware Trojan, Large Language Models, Red Teaming, Vulnerability Discovery, Adversarial Evolution

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_iXfEOpenReview ↗
Rating
1
Confidence
4
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper proposes SpecterRTL to generate hardware Trojans (HTs) in RTL. The approach is a multi-agent workflow: an LLM front-end (DAVE) mines design-specific vulnerabilities from a CWE/CVE knowledge graph and writes seed Trojans, and a genetic search (PHAGE) then evolves those seeds using LLMs as its variation operators (mutate/deepen/recombine), scoring each solution by a weighted sum of a severity score (weight 0.7) and detector evasion (0.3).

Critical strengths
  • HT generation is useful for the hardware community, as it might help train better models to detect zero-day HTs.
  • The evolutive part of the multi agent workflow has potential to generate unpredicted designs that offer richer HT distributions.
Critical weaknesses
  • The central empirical claim (evasion) is a volume artifact, not a real result. The headline 35×35\times/29.5×29.5\times more evasions come entirely from generating far more candidates, not from producing stealthier Trojans. The paper's own rate columns show evasion getting \emph{worse} where the method is applied: the evasion rate falls from 54.5%54.5\% to 43.4%43.4\% after evolution (Table3a), and is lower than the baseline on the shared designs (23.6%23.6\% theirs vs 28.6%28.6\% for GHOST, Table3b). Section5.3 nonetheless claims a frontier with ``stronger evasion,'' which directly contradicts these numbers. Because detector evasion is one of the two objectives the whole framework is built to optimize, a headline that inverts the sign of the underlying rate is not a presentation nit; it undermines the paper's core quantitative argument.
  • The work is LLM-driven HT generation, a space that is already well-populated in the literature, so the ``first end-to-end autonomous'' claim does not hold. The evaluation then measures success as evading HT detectors trained on scarce, unbalanced datasets, a limitation long known in that community, so results establish little w.r.t evading detection. Relatedly, the optimization formulation is decorative and internally inconsistent: the task is posed as a constrained multi-objective problem but it is never solved; it is scalarized to a single weighted sum for selection, while Section5.1 reports Pareto hypervolume as a multi-objective quality measure. Either commit to multi-objective optimization or present the scalarized heuristic plainly.
Questions for authors
  • Your headline gains are reported as evasion counts, yet the evasion rate drops after evolution and is below the baseline on the shared designs. How do you reconcile this with the claim in Section~5.3 of a frontier with "stronger evasion,'' and why should counts rather than rates be the figure of merit?
1 comment

Thank you for asking us to distinguish corpus scale, per-instance evasion, and the role of the optimization formulation.

Counts, rates, and the frontier comparison. Counts measure corpus output, while rates measure the fraction satisfying the detector criterion. On the shared designs, SpecterRTL produces 593 checked instances versus the 14 released by GHOST, including 140 versus 4 instances not flagged by the panel. GHOST's 28.6% rate rests on just 4/14 instances. Changing a single outcome shifts that rate by 7.14 percentage points, exceeding its approximately 5-point difference from SpecterRTL's 23.6%. Such a small baseline does not support a reliable performance ranking from this rate difference, nor does it negate the value of producing a much larger corpus of checked instances, including substantially more detector evasions. Section 5.3 compares non-dominated points against OpenEvolve, whereas Table 3 compares aggregate rates against initial seeds or GHOST. These different comparisons need not move in the same direction. “Stronger evasion” should therefore be tied to the specific comparison it describes, rather than presented as an overall improvement in evasion probability.

Contribution relative to prior generation methods. The contribution is the integration of design-aware seed construction, population-based expansion, and reuse of accumulated examples and feedback. The evidence includes coverage of 137 modules and the ablation yielding 722 checked seeds with the full workflow, compared with 139 without the knowledge graph and 220 with a monolithic agent. This identifies a concrete contribution without relying on a universal “first autonomous” claim. Detector responses are feedback and evaluation observations within this workflow, not a general certification of stealth against arbitrary defenses.

Scalarization and hypervolume. PHAGE uses a weighted scalar fitness to guide heuristic search subject to the design checks. Weighted-sum scalarization expresses a preference over multiple objectives, while hypervolume measures the region dominated by the resulting point set. The latter can be computed regardless of how those points were selected, so the two are mathematically compatible. The appropriate claim is heuristic search under a stated preference, with hypervolume describing the attained set. It is not a claim of recovering the complete Pareto frontier or guaranteeing global optimality.

Reviewer_uJQoOpenReview ↗
Rating
3
Confidence
2
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper studies automated insertion of hardware Trojans into RTL designs. The motivation is LLM-assisted hardware design: a compromised copilot, a malicious IP vendor or a poisoned training corpus could inject Trojans that pass standard verification. The authors argue that existing LLM-based Trojan generators depend on templates, do not adapt to detectors, and draw on scarce and homogeneous data.

The proposed framework, SpecterRTL, has two stages, both run with GPT-5.2. DAVE uses a team of agents and a CWE/CVE knowledge graph to write design-specific Trojan seeds, and keeps only those that pass linting, the original testbench and synthesis. PHAGE then evolves the seeds with three LLM operators (mutate, deepen, recombine), scoring offspring with an LLM-judged severity score and three Trojan detectors. On three designs shared with GHOST, the paper reports 593 verified instances and 140 detector evasions, versus the 14 cases and 4 evasions GHOST released.

Critical strengths
  • Executable verification. Every candidate must pass Verilator lint, the original read-only testbench and Yosys synthesis (App. I.2), so acceptance does not rely on the model's own claim.
  • Broad coverage. Mining is run on all 137 CORE modules across 24 categories (Table 6).
  • Clear method. The three operators are clearly defined, with a worked example on real RTL (Fig. 3, App. G). The ablation shows that the knowledge graph helps: 722 verified seeds with it vs 139 without (Table 4).
Critical weaknesses

W1. "Verified" does not show that the inserted logic is a working Trojan.

  • Acceptance (Eq. 1, App. I.2) means: lint passes, the original testbench passes, and synthesis exits cleanly. In the paper's words, a verified instance "disrupts no behavior that the original testbench observes, and remains realizable in hardware".
  • These checks show that the change does not break tested behavior. They do not show that the trigger can be reached, or that the payload does anything when it fires.
    • A change whose trigger can never be reached, or whose payload has no effect, passes all three checks. So does an unmodified copy of the clean design.
    • Synthesis is checked only by its exit status. Logic that can never affect an output may be removed during synthesis, so the netlist may carry no Trojan at all. The paper does not check whether the Trojan logic survives synthesis.

W2. Severity is measured with the same signals the search optimizes.

  • f_R is V-VIS, an LLM rubric score (App. F), and "high risk" means f_R ≥ 0.7.
  • The paper does not say which model(s) form the evaluator ensemble (Table 2: "LLM ensemble"). All agents run on GPT-5.2 (App. I.7).
  • The search is told the rubric and sees the scores:
    • the Deepen handbook "states the four dimensions of Appendix F as targets" (App. I.6);
    • the operator prompts receive the "numerical scores and reasoning text" (Sec. 5.3).
  • The rise from 12.5% to 92.3% high-risk (Table 3a) is therefore the search's own objective, measured by the same judge. It does not show that the Trojans are harder for a human reviewer or a verification engineer to find.
Questions for authors

I do not have a strong background in hardware security, so the feedback may not be that helpful for improvement. If the author think any concern listed above worth responding, feel free to do so.

1 comment

Thank you for highlighting the executable checks, broad coverage, and knowledge-graph ablation. We agree that the interpretation of verification and scoring deserves clarification.

W1. What “verified” establishes. An unchanged clean design would pass the three design checks in isolation. However, the supplementary implementation also applies an RTL-difference check that rejects candidates without detected logic changes. This excludes the unchanged-copy example, but structural differences do not establish trigger reachability or payload survival after synthesis. The experimentally supported contribution is automated construction of RTL corpora containing HT logic and passing specified design checks. These corpora provide material for code-level security research. Describing every accepted instance as a behaviorally validated Trojan would go beyond the reported evidence.

W2. Optimizing a known rubric. The evaluator is GPT-5.2, using five judgments per instance and per-dimension median aggregation as described in Appendix F. Giving the generator the rubric, scores, and explanations is an intentional part of directed search. That access does not itself invalidate the method. The relevant distinction is between improving the stated objective and independently validating what it represents. The 12.5% to 92.3% result demonstrates the former. V-VIS characterizes code properties associated with discovery difficulty, so its threshold is better described as “high-scoring” than as independently established severity. We agree that the score increase alone cannot establish greater difficulty for human reviewers or verification engineers.

Reviewer_8Lh9OpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
3
Soundness
1
Read review
Paper summary

This paper presents SpecterRTL, which is an LLM-based framework for generating and evolving hardware Trojan candidates at the RTL level. Its DAVE component uses a hardware vulnerability knowledge graph to identify design-specific injection opportunities, while PHAGE applies population-based mutation, deepening, and recombination guided by vulnerability scores and detector feedback. Generated instances are checked through linting, functional simulation, and synthesis, with a repair loop for unsuccessful candidates. The framework also stores previous instances and their evaluation results for subsequent retrieval.

Critical strengths
  1. The combination of DAVE and PHAGE is a meaningful systems contribution.
  2. The mining evaluation spans 137 modules and 24 functionality categories.
Critical weaknesses
  1. The validation establishes compatibility with the existing design flow, but does not sufficiently establish Trojan effectiveness. This paper defines verification through linting, the original functional testbench, and synthesis. These checks are useful, but passing them does not establish that the trigger is reachable under valid operating conditions, that activation produces the claimed malicious behavior, or that this behavior reaches an observable output or security-relevant state. Successful synthesis also does not establish that the malicious logic survives optimization, and passing regression tests after repair does not establish that repair preserved the intended Trojan mechanism.

  2. The evaluation does not sufficiently separate improvement on the search objectives from independently established threat quality and generalization. The representativeness of the evolution subset also needs clarification. Table 6 reports 300 high-risk seeds among 722 seeds overall, approximately 41.6%, whereas Table 3(a) starts from 14 among 112 seeds, or 12.5%.

Questions for authors
  1. What checks, beyond the original testbench, synthesis, and AST differences, establish reachable triggers and observable payload effects? How many reported instances have this evidence, and were the checks applied after repair and synthesis?
  2. Is there an independent assessment connecting V-VIS scores to expert judgments or measured detection or verification difficulty? Which evaluator model or models were used for the reported results?
  3. Are there results from detectors that were excluded from the evolutionary feedback loop? What are the evaluated detectors’ false-positive rates on the clean designs, and how are designs outside a detector’s intended scope handled, particularly for the FSM-specific SecRT-LLM?
  4. What operational criteria selected the 21 evolution cases, and were the full-corpus and subset statistics obtained from the same mining run and scoring configuration? What explains the difference between their initial high-risk fractions, and are repeated independent runs available to assess the stability of the reported improvements?
1 comment

Thank you for recognizing the systems contribution and the breadth of the mining evaluation. Your questions help distinguish the demonstrated results from the broader implications of the work.

Verification and the research contribution. SpecterRTL demonstrates automated construction and expansion of RTL corpora containing HT logic. The reported checks establish compatibility with the checked design flow. We agree that "verified Trojans" can imply more than this evidence establishes, particularly reachable activation and payload survival after synthesis. The reported evaluation contains no dedicated experiment validating those properties. The demonstrated contribution remains the construction of concrete RTL examples for studying injected logic across designs, whose value as code-level research material is distinct from certification of each example as a working hardware attack.

V-VIS and independent assessment. The evaluator is GPT-5.2, with five judgments per instance aggregated by the per-dimension median, as described in Appendix F. That appendix defines V-VIS through properties associated with the difficulty of uncovering an inserted modification, rather than the damage it causes. Providing this rubric and its feedback to the generator is directed optimization. The increase from 12.5% to 92.3% establishes progress under that rubric, not an independently measured increase in human review or verification difficulty. "High-scoring" is therefore more accurate than "high-risk". This study does not report an independent expert calibration of V-VIS.

Detector results and applicability. All three reported detectors participate in the feedback loop. The results characterize responses from that panel. On clean CORE designs, the false-positive rates were 14.17% (17/120) for GNN4TJ, 5.84% (8/137) for SecRT-LLM, and 6.57% (9/137) for BugWhisperer, calculated over successfully evaluated designs. No held-out detector experiment is reported. The released interface invokes enabled detectors without a separate design-applicability filter. In particular, a negative SecRT-LLM response outside its FSM-oriented scope should not be interpreted as defeating an applicable detector. The defensible conclusion concerns corpus generation under the configured feedback, rather than general resistance to hardware security analysis.

Case selection and the initial population. Evolution was limited to 21 cases because of its computational cost. Cases were chosen to cover different CORE categories, with preference for those whose seeds had lower initial scores and therefore more room for improvement. These were the cases actually run, not a favorable subset selected after evolving the full dataset. The evolution curves summarize variation across designs, rather than repeated independent runs of each design.

Table 6 reports 300/722 (41.6%) across all 137 modules, whereas Table 3(a) reports 14/112 (12.5%) for the 21 selected cases. These percentages describe different populations. The lower initial high-scoring fraction in the evolution subset is consistent with selecting cases that offer room for improvement, rather than reproducing the score distribution of the full corpus.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Discussion (0)

Sign in to comment.