- Rating
- 4
- Confidence
- 4
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
Trident is a RLVR-based approach to train automated red agents in ACD games. The approach uses a trainable planning agent, which outputs natural language descriptions of strategies, which are then translated into code by a frozen coding agent. The output is then used to generate traces in the simulators, which provides rewards to update the planner’s parameters. This method successfully degrades all blue agents it was trained against by wide margins, more successfully than the other works it was compared to, and contains thorough ablation studies in the supplemental material.
Critical strengths
- Research on automated attacks in simulated cyber environments is sparse. The findings of this work could in turn be used to train even better blue defenders, and enable further adaptive defense agents
- The experiments included a PPO-based agent to serve as a baseline to distinguish what makes the LLM-based approach work. I was surprised how poorly the DRL red agents did, even compared to weaker LLM-based attacks.
- In addition to the experiments, the authors released many traces of their games which appear diverse Having the only trainable parameter be the planner, and executing the model it generates is highly efficient, and makes LLM-based attackers more feasible than prior works which directly use LLM actions as policies.
- The paper is well-written and easy to follow, and contains many useful experiments beyond the main contribution
Critical weaknesses
- Table 1 should be organized to emphasize better which benchmarks are static datasets, evaluation tasks for LLMs, and multi-step environments. As it is now, the second row in particular feels very mixed
- The attack is trained on each defense separately. It would be interesting to see if it learns a general policy if defenders were selected randomly and to see how this generalized policy affects each of the blue agents individually
- As noted in the limitations section, this approach requires expert-crafted prompts to describe the actions and goals for each environment.
- Minor: King et al. is cited as a preprint, but it has since been published (DSN 2026)
Questions for authors
- Is there potential for data leakage about the network configuration? The baseline LLMs used were published in 2024, as was CAGE-4 and CyberWheel. Does this affect the POMDP assumption if it’s possible information about these environments was in the LLMs’ original training data?
- How generalizable are the trained attackers? E.g. can the attacker trained to exploit King et al. also degrade Singh et al. and Lyu et al.?
Revised Manuscript Uploaded and Appreciation for the Review Process
Dear Area Chairs and Reviewers,
We have uploaded the revised manuscript incorporating our responses to the reviewers' comments.
All revisions are highlighted in blue for ease of review.
We sincerely appreciate your time and consideration throughout the review process.
(minor) The definition of πc is added to improve the clarity of our manuscript (L157). (Updated on Nov 16)
Added the Random Search baseline over the CAGE 4 strategy space (Table 2, Section 4.1–4.2, and Appendix E.2, Eqs. 7–8) as requested by Reviewer M4hL. (Update on Nov 17)