- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper investigates how LLM-based cybersecurity agents can learn from failed interactions before discovering their first complete solution. It proposes Bi-PGSD, a framework combining execution-verified forward effects, backward goal reasoning, local reward-based policy updates, cross-task experience reuse, and progressive scaffold withdrawal. Experiments across several cybersecurity benchmarks demonstrate improvements in first-success discovery, experience transfer, and scaffold-free task completion.
Critical strengths
-
The paper addresses an important practical challenge in training LLM agents under sparse terminal rewards. Extracting verifiable local progress from unsuccessful trajectories is a meaningful direction.
-
The evaluation spans multiple cybersecurity benchmarks and model families, with extensive mechanism ablations and cost accounting. The results provide useful evidence that verified local experience and goal-directed exploration can improve agent performance.
Critical weaknesses
-
The proposed framework mainly integrates established techniques, including hindsight-style goal relabeling, verified experience memory, backward planning, local policy optimization with RLOO, and curriculum-based self-training. While their integration is reasonable, the paper offers limited methodological innovation beyond combining these existing components.
-
On Cybench, filtered local SFT already achieves 41.4% success, compared with 44.7% for the full framework. This relatively small additional gain raises questions about whether the proposed local RL machinery is necessary, particularly given the additional training and verification complexity.
-
Although the paper highlights a 62.5% versus 50.0% first-success result on Cybench-Z, it also reports that the average improvement across six trainable-policy candidates on CTF tasks is only 1.4 percentage points under matched end-to-end budgets. Maintenance accounts for 55.2% of the total cost, and the method performs worse at smaller budgets. The practical cost-effectiveness of the full framework therefore remains uncertain.
-
The method improves performance on several tasks, but the gains are not uniform. In particular, success-assisted InterCode-CTF results are consistently worse than strong-summary retrieval for multiple models. These negative results suggest that the framework is not universally beneficial, and the conditions under which its additional complexity is worthwhile require clearer characterization.
Questions for authors
-
Can the authors provide a more direct comparison against a simpler framework combining verified failed-fragment retrieval, local SFT or RLOO, and standard task scheduling, under identical computational budgets?
-
Given the relatively small improvement over filtered local SFT, what evidence establishes that local RL is essential rather than merely an incremental addition?
-
How do the authors explain the consistent performance degradation on success-assisted InterCode-CTF, and can they characterize when Bi-PGSD is expected to outperform simpler retrieval-based methods?
-
How robust is the framework when reliable local checkers, environment resets, or reproducible intermediate states are unavailable?