- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 1
Read review
Paper summary
The paper introduces FlowHarm, a benchmark of 200 software engineering tasks for studying the propagation of harmful objectives in workflow-based coding agents. The tasks are synthesized by LLMs guided by task metadata (task specifications). Harmful workflows are constructed by injecting a harmful objective into each task’s workflow, with variants that include a plausible rationale for the objective. The paper also introduces process-level metrics to quantify the extent of harmful objective propagation. Evaluations of static and dynamic workflows across a collection of (mostly small-sized) models show that harmful objectives survive workflow refinement and propagate through downstream dependencies.
Critical strengths
- The paper is generally clear and easy to follow, with logically organized sections and additional details documented in the appendix.
- The FlowHarm benchmark and its metrics for studying harmful objective propagation could be quite useful to the AI safety community, although I am not an expert in this area.
Critical weaknesses
-
Task validity does not establish necessity. All 200 tasks are synthesized by LLMs, and human verification assesses their validity rather than whether the injected harmful objectives would naturally arise at the specified points in the workflows. Would engineers introduce similar objectives during normal software engineering work? Without evidence supporting this assumption, the benchmark may be better interpreted as a worst-case stress test, and its relevance to realistic software engineering workflows remains unclear.
-
The connection to real-world impact is insufficiently established. I appreciate that synthetic tasks facilitate controlled studies, but I would be better if authors can provide a subset adapted from real-world software engineering tasks containing naturally occurring, human engineer-authored harmful objectives. Such a subset would help establish whether the findings extend beyond hypothetical scenarios.
-
The evaluation is limited and unrealistic. As this is a benchmark paper, I would appreciate evaluations of at least three or more frontier models, such as GPT-6, Opus 5.5, or Fable 5.1 (not necessarily these models; but the latest models released before the submission due date). The broader implications of the findings remain limited if they hold only for small models such as Qwen3.5-9B. Evaluating frontier models would help substantiate the paper’s claims of relevance to practical software engineering workflows beyond toy settings.
-
The paper does not evaluate mitigation strategies. Evaluating a few existing defense mechanisms would strengthen the study, particularly because the harmful objectives are deliberately injected rather than naturally occurring.
Questions for authors
Please see above