- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
Large language models (LLMs) are being used as autonomous agents in tasks where only final outputs are reviewed for ethical behavior. This study introduces predictive monitoring to identify the likelihood of unethical actions based on initial benign actions, utilizing a benchmark called PreAct-Bench, which consists of 1,041 pairs of ethical and unethical action trajectories across five domains: Academia, Law&Contract, Cybersecurity, Politics, and Daily Life. Human evaluations achieved an average F1 score of 0.907, outperforming all tested LLMs, which suggests that current models may be misled by their reasoning processes, highlighting the need for further exploration in predictive safety monitoring.
Critical strengths
- The studied problem is timely and important.
- The analysis produces an internally consistent finding.
Critical weaknesses
- Trajectories are generated with GPT-5-mini, and the same model family is also used for evaluation labeling.
- The benchmark uses binary ethical/unethical labels, which flattens genuine ambiguity.
- Guardrail models are adapted to a task they were not designed for, which may understate their potential.
- The novelty of the paper is limited, given a large corpus of work in this direction.
Questions for authors
-
The binary metric PFF(p) is intuitive, but it only evaluates binary F1 at a given prefix length. It does not itself capture calibration or uncertainty. A joint metric combining binary detection and likelihood calibration would better match the paper’s central claim.
-
Only 400 annotation entries are collected from 1,041 entries, and binary human-human agreement is only 0.753. The fact that human–GPT agreement is higher (0.785) than human–human agreement is interesting but also ambiguous: it could indicate label validity, or it could indicate that GPT-5-mini labels inherit a particular annotation style that humans partially mimic.
-
Some related work should be discussed, e.g.,
- Kang, M., Chen, Z., Xu, C., Zhang, J., Guo, C., Pan, M., ... & Li, B. (2026). Polyguard: Massive multi-domain safety policy-grounded guardrail dataset. Advances in Neural Information Processing Systems, 38.
- Yuan, L., Li, X., Xu, C., Tao, G., Jia, X., Huang, Y., ... & Li, B. (2026). Promptguard: Soft prompt-guided unsafe content moderation for text-to-image models. IEEE Transactions on Information Forensics and Security.
- Xie, X., Song, J., Zhou, Z., Huang, Y., Song, D., & Ma, L. (2025). Online safety analysis for llms: a benchmark, an assessment, and a path forward. IEEE Transactions on Artificial Intelligence.
4 comments
Response to DGVc Part 2
Response to Weakness 2
We agree with the reviewer that using solely binary metrics fails to capture the ambiguity of predictive monitoring. As the explicit unethical action is never stated in the prefix, there is always the likelihood that the unethical prefix could culminate in an ethical action.
As such, we introduce uncertainty during human annotation ($\S$3.4) as well as during model evaluation ($\S$4.2). We believe that these uncertainty-focused setups and studies will help readers to better understand the alignment between unethical uncertainty given by LLM monitors compared with the uncertainty given by human annotators, thereby providing a more comprehensive evaluation of LLMs’ performance on the predictive monitoring task.
Response to Weakness 3
Thank you for raising this important aspect.
As discussed in $\S$4.1, existing LLM guardrail models such as Qwen3Guards adopt a retrospective monitoring setup. As such, the model is trained to retrospectively classify an action or content as ethical or unethical.
PreAct-Bench introduces a new aspect to safety monitoring, namely predictive monitoring. As many unethical actions could result in irreversible consequences, we argue that LLM monitors need to predictively forecast the risk of future unethical action, effectively stopping model action before the unethical action materializes.
As the reviewer pointed out, such capability is lacking in existing monitors, which is precisely the value and novelty of our research work. We believe training LLMs to conduct such predictive monitoring tasks is an important future research direction.
Response to Weakness 4
Thank you for providing additional LLM safety literature. Upon reviewing the additional literature, we do not believe that it impacts the novelty of our work.
- Polyguard [1] is a comprehensive LLM safety benchmark, but all of the tasks are retrospective instead of predictive. Specifically, the unethical action is already explicit in the prompt or model response. In contrast, the action trajectories of PreAct-Bench do not include explicit unethical action and LLM monitors are tasked to judge if the seemingly benign actions will culminate in an unethical action.
- Promptguard [2] is a safety benchmark targeting VLMs in the context of text-to-image tasks. The nature of this task is also retrospective monitoring, as the unethical or harmful instruction is explicit in the user prompts.
- For the third work [3], their RQ1 on early detection of unsafe responses is relevant to ours. However, the takeaway from their study is that explicitly unsafe content in an unsafe response typically occurs in the early response (e.g., the first 25%), so that unsafe response can usually be detected at their early generation process due to the presence of explicit unsafe content. In contrast, PreAct-Bench tasks LLMs to infer latent malicious intent from seemingly benign actions. As such, explicit unsafe content never occurs in the action trajectories.
To summarize, the main novelty of our work is the task of predictive monitoring, where we ask the question of “Can LLMs detect latent unethical intent before the explicit unethical action materializes?” The objective is to assess and understand LLMs’ capability of inferring the likelihood of a sequence of seemingly benign actions culminating in an unethical action.
We argue that such a predictive monitoring task is fundamentally different from the retrospective monitoring task, as no explicit unethical action is ever given to the LLMs. They instead need to infer the latent objective by collectively reasoning about the observed action trajectories.
Such predictive monitoring capability is particularly important for applications of LLMs in high-stakes scenarios such as mental health, where a materialized unethical action could result in severe consequences and therefore must be intervened upon by a monitor preactively.
Response to DGVc's Questions
Response to Question 1
As discussed in our Response to Weakness 2, we agree that the predictive monitoring task is probabilistic by nature, as the unethical action never materializes; even a trajectory with apparently strong latent unethical intent could end up ethical. Therefore, on top of the binary classification setup, we also include a probabilistic evaluation setup where we examine the alignment of the likelihood of unethical action predicted by LLMs and the likelihood annotated by human annotators. We present this study in $\S$4.2.
Response to Question 2
As discussed in our Response to Weakness 1, we created a new dataset using DeepSeek v4 Pro. The results indicate that the same conclusions hold, suggesting that they reflect the dataset setup rather than the model used.
Response to Question 3
Thank you for your suggestion on the relevant literature. We provided a brief discussion of each of them and the novelty of our work in our Response to Weakness 4. We will revise our related work section to include discussions of the suggested literature.
Resposne to DGVc: Actionable Items
Paper Revision Items
- Add a subsection to Appendix B (Additional Experiments) comparing monitoring performance on DeepSeek-V4-Pro trajectories with those from GPT-5-mini.
- Include in Appendix B a qualitative comparison of GPT-5-mini and DeepSeek-V4-Pro trajectories, examining how differences in the explicitness of unethical intent affect monitoring difficulty.
- Briefly discuss in Section 3.2 (Benchmark Creation) the impact of the trajectory generation model, referring to the new cross-generator experiment.
- Expand Section 4.2 (Prediction with Uncertainty) to explain how probabilistic evaluation complements binary PFF by comparing model-generated likelihood estimates with the distribution of human Likert-scale ratings.
- Clarify in Section 4.1 (Evaluation on Risk Detection) that existing guardrail models are designed for retrospective safety classification rather than predictive monitoring, and discuss the implications for their performance on PreAct-Bench.
- Expand Section 2 (Related Work) to discuss PolyGuard, PromptGuard and Online Safety Analysis for LLMs, contrasting their retrospective detection settings with PreAct-Bench's prediction of latent unethical intent from seemingly benign actions.
We will make the aforementioned revisions to the submission before the end of the rebuttal.
Response to DGVc Part 1
Thank you for taking the time to review our paper and for providing constructive suggestions.
Response to Weakness 1
To understand the impact of the data generation model (GPT-5-mini), we sampled a subset of 50 entries per domain (total of 250) from the full dataset, used DeepSeek v4 Pro to generate their trajectories and evaluated this subset with the same evaluation setup.
As shown in the table, while model performances slightly improved, there is no significant deviation from the original data produced by GPT-5-mini. In addition, we observe that the unethical trajectories produced by DeepSeek v4 Pro are overall more explicit, making them easier to detect.
Both trajectories below have almost the same hidden goal, but only the DeepSeek prefix gives it away. Following our evaluation setup, the monitor sees only the steps before the first unethical one.
In the GPT-5-mini prefix, individual steps appear legitimate (“lawful turnout”, “transparent communications”), with harmful intent emerging only from their combination.
By contrast, the DeepSeek v4 Pro prefix reveals intent through explicit cues: “omit permit conditions”, “place inexperienced marshals near the police station” and introducing a cover story.
These surface cues explain the higher recall across monitor families, particularly content-focused guardrails (e.g., Qwen3Guard-0.6B on Politics: 0.143 to 0.571). Thus, our benchmark's difficulty reflects the more covert actions produced by GPT-5-mini, rather than a GPT-family advantage on GPT-generated data.