- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper investigates the limitations of world-model-driven predictive safety guardrails through three research questions. First, it examines how different levels of prediction involvement affect agent safety, task completion, and inference cost. Second, it evaluates the reliability of world-model predictions across different prediction horizons, policy information settings, and prediction modes. Third, it investigates whether predictive information is faithfully incorporated into downstream action decisions through controlled feedback perturbations and sandbox-based verification. Experiments across multiple models reveal that stronger prediction involvement does not consistently improve safety, long-horizon risk prediction remains unreliable, and safety improvements do not necessarily reflect faithful use of predictive information. These findings highlight a prediction-to-action gap in existing world-model-driven predictive safety mechanisms.
Critical strengths
- The paper organizes its investigation around three logically connected research questions, progressing from the effectiveness of prediction integration to prediction reliability and decision faithfulness. This decomposition provides a valuable perspective for understanding the limitations of predictive safety guardrails beyond conventional end-to-end performance evaluation.
Critical weaknesses
- Limited isolation of decision faithfulness from execution control. The decision faithfulness analysis in Section 5 provides interesting observations, but some conclusions require more careful interpretation. As specified in Appendix D, Table 10, several feedback perturbations retain the original binary risk judgment, allowing the execution gate to continue operating on unchanged risk signals. Therefore, the safety improvements observed under corrupted or irrelevant explanatory content (Figure 3) may reflect the continued effectiveness of the gating mechanism instead of unfaithful use of predictive information. The experiments would benefit from a clearer distinction between reliance on risk judgments and reliance on their semantic explanations.
Questions for authors
- Regarding RQ3, how do the authors distinguish the effect of corrupted explanatory content from that of the unchanged binary risk judgments leveraged by the execution gate? Could the authors further clarify what aspects of decision faithfulness are established by the current perturbation experiments?
- Regarding RQ2, how do the sample sizes and risk-label distributions vary across prediction horizons? Could the authors provide additional statistics or uncertainty estimates to support the conclusion that prediction reliability degrades with longer horizons?
3 comments
Response to Reviewer Tj6H(2/2)
Q2: Sample composition and prediction reliability across horizons
We thank the reviewer for highlighting the importance of horizon-specific sample statistics. We now report the number and prevalence of risk-positive instances at each horizon.
Table 2. Risk-positive samples / valid labeled samples (prevalence).
| Horizon | HA | IS-Bench |
|---|---|---|
| h1 | 67/182 (36.81%) | 7/160 (4.38%) |
| h2 | 74/182 (40.66%) | 23/160 (14.38%) |
| h3 | 75/182 (41.21%) | 18/137 (13.14%) |
| h4 | 68/182 (37.36%) | — |
| h5 | 63/182 (34.62%) | — |
| h6 | 72/182 (39.56%) | — |
HA maintains 182 labeled windows across all horizons, with relatively stable risk prevalence (34.62%–41.21%). IS-Bench exhibits greater variation in both sample size and risk prevalence, which warrants caution when interpreting individual horizon comparisons.
To further examine the horizon effect, we conduct supplied-action-sequence prediction experiments with fixed prediction origins. By explicitly providing future actions, this setting separates the difficulty of anticipating future actions from assessing their consequences and associated risks.
Table 3. Supplied-action-sequence prediction: HA · Full.
| WM | Metric | h1 | h2 | h3 | h4 | h5 | h6 |
|---|---|---|---|---|---|---|---|
| Gemini-3.7-Flash | Accuracy | 0.74 | 0.71 | 0.75 | 0.75 | 0.76 | 0.69 |
| Risk precision | 0.97 | 0.90 | 0.91 | 0.91 | 0.93 | 0.82 | |
| Risk recall | 0.36 | 0.29 | 0.30 | 0.34 | 0.27 | 0.23 | |
| Risk consistency | 0.20 | 0.23 | 0.25 | 0.28 | 0.19 | 0.14 | |
| GLM-5.3-Flash | Accuracy | 0.73 | 0.71 | 0.69 | 0.73 | 0.74 | 0.68 |
| Risk precision | 0.93 | 0.90 | 0.78 | 0.94 | 0.92 | 0.71 | |
| Risk recall | 0.35 | 0.29 | 0.24 | 0.28 | 0.23 | 0.25 | |
| Risk consistency | 0.19 | 0.25 | 0.16 | 0.14 | 0.13 | 0.14 |
Table 4. Supplied-action-sequence prediction: HA · Train-derived.
| WM | Metric | h1 | h2 | h3 | h4 | h5 | h6 |
|---|---|---|---|---|---|---|---|
| Gemini-3.7-Flash | Accuracy | 0.66 | 0.64 | 0.66 | 0.65 | 0.67 | 0.65 |
| Risk precision | 0.78 | 0.90 | 0.75 | 0.78 | 0.77 | 0.58 | |
| Risk recall | 0.12 | 0.17 | 0.15 | 0.11 | 0.13 | 0.16 | |
| Risk consistency | 0.08 | 0.05 | 0.05 | 0.06 | 0.07 | 0.05 | |
| GLM-5.3-Flash | Accuracy | 0.63 | 0.63 | 0.64 | 0.64 | 0.69 | 0.66 |
| Risk precision | 0.80 | 0.72 | 0.60 | 0.61 | 0.69 | 0.50 | |
| Risk recall | 0.21 | 0.12 | 0.15 | 0.18 | 0.16 | 0.08 | |
| Risk consistency | 0.02 | 0.04 | 0.03 | 0.07 | 0.08 | 0.07 |
Table 5. Supplied-action-sequence prediction: IS-Bench · Full.
| WM | Metric | h1 | h2 | h3 |
|---|---|---|---|---|
| GPT-5.6-Terra | Accuracy | 0.82 | 0.64 | 0.73 |
| Risk precision | 0.24 | 0.31 | 0.33 | |
| Risk recall | 1.00 | 0.78 | 0.89 | |
| Risk consistency | 0.32 | 0.29 | 0.24 | |
| GLM-5.3-Flash | Accuracy | 0.87 | 0.69 | 0.71 |
| Risk precision | 0.30 | 0.33 | 0.30 | |
| Risk recall | 1.00 | 0.65 | 0.78 | |
| Risk consistency | 0.30 | 0.28 | 0.28 |
Table 6. Supplied-action-sequence prediction: IS-Bench · Train-derived.
| WM | Metric | h1 | h2 | h3 |
|---|---|---|---|---|
| GPT-5.6-Terra | Accuracy | 0.87 | 0.71 | 0.75 |
| Risk precision | 0.29 | 0.20 | 0.21 | |
| Risk recall | 0.67 | 0.48 | 0.33 | |
| Risk consistency | 0.18 | 0.20 | 0.13 | |
| GLM-5.3-Flash | Accuracy | 0.92 | 0.79 | 0.80 |
| Risk precision | 0.27 | 0.39 | 0.29 | |
| Risk recall | 0.50 | 0.39 | 0.33 | |
| Risk consistency | 0.21 | 0.22 | 0.12 |
These findings provide additional evidence that near-term risk detection does not reliably generalize to longer horizons, even when future actions are supplied. While horizon-specific label distributions and limited risk-positive samples warrant caution, the additional results support our original conclusion regarding the limitations of long-horizon predictive safety.
Reply to Authors
Thank you for your detailed reply. Although some typos appear in the response of Q1, the response basically addresses my concerns. I will keep the score to 3.
Response to Reviewer Tj6H(1/2)
We thank the reviewer for the positive assessment of our study and the constructive suggestions. We have conducted additional experiments to clarify the respective roles of execution gating and predictive content (RQ3), and to examine sample composition and prediction reliability across horizons (RQ2).
W1 & Q1: Distinguishing decision faithfulness from execution gating
We agree that preserving binary risk judgments allows the execution gate to remain effective even when explanatory content is corrupted. As specified in Appendix D (Table 10), the original perturbations assess the sensitivity to explanatory content under largely unchanged risk-based gating, rather than fully isolating semantic reliance.
To address this concern, we introduce four additional controls that separate risk judgments from predictive content. In particular,
null_emptyandnull_realboth remove the risk label and numerical score, disable the hard veto, and retain the same pre-execution review procedure. They differ only in whether the original predictive content is provided. We additionally evaluaterandom_emptyandrandom_real, which use randomized risk labels with and without predictive content.Table 1. Supplementary results (SAR / TCR).
null_empty)null_real)random_empty)random_real)These results help disentangle decision faithfulness from execution gating. Safety improvements persist even without predictive content or hard gating, indicating that pre-execution review itself contributes to safety. Providing valid predictive content yields additional, model-dependent benefits, while randomized risk judgments can produce high safety scores at the expense of task completion. Together, these findings show that safety gains can arise from mechanisms beyond faithful use of predictive content, and therefore should not be taken as direct evidence of decision faithfulness.