acceptodds
Under review as a conference paper at ICLR 2027

Predictive Safety Guardrails Do Not Always Make LLM Agents Safer

Abstract

Large language model agents increasingly take consequential actions in digital and physical environments, where safety risks may emerge only after multiple interaction steps. This has motivated world-model-driven predictive safety guardrails, which anticipate future consequences before execution. However, existing evaluations largely focus on end-to-end safety outcomes, leaving unclear whether safety gains arise from reliable prediction and faithful use of predicted risks. In this work, we systematically diagnose the prediction-to-action pipeline along three dimensions: how world-model predictions are integrated into decision making, whether the predictions themselves are reliable, and whether their content is faithfully used. Across seven policy models and two safety environments, we find that stronger prediction involvement does not yield monotonic safety gains and can reduce task completion while substantially increasing inference cost. We further find that world-model predictions remain insufficiently reliable, especially over longer horizons, where safety prediction degrades substantially relative to near-term forecasting. Finally, we find that safety gains can persist even under corrupted predictions, while locally verified predictions still leave subsequent safety violations, showing that predictive content is not faithfully incorporated into the decision process. Together, these findings expose a fundamental prediction-to-action gap in current predictive guardrails: producing more or seemingly accurate foresight is insufficient unless that foresight is both reliable and faithfully translated into safer decisions.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, agent safety, world models, predictive guardrails, risk prediction, safety evaluation

Reviews

Among:
Shape:
1234this paper, Top 15%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.00 over 3 reviews, above 85% of the others.
Reviewer_Tj6HOpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

This paper investigates the limitations of world-model-driven predictive safety guardrails through three research questions. First, it examines how different levels of prediction involvement affect agent safety, task completion, and inference cost. Second, it evaluates the reliability of world-model predictions across different prediction horizons, policy information settings, and prediction modes. Third, it investigates whether predictive information is faithfully incorporated into downstream action decisions through controlled feedback perturbations and sandbox-based verification. Experiments across multiple models reveal that stronger prediction involvement does not consistently improve safety, long-horizon risk prediction remains unreliable, and safety improvements do not necessarily reflect faithful use of predictive information. These findings highlight a prediction-to-action gap in existing world-model-driven predictive safety mechanisms.

Critical strengths
  1. The paper organizes its investigation around three logically connected research questions, progressing from the effectiveness of prediction integration to prediction reliability and decision faithfulness. This decomposition provides a valuable perspective for understanding the limitations of predictive safety guardrails beyond conventional end-to-end performance evaluation.
Critical weaknesses
  1. Limited isolation of decision faithfulness from execution control. The decision faithfulness analysis in Section 5 provides interesting observations, but some conclusions require more careful interpretation. As specified in Appendix D, Table 10, several feedback perturbations retain the original binary risk judgment, allowing the execution gate to continue operating on unchanged risk signals. Therefore, the safety improvements observed under corrupted or irrelevant explanatory content (Figure 3) may reflect the continued effectiveness of the gating mechanism instead of unfaithful use of predictive information. The experiments would benefit from a clearer distinction between reliance on risk judgments and reliance on their semantic explanations.
Questions for authors
  1. Regarding RQ3, how do the authors distinguish the effect of corrupted explanatory content from that of the unchanged binary risk judgments leveraged by the execution gate? Could the authors further clarify what aspects of decision faithfulness are established by the current perturbation experiments?
  2. Regarding RQ2, how do the sample sizes and risk-label distributions vary across prediction horizons? Could the authors provide additional statistics or uncertainty estimates to support the conclusion that prediction reliability degrades with longer horizons?
3 comments
AuthorsOpenReview ↗

Response to Reviewer Tj6H(1/2)

We thank the reviewer for the positive assessment of our study and the constructive suggestions. We have conducted additional experiments to clarify the respective roles of execution gating and predictive content (RQ3), and to examine sample composition and prediction reliability across horizons (RQ2).

W1 & Q1: Distinguishing decision faithfulness from execution gating

We agree that preserving binary risk judgments allows the execution gate to remain effective even when explanatory content is corrupted. As specified in Appendix D (Table 10), the original perturbations assess the sensitivity to explanatory content under largely unchanged risk-based gating, rather than fully isolating semantic reliance.

To address this concern, we introduce four additional controls that separate risk judgments from predictive content. In particular, null_empty and null_real both remove the risk label and numerical score, disable the hard veto, and retain the same pre-execution review procedure. They differ only in whether the original predictive content is provided. We additionally evaluate random_empty and random_real, which use randomized risk labels with and without predictive content.

Table 1. Supplementary results (SAR / TCR).

ConditionQwen3.5-27B: SAR / TCRGLM-5.3-Flash: SAR / TCRGemini online group: SAR / TCR
No Prediction76.13 / 81.1264.41 / 85.6969.51 / 88.07
No hard veto + empty content (null_empty)88.33 / 74.0779.17 / 71.1377.89 / 86.05
No hard veto + original content (null_real)90.00 / 68.5086.64 / 74.7582.71 / 82.37
Random risk label + empty content (random_empty)97.73 / 34.0098.30 / 12.0798.19 / 34.87
Random risk label + original content (random_real)97.84 / 46.9798.21 / 10.5297.90 / 59.19

These results help disentangle decision faithfulness from execution gating. Safety improvements persist even without predictive content or hard gating, indicating that pre-execution review itself contributes to safety. Providing valid predictive content yields additional, model-dependent benefits, while randomized risk judgments can produce high safety scores at the expense of task completion. Together, these findings show that safety gains can arise from mechanisms beyond faithful use of predictive content, and therefore should not be taken as direct evidence of decision faithfulness.

AuthorsOpenReview ↗

Response to Reviewer Tj6H(2/2)

Q2: Sample composition and prediction reliability across horizons

We thank the reviewer for highlighting the importance of horizon-specific sample statistics. We now report the number and prevalence of risk-positive instances at each horizon.

Table 2. Risk-positive samples / valid labeled samples (prevalence).

HorizonHAIS-Bench
h167/182 (36.81%)7/160 (4.38%)
h274/182 (40.66%)23/160 (14.38%)
h375/182 (41.21%)18/137 (13.14%)
h468/182 (37.36%)—
h563/182 (34.62%)—
h672/182 (39.56%)—

HA maintains 182 labeled windows across all horizons, with relatively stable risk prevalence (34.62%–41.21%). IS-Bench exhibits greater variation in both sample size and risk prevalence, which warrants caution when interpreting individual horizon comparisons.

To further examine the horizon effect, we conduct supplied-action-sequence prediction experiments with fixed prediction origins. By explicitly providing future actions, this setting separates the difficulty of anticipating future actions from assessing their consequences and associated risks.

Table 3. Supplied-action-sequence prediction: HA · Full.

WMMetrich1h2h3h4h5h6
Gemini-3.7-FlashAccuracy0.740.710.750.750.760.69
Risk precision0.970.900.910.910.930.82
Risk recall0.360.290.300.340.270.23
Risk consistency0.200.230.250.280.190.14
GLM-5.3-FlashAccuracy0.730.710.690.730.740.68
Risk precision0.930.900.780.940.920.71
Risk recall0.350.290.240.280.230.25
Risk consistency0.190.250.160.140.130.14

Table 4. Supplied-action-sequence prediction: HA · Train-derived.

WMMetrich1h2h3h4h5h6
Gemini-3.7-FlashAccuracy0.660.640.660.650.670.65
Risk precision0.780.900.750.780.770.58
Risk recall0.120.170.150.110.130.16
Risk consistency0.080.050.050.060.070.05
GLM-5.3-FlashAccuracy0.630.630.640.640.690.66
Risk precision0.800.720.600.610.690.50
Risk recall0.210.120.150.180.160.08
Risk consistency0.020.040.030.070.080.07

Table 5. Supplied-action-sequence prediction: IS-Bench · Full.

WMMetrich1h2h3
GPT-5.6-TerraAccuracy0.820.640.73
Risk precision0.240.310.33
Risk recall1.000.780.89
Risk consistency0.320.290.24
GLM-5.3-FlashAccuracy0.870.690.71
Risk precision0.300.330.30
Risk recall1.000.650.78
Risk consistency0.300.280.28

Table 6. Supplied-action-sequence prediction: IS-Bench · Train-derived.

WMMetrich1h2h3
GPT-5.6-TerraAccuracy0.870.710.75
Risk precision0.290.200.21
Risk recall0.670.480.33
Risk consistency0.180.200.13
GLM-5.3-FlashAccuracy0.920.790.80
Risk precision0.270.390.29
Risk recall0.500.390.33
Risk consistency0.210.220.12

These findings provide additional evidence that near-term risk detection does not reliably generalize to longer horizons, even when future actions are supplied. While horizon-specific label distributions and limited risk-positive samples warrant caution, the additional results support our original conclusion regarding the limitations of long-horizon predictive safety.

Reviewer_Tj6HOpenReview ↗

Reply to Authors

Thank you for your detailed reply. Although some typos appear in the response of Q1, the response basically addresses my concerns. I will keep the score to 3.

Reviewer_PWRKOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This paper presents a diagnostic study of world-model-driven predictive safety guardrails for LLM agents. It examines three questions: how prediction integration affects safety and task utility, whether world models reliably predict safety risks, and whether predictive information is faithfully used in subsequent decisions. Through online evaluation, offline risk prediction, feedback perturbations, and sandbox-derived feedback, the authors report inconsistent safety benefits, substantial inference costs, and weaknesses in longer-horizon risk prediction. They argue that these findings reveal a gap between producing predictions and translating them into safer actions.

Critical strengths
  1. A coherent and reasonably comprehensive diagnostic framework. The paper clearly distinguishes prediction integration, prediction reliability, and downstream decision faithfulness. Connecting these questions through online evaluation, offline prediction tests, and feedback interventions provides a useful structure for investigating predictive guardrails. This organization helps identify limitations that would be difficult to understand from a single end-to-end safety score.

  2. Joint consideration of safety, task utility, and inference cost. The evaluation spans digital workflows and household interaction tasks, with multiple model configurations. Reporting task completion and computational cost alongside safety makes the practical consequences of stronger guarding visible. This provides useful evidence for assessing whether an apparent safety improvement represents an effective intervention.

Critical weaknesses

Limited depth of new insight beyond prior work.

The paper provides a comprehensive diagnostic framework and useful empirical coverage. However, several central observations, including ineffective use of accurate world-model simulations and trade-offs between stronger intervention and task completion, overlap with prior studies such as Qian et al. (2026) and Pro2Guard/ProbGuard. The analysis offers limited explanation of why outcomes differ across models and environments, which factors account for the observed failures, or under what conditions predictive guardrails are beneficial. For an analysis-focused paper, the substantially deeper understanding gained beyond these existing observations remains insufficiently demonstrated.

Questions for authors

What deeper insight does the unified diagnostic framework provide beyond prior work? Specifically, can it explain why the same integration strategy has different effects across models and environments, and which experiments support this explanation?

5 comments
AuthorsOpenReview ↗

Response to Reviewer PWRK(1/3)

We thank the reviewer for the positive assessment of our work and the suggestions regarding our contributions and the depth of our analysis. To address these questions, we conducted matched-state comparison experiments, prediction-reliability analyses, and feedback-behavior analyses to further explain the differences in predictive-guarding effectiveness and the conditions under which it is beneficial.

W1.1: What new insights does this work provide beyond prior work?

Prior work has primarily revealed the underuse of world-model simulations and the safety–task-completion trade-off that may arise from stronger safety interventions. In contrast, we further analyze predictive safety guardrails from three perspectives:

  1. RQ1: System-level effectiveness and its sources of variation. The same prediction may help one Actor correct an unsafe action while causing another Actor to abandon an already safe choice. Its effect depends on the interaction among the Actor, the WM, and the environment's safety constraints. We provide a detailed analysis in Q1.

  2. RQ2: Safety-oriented prediction reliability. We evaluate not only overall prediction accuracy, but also future-risk detection and the identification of specific violated constraints. Our results show that high accuracy can conceal missed risks, and that risk coverage remains insufficient at longer prediction horizons.

  3. RQ3: How predictive feedback translates into safe behavior. Actors do not always adjust their behavior faithfully according to WM predictions. Even when given a correct risk warning, an Actor may fail to implement the corresponding safety requirement. Conversely, the system may maintain or improve its safety score even when the feedback is irrelevant, corrupted, or inverted.

The three RQs form a complete analytical framework spanning system-level effects, prediction reliability, and decision use. Together, they further explain the differences in predictive-guarding effectiveness, its failure modes, and the conditions under which it is beneficial.

AuthorsOpenReview ↗

Response to Reviewer PWRK(2/3)

Q1: Why does the same integration strategy produce different outcomes across models and environments?

To explain the differences across models and environments, we conducted matched-state comparisons by fixing the WM and varying the Actor, and by fixing the Actor and varying the WM.

Table 1. Corrections and regressions across Actors with Gemini-3.7-Flash as the fixed WM.

EnvironmentActorCorrections / Initially unsafe candidatesRegressions / Initially safe candidates
HAGPT-5.6-Terra29/161 (18.0%)22/310 (7.1%)
HAGemini-3.7-Flash21/132 (15.9%)23/339 (6.8%)
HAGLM-5.3-Flash25/172 (14.5%)14/299 (4.7%)
IS-BenchGPT-5.6-Terra12/31 (38.7%)4/159 (2.5%)
IS-BenchGemini-3.7-Flash2/14 (14.3%)10/176 (5.7%)
IS-BenchGLM-5.3-Flash19/37 (51.4%)3/153 (2.0%)

Table 2. Corrections and regressions across WMs with Gemini-3.7-Flash as the fixed Actor.

EnvironmentWMCorrections / Initially unsafe candidatesRegressions / Initially safe candidates
HAGPT-5.6-Terra16/146 (11.0%)25/355 (7.0%)
HAGemini-3.7-Flash21/146 (14.4%)23/355 (6.5%)
HAGLM-5.3-Flash18/146 (12.3%)25/355 (7.0%)
IS-BenchGPT-5.6-Terra3/13 (23.1%)12/170 (7.1%)
IS-BenchGemini-3.7-Flash2/13 (15.4%)8/170 (4.7%)
IS-BenchGLM-5.3-Flash3/13 (23.1%)10/170 (5.9%)

Quantitative analysis. Table 1 shows that, on IS-Bench, Terra and GLM corrected 12 and 19 initially unsafe candidates, respectively, whereas Gemini made fewer corrections and introduced more regressions. Table 2 shows that changing the WM led to candidate-safety disagreements in 55 HA windows and 17 IS-Bench windows, 63.6% of which can be explained by differences reflected in the original feedback. In addition, 133 of the 146 initially unsafe HA candidates involved tool or resource overreach. Among the 90 IS-Bench tasks, 35 involved action preconditions and 54 involved terminal-state safety conditions.

Qualitative analysis. In the lamp case, given the same warning to switch off the lamp before moving it, GLM added the switch-off step, Gemini abandoned the safe choice it had made without feedback, and Terra retained the precaution. The glass-cleaning case shows that different WMs focus on different risks: Terra focuses on heat and electrical hazards, GLM on the cleaning-tool condition, and Gemini on the risk of colliding with the glass. The correction paths also differ across environments: corrections in HA mainly involve authorization boundaries, such as replacing an unauthorized tool call with compliant communication, whereas corrections in IS-Bench more often involve object states and action order, such as switching off a device or moving a glass before proceeding.

These results show that predictive-guarding effectiveness is jointly influenced by Actors' initial choices and responses to feedback, WMs' judgments about environment-specific constraints, and the safety requirements of different environments.

AuthorsOpenReview ↗

Response to Reviewer PWRK(3/3)

W1.2: Under what conditions are predictive guardrails beneficial, and how do they inform system design?

To further examine the conditions under which feedback is beneficial, we analyzed action transitions across matched HA windows to determine whether feedback helps the Actor turn an unsafe candidate into a compliant alternative.

Table 3. Changes to initially unsafe HA candidates under feedback, with Gemini as the fixed WM.

ActorInitially unsafe windows without feedbackActions changed after feedbackChanged actions becoming safeChanged actions remaining unsafeUnsafe tool call → compliant communication (windows/tasks)
GPT-5.6-Terra1618529 (34.1%)56 (65.9%)16/12
Gemini-3.7-Flash1323721 (56.8%)16 (43.2%)10/9
GLM-5.3-Flash1727725 (32.5%)52 (67.5%)12/12

Quantitative analysis. Table 3 shows that all three Actors produced compliant alternatives across multiple tasks. For GLM, 25 windows changed from an unsafe candidate without feedback to a safe candidate with feedback, including transitions from unsafe tool calls to compliant communication across 12 tasks. Meanwhile, 52 of the 77 GLM windows with changed actions still contained unsafe candidates, showing that changing an action does not necessarily resolve the original constraint.

Qualitative analysis. In the insurance-assessment case, GLM abandoned an unauthorized query and instead used existing information to report to an authorized supervisor, illustrating how changing the operation path can satisfy an authorization constraint. In contrast, the lamp case shows that feedback can also cause an Actor to abandon a safety precaution it had already selected correctly.

These results indicate that predictive feedback is most valuable when it helps the Actor address a safety requirement that has not yet been implemented, generate a compliant and executable alternative, and preserve precautions that are already correct.

We thank the reviewer for these valuable suggestions. We will incorporate the above experiments and analyses into the revised manuscript, further clarify the connections among the three RQs, and highlight our contributions to prediction reliability, feedback use, and the conditions under which predictive guardrails are effective.

Reviewer_FS1vOpenReview ↗
Rating
3(+1)
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper audits predictive safety guardrails along three questions: how predictions enter the decision process, whether the risk predictions themselves are reliable, and whether the content of those predictions is actually used in later actions. Online experiments in two environments compare no prediction, predictions used as advice, and predictions that control execution. Offline experiments examine risk prediction, and then use content perturbations and sandbox feedback to study the gap between prediction and action.

Critical strengths

I appreciate the audit perspective. Looking only at a final safety score can hide the fact that fewer tasks were completed, that less execution took place, or that cost increased substantially. Tables 1-2 place these outcomes side by side, which gives a more concrete picture of what predictive guardrails actually do in use. The paper also tries to distinguish errors in the predictions themselves from errors in how those predictions are used. That distinction is more valuable than simply stacking additional guardrail layers.

Critical weaknesses

The potentially strongest conclusion, about decision faithfulness, is also the least securely supported. RQ3 uses a system with execution gating, but the main text does not sufficiently specify, for each perturbation, which prediction fields change, what the gate conditions on, and what information remains visible to the action model. Under Full Inversion, the safety score reaches 90.1 while task completion is only 13.7. In that setting, simply executing less or terminating earlier could itself explain part of the safety increase. Conversely, a correct local sandbox check of a candidate action does not guarantee that the rest of the trajectory will be safe. A later violation also does not automatically show that the model ignored a warning that had correctly anticipated that same violation. Neither line of evidence yet separates unfaithful use of prediction content from these alternative explanations.

The long-horizon prediction experiment also needs a more precise connection to the online setting. Equation 5 predicts the risk associated with a future action in a recorded trajectory, and therefore jointly assesses what action will be taken and how risky that action is. The authors acknowledge this, but it is still unclear how that target corresponds to an online system that has to assess the consequences of a given candidate action. The main text also lacks effective sample counts and risk prevalence at each horizon, so it is difficult to judge how much of the reported change comes from differences in the evaluation samples themselves.

Overall assessment: The descriptive audit is useful, but showing that more checking is more expensive and can also be more conservative is not by itself a strong contribution. The decision-faithfulness analysis, which I value most, has not yet ruled out explanations that come from execution gating and from the scope of the prediction. I therefore currently lean toward rejection.

Questions for authors

For each RQ3 perturbation, please specify the fields used by the gate, the fields received by the action model, and the resulting execution or termination outcomes. Do the existing trajectories contain concrete warnings that correctly predicted the eventual violation but were ignored?

What are the sample counts and risk prevalence at each prediction horizon? Could a concrete example explain how the prediction target in RQ2 corresponds to online checking of a candidate action?

7 comments
AuthorsOpenReview ↗

Response to Reviewer FS1v(1/5)

We thank the reviewer for the positive assessment of our study and the suggestions regarding the experimental setup and supporting evidence. To address these concerns, we conducted additional experiments comparing risk control and predictive content, analyzed warning-to-execution traces, and evaluated horizon-specific risk distributions and prediction with supplied action sequences.

W1.1 & Q1: Which fields are perturbed, and how do we distinguish predictive content from execution gating?

Appendix D (Table 10) of the original manuscript details which risk judgments, numerical risk scores, explanatory text, and policy/action guidance are retained, removed, or modified under each perturbation.

To further distinguish predictive content from execution gating, we introduce four additional controls. null_empty and null_real both remove the risk label and numerical score, disable the hard veto, and retain the same pre-execution review procedure. They differ only in whether the original predictive content is provided. We additionally evaluate random_empty and random_real, which use randomized risk labels with and without predictive content.

Table 1. Four supplementary conditions and the No Prediction baseline from the same experimental batch (SAR / TCR).

ConditionQwen3.5-27BGLM-5.3-FlashGemini-3.7-Flash
No Prediction76.13 / 81.1264.41 / 85.6969.51 / 88.07
No hard veto + empty content (null_empty)88.33 / 74.0779.17 / 71.1377.89 / 86.05
No hard veto + original content (null_real)90.00 / 68.5086.64 / 74.7582.71 / 82.37
Random risk label + empty content (random_empty)97.73 / 34.0098.30 / 12.0798.19 / 34.87
Random risk label + original content (random_real)97.84 / 46.9798.21 / 10.5297.90 / 59.19

These results show that improvements in safety scores do not depend exclusively on specific predictive content or hard gating. Compared with No Prediction, null_empty achieves higher SAR after removing both, indicating that even a single Actor review can yield some safety improvement. The contribution of predictive content should therefore be assessed by comparing content conditions under the same review procedure, rather than inferring decision faithfulness solely from the final safety score.

AuthorsOpenReview ↗

Response to Reviewer FS1v(2/5)

W1.2: Do the high safety scores under Full Inversion reflect reduced execution or early termination?

We agree that reduced execution or early termination can explain part of the safety-score increase under Full Inversion. This condition tests how erroneous risk signals affect execution: it reverses the WM's safe/unsafe judgments and associated guidance, so candidates originally judged safe may instead trigger revision, blocking, or termination. We compare it with Sandbox feedback derived from actual execution evidence.

Table 2. Original submission: Sandbox feedback versus Full Inversion (SAR / TCR).

ConditionQwen3.5-27BGLM-5.3-FlashGemini-3.7-Flash
Sandbox feedback84.71 / 78.3073.77 / 78.5982.38 / 89.70
Full Inversion90.13 / 13.6787.37 / 19.77100.00 / 4.70

Across all three models, Full Inversion yields higher SAR and substantially lower TCR. The randomized-risk conditions in Table 1 similarly combine high safety scores with low task-completion scores. These results show that erroneous risk signals can make the system more conservative while impairing task progress.

AuthorsOpenReview ↗

Response to Reviewer FS1v(3/5)

W1.3: How does local sandbox verification relate to trajectory-level safety?

We distinguish candidate-level verification from trajectory-level safety. Sandbox verifies a candidate action and its local consequences in the current decision state, whereas the safety of the remaining trajectory also depends on subsequent actions and checks. A later violation therefore cannot, by itself, be attributed to the Actor's failure to use earlier correct feedback faithfully.

To distinguish these possibilities, we analyzed the correspondence among warning delivery, action revision, and observed violations (see the trace evidence under Q1 below). This analysis shows that some failures occur after a correct warning has been delivered and involve the same safety constraint identified by that warning, providing direct evidence that predictive feedback was not fully translated into compliant action.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Author-Editor Confidential Comment by Autho

Dear ACs,

We would like to summarize the strengths of our work acknowledged by the reviewers and our efforts to address their main concerns.


Strengths acknowledged by the reviewers:

  1. Timely and important problem. (All Reviewers)

    • Reviewers recognize the importance of evaluating predictive safety guardrails for LLM agents in long-horizon and safety-critical environments.
  2. Comprehensive experimental design. (All Reviewers)

    • Reviewers appreciate the systematic evaluation across multiple models, environments, and prediction settings, as well as the diagnostic framework connecting prediction reliability and downstream decision-making.
  3. Clear presentation and practical implications. (All Reviewers)

    • Reviewers acknowledge the clarity of the paper and its practical implications for understanding and designing predictive safety systems.

Main concerns raised by the reviewers:

  1. Insufficient explanation of differences across models and environments. (Reviewer PWRK)

    • Rebuttal: We conducted matched-state comparisons by fixing the WM and varying the Actor, and vice versa. Additional quantitative and qualitative analyses reveal how Actor responses, WM interpretations, and environment-specific constraints jointly affect predictive feedback.
    • The concern has been addressed through additional experiments and analyses, and the reviewer increased their rating.
  2. Insufficient analysis of long-horizon prediction reliability. (Reviewers Tj6H and FS1v)

    • Rebuttal: We added horizon-specific risk statistics and supplied-action-sequence prediction experiments. The results show that long-horizon risk coverage remains limited even when future actions are explicitly provided, demonstrating that the difficulty lies not only in anticipating future actions but also in assessing their consequences and risks.
    • The concern has been addressed with additional experimental evidence, and Reviewer Tj6H increased their rating.
  3. Unclear contributions of predictive content, pre-execution review, and execution gating. (Reviewer FS1v)

    • Rebuttal: We introduced four controlled conditions to disentangle these factors. The results show that pre-execution review can improve safety scores without predictive content, while predictive content provides additional model-dependent effects. We also analyze how restrictive execution under unreliable risk signals affects task completion.
    • The reviewer acknowledged our additional experiments and clarified findings, while maintaining the original rating.
  4. Insufficient insights into when predictive guardrails are beneficial and their design implications. (Reviewer PWRK)

    • Rebuttal: We analyzed differences in Actor responses to identical feedback, WM interpretations of safety constraints, and environment-specific correction paths. Our findings identify when predictive feedback is beneficial: when it addresses unimplemented safety requirements, supports compliant alternatives, and preserves existing valid precautions.
    • The concern has been addressed through additional analyses, providing deeper insights and concrete design implications.

We have incorporated the additional experiments and analyses into the revised manuscript, with all changes highlighted in orange.

We sincerely appreciate the reviewers' constructive feedback, which has helped us strengthen the empirical evidence and clarify the contributions of this work.

Thank you for your time and consideration.

Best regards, The Authors

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.