acceptodds
Under review as a conference paper at ICLR 2027

Beyond Plausible Prediction: Safety Sufficiency in Web World Models

Abstract

Web world models increasingly predict the consequences of candidate actions before agents act, yet a plausible predicted future does not necessarily support the correct safety decision. In a matched human evaluation of 3,000 predictions from ten frozen open models, 86.1% are judged semantically plausible. However, several models exhibit highly asymmetric authorisation behaviour, combining low false-safe execution with poor recall of clarifiable ASK cases and a strong preference for BLOCK. This motivates safety sufficiency: whether a predictive interface preserves and exposes the policy-decisive information required to distinguish actions that may be EXECUTED, require clarification through ASK, or should be BLOCKED. We introduce SafeWebWorld, a benchmark for evaluating safety sufficiency across five web-safety domains through action-conditioned futures, policy-decisive evidence, and policy-conditioned decisions. Under a controlled Security-Aligned Supervised Fine-Tuning protocol, evidence preservation can improve substantially while authorisation-sensitive decisions deteriorate. To determine whether this deterioration reflects loss of authorisation-sensitive score structure or a failure of the decision readout, we introduce Latent Authorisation Recovery (LAR). Frozen-score recalibration reveals substantial but backbone-dependent recovery, while paired recovery provides only a modest additional gain. Yet within-family and matched-authorisation evaluations remain poor, showing that aggregate recoverability does not imply robust fine-grained authorisation behaviour. Overall, plausible prediction, evidence preservation, and aggregate recovery are each insufficient on their own to establish robust authorisation.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
web world models, web agents, AI safety, security alignment, safety evaluation, benchmarks, evidence sufficiency, authorisation

Reviews

Among:
Shape:
1234this paper, Bottom 10%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.67 over 3 reviews, above 10% of the others.
Reviewer_nB75OpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper argues that a web world model can predict a plausible future for a candidate action while leaving out the information needed to decide whether the action should be taken, and calls the needed property safety sufficiency. In SafeWebWorld (30,000 / 1,000 / 3,000 train / development / test examples from existing benchmarks and a synthetic set), a model receives a goal, policy, observation, and candidate action, and outputs a predicted future, security evidence, consequence, risk, and an EXECUTE, ASK, or BLOCK decision. In a human study, 86.1% of 3,000 sampled futures are judged plausible, while several models almost never choose ASK. After fine-tuning (SA-SFT), Llama’s Evidence Recall rises while its ASK recall falls to almost zero (Authorisation-Policy Collapse). The LAR analysis shows that recalibrating the decision scores recovers much of the accuracy and that combining two backbones (DCPF) reaches 96.23% balanced accuracy, while fine-grained checks remain poor.

Critical strengths

The three-way decision is a useful lens. Separating ASK from BLOCK and including safe controls exposes behavior that a binary safe/unsafe metric hides: WebSynthesis-Qwen3-8B almost never executes unsafe actions (0.08% false safe) but never chooses ASK when it is required, and GPT-5.6, Claude Opus 4.8, and Gemini 2.5 Pro reach only 4.88%–14.63% ASK recall (Table 2).

The paper does not over-claim from its aggregate results. It tests DCPF within a mixed-label risk family and on matched EXECUTE/ASK/BLOCK triplets and reports that it drops to 35.55% balanced accuracy and 8.89% exact-triplet accuracy (vs. 73.33% for base Llama). This honest reporting is valuable.

Critical weaknesses

The central claim, that plausible futures omit policy-decisive evidence, is not measured directly. The model sees the full observation and policy and writes the future, evidence, and decision in one call, so the decision does not need to rely on the future. Evidence Recall is word overlap between the predicted security-evidence field and the reference (Eq. 14), not a check on the future, and the human study judges only plausibility; as the authors note, it “does not annotate policy-evidence omission at the individual prediction level” (line 1767). The two key numbers thus come from different output fields. The rise in Evidence Recall after SA-SFT (e.g., 28.68% to 93.32% for Qwen) may also mostly reflect learning the annotation wording. The input-source ablation in Appendix F.5, which decides from the predicted future alone, is the closest direct test and should be the main experiment.

Label quality limits what the numbers mean. The test labels were checked by a single reviewer without inter-annotator agreement (line 866), 54.2% of the test set is synthetic with no description of how it was generated, and the authors’ own probes show that risk family, unigrams, and templates predict the labels (line 897). Models trained on this data may learn these patterns, which fits the large gap between DCPF and zero-shot models and DCPF’s drop within a family. It is therefore unclear whether the low ASK recall of strong models is a real failure or a mismatch with this benchmark’s labeling convention.

Questions for authors

For Llama, the ASK collapse already appears in the “supervised foundation” checkpoint before SA-SFT (Table 28). What data and loss were used for this checkpoint, and was the class imbalance (66.4% BLOCK, 13.6% ASK in training) handled? If the collapse comes from imbalance, recalibration fixing it is expected, and Authorisation-Policy Collapse may be specific to this training recipe.

1 comment
AuthorsOpenReview ↗

Response to Reviewer nB75

Thank you for examining the evaluation protocol, dataset construction and checkpoint lineage. We have clarified the measurement scope, brought the existing future-usefulness result into the main discussion and added verified foundation-training details.

W1. Joint prediction versus direct evidence-omission measurement

The main evaluation is a joint structured-prediction interface: all five fields are generated in one autoregressive response, and the decision can access the original input and preceding output tokens. It does not impose a future-only bottleneck. Section 3.1 now makes this explicit. ER measures the evidence field, while the human study evaluates future plausibility; neither directly measures individual future-text omissions. The Introduction and Section 3.5 now make that distinction explicit, Figure 1 is labelled illustrative, and Sections 2.1 and 3.4 describe the diagnostics used to study aspects of safety sufficiency.

We now highlight the existing input-source ablation from Appendix F.5 in Section 5.2. Under this separate offline decision protocol, Llama’s BA is 90.366% with original context X, 54.284% with frozen future F and 74.732% with X+F. Adding F to X lowers BA by 15.634 points (paired-bootstrap 95% CI [−17.529, −13.804]); Qwen’s X+F gain has a CI including zero. These are decision-usefulness results, not lexical-overlap measurements. F also removes the policy and other context, and X+F does not isolate a specific missing cue. The result therefore supports the conclusion that these futures are neither sufficient substitutes for context nor uniformly beneficial additions under this protocol, without identifying evidence omission as the cause of each error. We retain the joint benchmark as the main evaluation and use F.5 as a separate test of future usefulness, since they answer different questions.

W2. Labels and shortcut sensitivity

Appendix A.4 already describes the annotation and manual review of all 3,000 TEST records by one author. With no independent annotators, inter-annotator agreement was not measured. Section 3.3 now highlights the audit description, and Section 6 identifies the agreement limitation. The policy-conditioned contract in Section 3.2 distinguishes unresolved authorisation from prohibition; low ASK recall is measured against that contract.

We have expanded Appendix A.3 to explain how the synthetic examples were generated and processed. Python templates construct the scenarios, policies, reference futures and evidence, with explicit rules assigning labels and decisions. An AI coding assistant helped develop the code; individual examples are generated without LLM calls. Public inputs are deduplicated, synthetic examples fill the required risk-category counts, and a fixed seed assigns the splits. Later text revisions preserve split membership and gold targets. Final checks cover missing fields, duplicates, cross-split instance overlap and visible answer markers.

Appendix A.5 reports lexical, template and family regularities that predict labels. Group-aware and source-transfer checks do not eliminate these concerns. The existing Section 5.5 and Appendix K.3 show the limit of aggregate recovery: on the matched-authorisation diagnostic, DCPF achieves 8.89% exact-triplet accuracy versus 73.33% for base Llama. We therefore do not interpret high pooled performance as evidence of shortcut-free authorisation reasoning.

Q1. Foundation training, imbalance and onset of degradation

Appendix K.1 now specifies the checked foundation used to initialise the three Llama SA-SFT runs. It completed two epochs over all 30,000 frozen training messages: 6,000 EXECUTE, 4,080 ASK and 19,920 BLOCK. Training comprised 7,500 optimiser updates with batch size one and gradient accumulation eight, using full-conversation causal-LM loss with labels equal to input IDs. No class weighting, class-dependent resampling or decision-specific weighting was applied. Final SA-SFT instead uses response-only supervision and ten optimiser updates.

Table 29 reports emitted decisions on DEV-1000: ASK recall falls from 48.94% to 4.26% before final SA-SFT, then to 0.00%. The Abstract, Introduction, Section 5.3 and Conclusion now reflect this timing. A training-recipe explanation remains possible; these comparisons do not isolate imbalance or supervision format as the cause. The frozen-score recovery analysis consequently tests recoverability on the evaluated distribution, not a universal collapse mechanism or robust authorisation understanding.

Reviewer_mswJOpenReview ↗
Rating
1
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper introduces SafeWebWorld, a benchmark for evaluating whether web world models can make correct safety decisions before taking specific web actions. Each example provides a user goal, web observation, security policy, and candidate action. The model must predict the action’s consequences, identify policy-decisive evidence, and choose EXECUTE, ASK, or BLOCK under the supplied policy. ASK means seeking clarification or authorisation before proceeding. The evaluation is offline and does not execute the candidate actions. The test set contains 3,000 examples across five safety-risk families. The authors evaluate ten open models with their balanced accuracy (mean recall across EXECUTE, ASK, and BLOCK) ranges from 64.00% to 82.95%, with Llama-3.1-8B-Instruct achieving the highest score. The authors apply Security-Aligned Supervised Fine-Tuning (SA-SFT) to Qwen and Llama to test whether training on the benchmark’s reference outputs improves safety decisions. Compared with the original models, both achieve higher Evidence Recall, but balanced accuracy increases for Qwen and decreases for Llama. Finally, the authors test whether decision errors can be corrected without further updating the language models. They train linear classifiers on one or both frozen models’ scores for EXECUTE, ASK, and BLOCK. The two-model variant, DCPF, achieves 96.23% balanced accuracy (Table 1), but gets all three decisions right in only 8.89% of matched triplets with different authorisation conditions (Figure 5),indicating that high aggregate scores do not ensure reliable decisions when authorisation conditions change.

Critical strengths
  1. The paper studies a practically important problem: deciding when a web agent should execute an action, ask the user, or block it according to safety principles under specific conditions. This matters for preventing unsafe actions without unnecessarily rejecting legitimate tasks.
  2. The paper presents substantial experimental work, covering ten open models, supervised fine-tuning, recalibration, and matched-authorisation diagnostics. The authors include several controls and report negative results alongside improvements on the main benchmark. These experiments help clarify both the potential and the limitations of their approach.
Critical weaknesses
  1. The contribution appears limited compared with existing work. Predicting plausible consequences and determining whether an action is permitted under a security policy are two different requirements, and prior work already uses predicted consequences for safety interventions, such as SafePred discussed in Section 2.1. The contribution here seems mainly to be distinguishing EXECUTE, ASK, and BLOCK and evaluating security evidence. The paper does not sufficiently establish what important new insights these distinctions provide or whether they justify a standalone paper. I am not convinced that the problem formulation and analysis constitute a substantial contribution.
  2. The necessity and value of calibrating decision-label scores are not established. Training a linear classifier on frozen models’ decision-label scores is a standard calibration approach. Although the authors present it as a diagnostic tool, they do not adequately explain why this particular approach is needed. It feels like applying an established method to a specific task, without a distinct contribution. DCPF requires two models, but improves balanced accuracy by only 1.25 percentage points over the strongest single-backbone recalibration on the development set. This incremental gain is small and does not seem to constitute a substantial contribution in performance either.
  3. The Introduction states that “Under controlled Security-Aligned Supervised Fine-Tuning (SA-SFT), evidence preservation can improve substantially while authorisation-sensitive decisions deteriorate”. However, the explanation for this gap remains limited: if models understand the security evidence better, why do their authorisation decisions still fail? I doubt that the higher Evidence Recall may reflect learning to match the reference wording rather than better preservation of policy-decisive evidence since this metric is based on token overlap (Appendix B.1).This gap needs more detailed analysis and explanation. As presented, the observation remains superficial and insufficiently explained.
Questions for authors

Please address my concerns stated in the weakness part.

2 comments
AuthorsOpenReview ↗

Response to Reviewer mswJ

Thank you for recognising the practical importance of policy-conditioned authorisation and the experimental analysis. We have clarified the contribution, the role of the recovery controls and the interpretation of the training results.

W1. Contribution relative to prior predictive-safety work

Our distinction from prior predictive-safety methods, including SafePred, concerns the evaluation target and diagnostic analysis. SafeWebWorld evaluates whether a pre-execution predictive interface distinguishes actions permitted immediately, requiring clarification, or prohibited under an explicit policy. It then examines the relationships among future plausibility, evidence-token recovery, emitted decisions and recoverability from frozen scores. Section 2 and Appendix Table 3 position this evaluation relative to prior work; Sections 2.1 and 3.4 now state the evaluation scope more precisely.

The empirical contribution is the separation of three failure relationships: low false-safe execution can conceal complete ASK failure (WebSynthesis: 0.08% false safe, 0.00% ASK recall); improved evidence-token recovery can accompany near-complete ASK failure (Llama, Table 1); and high aggregate recovery can coexist with poor matched-authorisation correctness (DCPF: 96.23% TEST BA, 8.89% exact-triplet accuracy on the separate diagnostic). These findings motivate evaluating the properties separately rather than treating one as a substitute for another.

Additional prompting control. We have added matched zero-shot and fixed three-shot controls on DEV-1000 using the original Qwen3-8B and Llama-3.1-8B-Instruct checkpoints, without training or recalibration. Three fixed TRAIN demonstrations improve BA by 26.662 percentage points for Qwen (paired-bootstrap 95% CI [23.917, 29.146]) and 5.707 points for Llama ([3.112, 8.435]). Qwen’s ASK recall increases from 3.546% to 81.560%, while its False Safe decreases from 6.875% to 2.875%. Llama’s ASK recall also improves, but BLOCK recall remains 50.228% and False Safe changes from 4.875% to 5.000%. These results show that task demonstrations can substantially affect authorisation performance, with different effects across backbones. The complete protocol and results are reported in the new Appendix F.6 and briefly referenced in Section 5.3.

W2. Why use frozen-score recalibration?

Recalibration is a controlled test of recoverability: keeping the language model and score inputs fixed tests how much decision performance can be recovered by changing only the readout. Section 4.2 now clarifies that each complete candidate label is scored token by token and that semantic-score argmax may differ from the emitted label. Section 4.3 specifies the TRAIN-fitted readouts and frozen backbones.

Llama retains 0.9315 ASK/BLOCK AUC on DEV; affine recalibration increases BA from 66.32% to 95.24% relative to semantic-score argmax on the same scores (Tables 15–16). The existing scalar controls reach 68.43% BA in probability space and 88.75% with an ASK/BLOCK threshold, compared with 95.24% for affine recalibration (Table 30). These controls distinguish recoverability under different readouts; they do not identify a training-time mechanism.

The original Section 5.3 already reports that DCPF adds only 1.25 BA points over the strongest calibrated backbone. The revised discussion makes clearer that same-backbone recoverability and additional paired gain are separate questions: most aggregate recovery is available from one backbone, while pairing adds a modest descriptive increment. Our contribution centres on the benchmark and empirical diagnosis, rather than a new affine estimator or a large paired performance advantage. Section 5.5 and Appendix K.3 test the limits of that recovery.

W3. Evidence Recall and the explanation of collapse

ER can reward reference-like wording and should not be interpreted as semantic understanding. Section 3.5 now states this explicitly. The existing lineage analysis places Llama’s principal observed ASK decline before final SA-SFT: 48.94% at base, 4.26% at foundation and 0.00% after final adaptation (Table 29). Appendix K.1 adds the verified training recipe, including class counts, full-conversation loss and no class balancing. The Abstract, Introduction, Section 5.3 and Conclusion now reflect this timing. The analysis establishes an observed dissociation and its checkpoint interval; imbalance and supervision format remain possible causes requiring controlled tests, not established explanations.

Reviewer_mswJOpenReview ↗

Response to the reviewer

Thanks for your clarification. I will take into consider for the scoring.

Reviewer_f6QgOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
1
Soundness
2
Read review
Paper summary

This paper contributes a safety dataset for the web agent domain that instead of using binary safe/unsafe classification labels it uses three-class labels with an added "ASK" class for seeking user feedback. They propose adding "safety sufficiency" as an evaluation criteria for world model predictions, analyzing how much information about the safety aspects of an action are included in the predicted state of the world, and discuss the links between safety sufficiency and success at the three-class prediction task. They also present findings to the effectiveness of improving capabilities on this task through SFT training.

Critical strengths
  1. The creation of a 34k sample dataset that adds three-class EXECUTE/BLOCK/ASK labels to a collection of existing web agent datasets is a valuable contribution.
  2. The finding that even world models whose predictions contain accurate safety-relevant information struggle to correctly label ASK situations is interesting.
Critical weaknesses
  1. There is a lack of evidence for the quality and human agreement with the labels of the new dataset.
  2. The paper argues for adding "safety sufficiency" to the evaluation process, but doesn't show how evaluating that metric contributes to improving any overall objectives or provides new insights. In fact, the paper shows a disconnect between successful SFT for preserving safety information in world model outputs and the task of correctly labeling the ASK situation.
Questions for authors
  1. In what ways did you measure human agreement with the labels on SafeWebWorld?
  2. Why do you think that successful SFT for preserving safety evidence did not lead to improved performance on ASK recall on SafeWebWorld?
  3. What insights can be gained from evaluating safety sufficiency? Is there a divide between models with low safety scores due to inaccurate safety predictions verses models that have insufficient safety predictions, and this insight leads to different suggested actions for improvement?
3 comments
AuthorsOpenReview ↗

Response to Reviewer f6Qg

Thank you for recognising the dataset’s value and highlighting ASK failures. We have revised Figure 1’s caption and clarified the protocol, decision-label scoring and result interpretation in Sections 3.1, 4.2 and 5.3.

W1 / Q1. Label quality and human agreement

As described in Appendix A.4, one author annotated the dataset and manually reviewed all 3,000 TEST records for policy–decision consistency, evidence alignment, visible-input sufficiency and field consistency. There were no independent annotators, so inter-annotator agreement was not measured. The revision makes this existing audit description more visible in Section 3.3 and identifies the agreement limitation in Section 6. Section 3.2 specifies the policy-conditioned labels: EXECUTE permits proceeding without further authorisation, ASK requires clarification of authority, scope or intent, and BLOCK denotes prohibition.

W2 / Q2. Why evidence recovery and ASK recall diverge

Evidence Recall measures reference-token overlap in the evidence field. Reproducing more reference wording does not itself establish better semantic understanding or a better distinction between clarification and prohibition. On TEST-3000, Llama’s ER rises from 23.22% to 87.95%, while ASK recall falls from 74.33% to 0.49% (Table 1). Section 3.5 now states this metric boundary explicitly. The measured dissociation does not establish that the supervision targets contradict each other.

The existing checkpoint analysis locates most of Llama’s observed decline before final SA-SFT: emitted DEV ASK recall is 48.94% at base, 4.26% at foundation and 0.00% after final adaptation (Table 29). Appendix K.1 now provides the foundation’s training data, loss and absence of class balancing. Imbalance and supervision format are possible explanations, but their effects are not isolated. The Abstract, Introduction, Section 5.3 and Conclusion now distinguish the training lineage from the final adaptation stage. Qwen follows a different trajectory.

W2 / Q3. What the findings imply for evaluation

The findings guide evaluation of upstream predictive interfaces in three ways. First, assess clarification separately from unsafe execution: WebSynthesis has 0.08% false-safe execution but 0.00% ASK recall (Table 1). Second, distinguish poor emitted decisions from score-level discrimination: Llama retains 0.9315 ASK/BLOCK AUC, and an affine readout raises DEV BA from 66.32% to 95.24% on the same frozen scores (Tables 15–16). Third, test recovery under authorisation changes: DCPF’s 96.23% TEST BA coexists with 8.89% exact-triplet accuracy on the separate matched diagnostic (Section 5.5).

Future usefulness also needs direct testing. Section 5.2 now highlights the existing F.5 ablation: adding frozen future F to original context X lowers Llama’s BA by 15.63 points (95% CI [−17.53, −13.80]), while Qwen’s gain is inconclusive. Thus, future plausibility, evidence-token recovery and authorisation performance provide different information when evaluating this interface.

These diagnostics separate observable failure signatures, but do not by themselves attribute each error to an inaccurate future versus an omission of policy-decisive information within that future. The new fixed three-shot control in Appendix F.6 further shows that baseline authorisation performance is sensitive to task demonstrations.

In several places you highlight facts like "WebSynthesis has 0.08% false-safe execution but 0.00% ASK recall", but there isn't enough explanation of the ways this matters. From first principles it is reasonable that we would want models that seek clarification, but I would have a hard time defining the boundaries of when I want that clarification. It might be difficult to get high levels of inter-annotator agreement on this, and you would need both that and a demonstration of impact to make this compelling.

AuthorsOpenReview ↗

Response to Reviewer f6Qg

Thank you for the helpful follow-up. Our point in the WebSynthesis example is not that low false-safe execution alone is sufficient. In a pre-execution web-agent setting, an action may be already prohibited, or it may be permissible once a missing authorisation, scope, or intent is clarified. Treating both cases as BLOCK can avoid unsafe execution, but it cannot support authorised action when clarification would suffice. Thus, 0.08% False Safe together with 0.00% ASK recall shows that the model avoids direct unsafe execution while failing to make this distinction.

This is the purpose of evaluating ASK and BLOCK separately under the policy-conditioned contract in Section 3.2; Figure 1 illustrates why a plausible predicted future can still omit the information needed for that decision. The construction and manual-review procedure, together with the scope of the single-reviewer audit and the absence of an independent agreement statistic, are stated in Appendix A.4 and Sections 3.3 and 6.

Comments on the paper

1 comment
AuthorsOpenReview ↗

Overview of revisions and responses

We thank all three reviewers for their careful reading and constructive feedback. The revised manuscript clarifies the evaluation protocol, metric interpretation, checkpoint lineage and scope of the recovery findings. Changes to the manuscript are highlighted in blue, and our reviewer-specific responses identify the relevant sections.

For clarity, W and Q refer to the points under “Critical Weaknesses” and “Questions for Authors,” respectively, numbered in their original order. Related points are answered together where appropriate. For Reviewer mswJ, whose question refers back to the weaknesses, we address W1–W3 directly.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.