acceptodds
Under review as a conference paper at ICLR 2027

CAF: Task-Scoped Authorization against Cross-Channel Composition Attacks in Web Agents

Abstract

Web agents are increasingly used in everyday work and daily life. In this context, an agent often needs to combine information from multiple channels, including DOM/accessibility, visual content, and tool returns. This creates a new attack surface: cross-channel composition attacks, where individually plausible information from different channels can jointly form a complete unauthorized operation. Such attacks can cause the agent to execute operations that fall outside the user’s intended task. However, existing defenses have paid limited attention to this threat, and benchmarks also provide limited support for evaluating such attacks. Therefore, we propose CAF, a pre-execution gateway that checks whether the complete operation proposed by the agent remains within the user’s authorized task scope. We further build C3 Bench, a gym-like execution benchmark that provides a standardized evaluation framework for cross-channel composition attacks and defenses. In the main C3 replay across five models, CAF reduces observed unauthorized-effect rates from 7.7–21.5% to 0% while retaining all authorized effects in the paired baseline runs.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
Web agents, agent security, cross-channel composition attacks, runtime authorization, provenance, prompt injection

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_1QXrOpenReview ↗
Rating
3(+1)
Confidence
2(−1)
Informativeness
3
Presentation
2
Soundness
2
Read review
Paper summary

This paper addresses the problem of Web Agents combining information from the DOM, visual inputs, and tool outputs to perform complete operations that lack user authorization—such as pairing a legitimate object with a legitimate recipient in a way that was not explicitly authorized. The authors propose CAF, a framework that constructs task contracts from trusted task requests and pre-registered policies, normalizes candidate tool calls into complete semantic relationships, and grants authorization based on provenance evidence and compositional permissions; it then constrains actual dispatch and side-effect submission through parameter binding and a CapabilityBroker. The paper also introduces C3 Bench, covering eight task categories and twelve channel distributions. In replay tests using fixed call pairings across five models, CAF reduced the observed rate of unauthorized effects from a range of 7.7%–21.5% to 0%, while preserving all 1,987 authorized effects found in the baseline. Supplementary experiments included contract compilation, new schema validation, closed-loop execution in AgentDojo, pre-execution intent checking in WebArena, and attack tests involving fixed and limited feedback. These results demonstrate the framework's effectiveness across registered policies, schemas, and protected execution paths.

Critical strengths
  1. The paper's most valuable contribution is the insight that a set of valid values ​​does not define a set of valid operations. When users authorize specific pairings between two objects and two recipients, simply checking the validity of the object, recipient, and tool individually can still allow erroneous pairings to slip through. CAF explicitly incorporates these pairings into task contracts and composition permissions, enabling security policies to express the operations actually authorized by the user. This advantage is supported by specific experimental evidence:
  • Table J2 categorizes samples into 133 pairs distinguishable at the field level and 59 pairs that pass individual field checks but differ in their full relational authorization. This classification helps identify the conditions under which relational checks are truly effective.
  • Even more compelling is Table E7: when domain constraints are applied to four newly registered schemas, independent field policies still result in 19/48 unauthorized effects, whereas schema-aware CAF results in 0/48; both successfully preserve all 32/32 authorized effects. This latter comparison rules out a significant alternative explanation: that CAF’s benefits stem merely from stricter schema checks or the rejection of more operations. It provides substantive evidence that checking combined relationships is genuinely necessary. Rationale for acceptance: This insight has concrete implications for the design of agent security systems and has been implemented as an executable, auditable authorization mechanism.
  1. The paper goes beyond merely determining whether a candidate invocation is safe; it binds approved parameters and verifies actual side effects along the protected execution path. The experiment in Table H4 is particularly valuable: when unmodeled parameters are permitted, both approaches achieve 1,600/1,600 handler reachability; however, with the introduction of the Broker, unauthorized effects drop from 1,600/1,600 to 0/1,600, while authorized control is maintained. Table J3 further demonstrates the versatility of this execution-binding layer by applying it to different authorization backends: in all cases, the 48 instances of parameter modification following approval were eliminated. This demonstrates the author's ability to decouple the benefits of execution binding from those of backend authorization, rather than attributing all improvements solely to the relational representation of the CAF. The contract compilation experiments also employed a corresponding isolation design: the compiler runs prior to the emergence of candidate calls—without accessing the calls or result labels themselves—and verification takes place within a new task context after the state is frozen. These design choices mitigate the risk of the experiments relying on information regarding the expected answers. The paper presents a comprehensive method for failure decomposition, enabling readers to pinpoint exactly which stage of the process yields the security benefits. Its value lies in both the system implementation and the evaluation methodology.
Critical weaknesses

W1. Compared to backends with relational constraints, CAF's advantages have not fully ruled out differences arising from policy encoding versus compilation coverage.

In Table J2, CAF-Compiled-Rule resolves 44 out of 59 pairs within the strict relational difference subset, whereas the two baselines incorporating relational constraints resolve 20 out of 59. Table J3 reports a discrepancy of 15/144 versus 39/144 regarding unauthorized effects, even after sharing execution bindings.

However, the equivalence claimed in Section J.4 is primarily based on the fact that the backends construct representations from the same canonical manifest and process identical instances. Sharing inputs and expected labels is not equivalent to actually encoding semantically equivalent, complete authorization predicates. The current PDF lacks representative rules and failure instances, making it impossible to determine whether the additional 24 errors stem from limitations in relational representation or from the adapter overlooking specific joint constraints, provenance contexts, or channel rules.

My confusion arises because Section A.1 explicitly acknowledges that existing systems are not theoretically incapable of expressing equivalent relational constraints. On the controlled subset in Table D2, both Source/Field and Task Shield-style approaches successfully prevent all observed unauthorized effects under full role integrity conditions.

W2. Unresolved ambiguities remain regarding execution security endpoints, particularly the state anomalies in Appendix J.

First, Tables J5–J7 report 5 out of 576 end-to-end unauthorized state discrepancies for Full CAF, while simultaneously stating that no unauthorized protected path submissions occurred. Based on the content, we can infer the following:

  • Four cases occurred after a block or unconfirmed confirm operation and were attributed to parser fallback or unprotected harness routing.
  • In the fifth case, a parameter modification was rejected by the Broker yet persisted in the end-to-end UER (Unauthorized Effect Report).

The current documentation fails to clarify: What specific environmental state change occurred in the fifth case that qualified as a UER-relevant event? Are the first four cases actual, exploitable integration gaps, or are they tests specifically constructed to probe boundaries outside the protection scope?

This does not directly invalidate the conclusions regarding the local security of protected paths. However, if these routes belong to the complete Agent system under evaluation, merely excluding them from the path analysis is insufficient to establish the system's end-to-end security.

Furthermore, while J.6 claims that the four variants merely add mechanisms incrementally without altering prior logic, Table J5 shows pre-gateway error counts of 3, 2, 4, and 0, and the authorization retention rate rose from 142/144 to 144/144. These changes require case-by-case explanation to confirm that the ablation did not simultaneously alter parsing or execution behavior.

It remains difficult to determine the security outcomes of the Full CAF when deployed as a complete system, or to fully accept the attribution of effects to specific mechanisms in the staged ablation.

W3. Automated contract construction is validated but primarily covers explicit field extraction; it has not yet been proven capable of supporting tasks where the authorization target depends on environmental information.

The compiler in I.3 requires the action to match the tool mapping precisely and mandates that object, destination, and scope appear verbatim in the task text; it returns confirm in cases of omission or ambiguity. The successful recovery of context in 31 out of 32 cases after freezing is a valuable result, yet it primarily validates this constrained input scenario.

Meanwhile, §4.2 allows runtime observation to instantiate parameters within the contract's authorized value range. A sufficiently clear link between the two is still missing: for tasks where object identity or destination must be determined via environmental queries, how does the contract express a constrained yet uninstantiated authorization and subsequently securely bind the query results?

Table E8 reports the configuration scale for the new schema and indicates rule reuse; while this addresses some concerns regarding configuration costs, it does not substitute for the validation of dynamic task binding.

W4. Binding rules based on tracing parameter values ​​back to their sources lack clear semantics for scenarios involving identical values ​​from multiple sources or derived values.

I.4 explains that the normalizer binds parameters to a registered SourceRef based on parameter values, current observation records, and the schema, while verifying consistency across value, locator, and channel. Refusing to trust self-reported sources is a sound design choice, yet parameter value matching may still yield non-unique results. This does not require the system to infer the model's internal reasoning process; rather, it requires specifying: what verifiable conditions are sufficient for a source to serve as authorized evidence for that parameter. Without this, the practical implications of channel authorization and witness independence remain insufficiently defined.

Tables E5 and H3 have tested erroneous channels, missing sources, and duplicate witnesses; however, the current PDF does not present the corresponding rules and results for cases where source ambiguity exists.

W5. A more specific, record-level explanation is needed regarding the relationship between "strict cross-channel success" and "incomplete-role conditions."

Section 3.3 requires participating channels to jointly provide the necessary roles for the target relationship, whereas B.1–B.2 define certain masks as lacking at least one core role. The paper explicitly states that these construction labels are not equivalent to authorization labels; thus, it is not inherently contradictory for incomplete-role conditions to yield authorized outcomes.

However, the specific tallying of "strict cross-channel success" requires further detailed explanation. Taking GPT-5.5 as an example:

  • The combined count of target effects in Tables D1 and D9 is 37;
  • The total number of target effects under the five complete-role conditions is 26;
  • Yet, Table D10 shows 35 effects satisfying the strict cross-channel condition.

This implies that at least 9 instances of strict success originated from incomplete-role conditions. The authors need to present the QreqQ_{\mathrm{req}} values, the roles provided by each channel, and the actual relationships for these instances, explaining why the missing core roles did not prevent the coalition conditions from being met.

Questions for authors
  1. For the 24 cases in Table J3 where CAF succeeded but the relationship-augmented baseline failed, could you provide representative, complete authorization predicates and the three types of backend encodings? Do these errors stem from limitations in expressive power, compilation omissions, or implementation discrepancies? Do the discrepancies persist after applying manual equivalence verification?

  2. Please detail the actual state transitions (before and after) for the five cases in J7. Specifically regarding J7-005: given that the submission was rejected, what state change caused it to still be counted as a UER? Additionally, which experiments utilized the same final CAF configuration, and which relied on older compiler versions or different execution paths?

  3. Regarding contract applicability: How does the current compiler generate contracts for authorization operations where the task text does not explicitly specify the final object ID or destination, requiring tool queries to resolve them? Among the reported AgentDojo successes, how many involve such dynamic binding, and how are the confirmations processed and factored into utility?

  4. How does SourceRef make a selection when the same parameter value appears across different trust levels or channels? When multiple consistent sources exist, what rules prevent satisfying composite permissions by selectively choosing a favorable witness?

  5. Could you provide at least nine records of strict success from GPT-5.5 involving "incomplete role" conditions, listing the actual relationship, QreqQ_{\mathrm{req}}, and QD,QV,QTQ_D, Q_V, Q_T? This would directly clarify the correspondence between Table D10 and the definitions.

  6. Did the held-out runs in Table G3 record task success rates, confirmation counts, and privilege reduction outcomes? If so, please report these by defense condition to determine whether the "zero unauthorized effects" result is accompanied by a significant loss in task completion performance.

Reviewer_p3RmOpenReview ↗
Rating
2
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The paper studies attacks where individually valid cues from different channels (DOM, screenshot, tool output) combine into an unauthorized tool call. It proposes CAF, a gateway that checks each call's full relation against a contract compiled from the user request. On the new C3 Bench, CAF cuts unauthorized effects from 7.7–21.5% to 0% across 5 models and keeps all authorized effects.

Critical strengths
  1. The evaluation is careful. It uses paired replay and exact confidence intervals. It also separates what the model proposes from what actually happens.

  2. The relation-gap test shows a real benefit. Each field passes its own check, but the combination is unauthorized.

Critical weaknesses
  1. The 0% result is largely expected by design. The contract copies allowed values directly from the task text. On the 96-episode subset, a simple Source/Field control matches CAF: no unauthorized effects in role-complete cases and no extra blocks.

  2. CaMeL, a closely related defense, is cited but not compared.

Questions for authors

How does CaMeL, or a similar policy-based defense, perform on the full C3 grid and on all AgentDojo suites?

3 comments

Dear reviewer p3Rm,

We appreciate your valuable comments and have addressed your concerns one by one. Since the rebuttal deadline is approaching, we would be very grateful if you could kindly let us know where further improvements are needed.

Best regards,

The authors

Dear Reviewer p3Rm,

We sincerely appreciate your valuable comments and have addressed each concern point by point.

As the rebuttal deadline is fast approaching, we would be grateful if you could kindly let us know whether our responses have resolved your concerns, or where further clarification and improvement are still needed. Even if only a few hours remain, we would do our best to respond to any additional comments you may have, at least in part.

Thank you again for your time and thoughtful feedback.

Best regards,

The Authors

Dear Reviewer p3Rm,

We sincerely appreciate your valuable comments and have addressed each concern point by point.

As only 4 hours remain before the rebuttal deadline, we would be grateful if you could kindly respond to our rebuttal.\color{blue}{\text{As only 4 hours remain before the rebuttal deadline, we would be grateful if you could kindly respond to our rebuttal.}} We would also do our best to respond to any additional comments you may have, at least in part (even if only a few hours remain).

Thank you again for your time and thoughtful feedback.

Best regards,

The Authors

Reviewer_QFD4OpenReview ↗
Rating
3
Confidence
2
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

A web agent can be pushed into unauthorized actions by combining inputs that arrive on different channels. Each field may look legitimate in isolation; the combination is what goes beyond what the user authorized. CAF builds a task contract from trusted inputs and checks the full semantic relation behind a proposed tool call instead of vetting each argument separately. At runtime it validates where the information came from, decides whether to allow, restrict, confirm, or block the call, and binds the approved arguments to execution. Proposing an action and authorizing it are deliberately kept apart.

Critical strengths
  • The weakness of per-field checking is stated plainly, and the alternative is concrete: authorize against the full relation among action, target, destination, and scope, not against a list of field values.
  • Task contracts, source validation, and execution binding sit inside one auditable pipeline, so a decision can be traced through to what the tools actually do.
  • Paired replay with frozen proposals and reset environments removes most of the run-to-run noise that muddies defense comparisons. The authors also report statistical uncertainty, separate proposed attacks from gateway decisions from realized effects, and are candid about where their compiler stops and how baselines were adapted.
Critical weaknesses
  • Evidence does not extend to long interactive tasks. Most results come from controlled cases with frozen proposals, and the WebArena runs stop before sensitive actions are dispatched. AgentDojo supplies closed-loop evidence, but its coverage of tasks and attacks is too narrow to support claims about extended interaction, shifting permissions, replanning, or user confirmation. Retaining effects authorized earlier is also not the same as completing whole tasks.
  • The contribution is mostly a system that assembles known authorization and enforcement mechanisms. Several comparisons route through mechanism adapters, so the numbers do not cleanly separate the value of the authorization design from differences in policy encoding, adapter implementation, and benchmark coverage.
  • Deployment costs are unmeasured. Registering complex tools, keeping their semantic specifications current, and handling omitted effects all take effort, and the appendix still leaves unprotected routing and parser fallback paths open. Testing against a stronger adaptive attacker matters as much.
Questions for authors
  • How do you ensure the comparison adapters receive equivalent policy information and express comparable constraints? Which results isolate the contribution of CAF's authorization design most cleanly?
  • Can you register more complex tools and report the manual setup effort, the maintenance burden, and the failures caused by incomplete specifications?
  • How does the full system hold up against a broader adaptive attacker, and how are the unprotected routing and parser fallback paths noted in the appendix clo
1 comment

Thanks for the additional clarification and evaluation. I acknowledge the results and will maintain my positive score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.