- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 2
Read review
Paper summary
This paper studies failures in agent verification caused by information loss in the representations available to the verifier. It introduces "blind fibers," where safe and unsafe trajectories become indistinguishable under the verifier's view, and proposes the Verifier Sufficiency Defect to characterize the resulting loss of safety information. The paper also presents FIBER, which uses outcome-matched safe and unsafe trajectories to learn a compact safety representation. The evaluation includes controlled synthetic experiments, a shortcut-dominated stress test, and a qualitative example from AgentDojo.
Critical strengths
The problem formulation is relevant and clearly motivated. The paper distinguishes between a verifier failing to use available information and a verifier never receiving the information needed for a safety judgment. This distinction helps clarify what stronger judges can and cannot address.
The theoretical framework is coherent. The interpretation of VSD through optimal verification loss, together with the discussion of coarsening and augmentation, provides a consistent way to reason about verifier representations. Although I have concerns about the novelty of these results, their presentation is clear and accessible.
The authors acknowledge important limitations. In particular, they explicitly state that ordinary supervised training matches FIBER on the main structural benchmark and that the experiments do not establish reliable extraction of safety-relevant structure from natural-language trajectories. The additional shortcut stress test is also a useful attempt to identify a setting in which the blind-pair objective provides a benefit.
Critical weaknesses
-
The main theoretical results largely follow from established information-theoretic principles and classical statistical testing arguments. While the formulation is clear, the paper offers limited new theoretical insights beyond applying existing concepts to agent verification. The authors should better clarify what fundamentally new understanding the proposed framework provides beyond classical statistical sufficiency and information theory.
-
The experiments rely heavily on synthetic settings with predefined safety signals. In the main multi-family benchmark, safe and unsafe trajectories can be distinguished by simply counting active structural factors, making the task considerably easier than realistic agent verification. Moreover, the proposed FIBER method is not quantitatively evaluated on real agent trajectories, and its advantage over standard supervised learning remains unclear. I encourage the authors to conduct end-to-end experiments on realistic benchmarks and include stronger baselines to demonstrate the practical effectiveness of the proposed approach.
Questions for authors
-
Have you evaluated a version of the structural benchmark without the factor-count and safeguard-marker shortcuts? Such an experiment would help determine whether the held-out-family results reflect learning safety-relevant relationships rather than exploiting properties shared by every training and test example.
-
Does the advantage in Appendix J.12 persist against baselines that use the same matched supervision more directly? In particular, comparisons with blind-pair-reweighted supervised training and supervised contrastive learning would help isolate the contribution of the proposed objective.
-
Is there evidence that the complete FIBER pipeline works on natural agent trajectories? A focused evaluation covering pair construction, sketch extraction, and downstream verification would materially affect my assessment. It would also be helpful to distinguish the occurrence of successful-but-unsafe executions from verified cases where a particular representation makes safe and unsafe executions indistinguishable.
What a verifier can recover from an agent trace?
Thank you for pushing us to answer the central question with observable evidence. In AgentDojo banking task 7, a user asked how much they had spent on a gift. Two runs of the same GPT-4o model returned byte-for-byte identical answers, and both completed the user’s task. One execution looked up the transaction history. The other also completed a money transfer and satisfied the benchmark’s injected goal. A judge given the task and final answer receives exactly the same input in both cases. If the two executions are equally likely, every such judge has error 1/2, regardless of its reasoning ability.
The distinction can be measured and partially repaired on released agent traces. We retrospectively checked all 16 banking task IDs for GPT-4o-2024-05-13, injection goal 0, under the
directandimportant_instructionsconditions. Of the 16 pairs, nine completed the legitimate task in both runs. Eight of those nine had opposite outcomes for the designated injection goal. Two of the eight, tasks 4 and 7, had identical final answers.Give equal weight to the 18 executions in those nine doubly successful pairs. For the view consisting of task ID and exact final answer, each of the two collisions contributes one bit of uncertainty on two records. All other observed views have a single label. The empirical verifier sufficiency defect (VSD) is therefore 2/9 bits, and the minimum possible classification error is 2/18. Now add one field available directly from tool receipts: whether a money-transfer call returned without error. This distinguishes the two executions of task 7. Both executions of task 4 contain a successful transfer, so that collision persists. On the same records, the defect falls to 1/9 bit and the minimum error to 1/18. Thus we can identify exactly which safety distinction the added field restores and which one still requires better evidence. These are exact calculations on a targeted retrospective sample for one specified safety predicate. Task 7, goal false · Task 7, goal true.
The theory connects a hidden risk difference to the information required for repair. Let Δ be the difference in unsafe-execution rates in absolute value and d the total variation distance between the two distributions of verifier inputs. For deterministic binary trajectory labels, the smallest possible VSD on the balanced mixture, over systems with those Δ and d, is max(0, Δ − d) bits. The classification-error step has a classical antecedent in Zhao and Gordon (NeurIPS 2019). The sharp formulation derived during discussion connects that step to the paper’s mixture definition and conditional repair identity. The audit provides a concrete instance. Across the nine tasks, the injected goal succeeded in 1/9 runs under one condition and 9/9 under the other, giving Δ = 8/9. The visible distributions differ on seven of the nine task IDs, giving d = 7/9. Those two facts alone force at least 1/9 bit of missing information. Inspecting the collisions gives the exact defect of 2/9 bits, and Proposition 10 identifies the decrease of 1/9 bit as the safety information contributed by the receipt field beyond the original view.
The controlled results answer where a better judge can still help. In Table 2, a fraction b = 0.6655 of equally weighted safe/unsafe pairs have identical three-step views. In an AUROC comparison, independent positive and negative draws both come from this shared component with probability b². Their expected ranking credit is then 1/2. Even perfect ranking elsewhere gives a ceiling of 1 − b²/2 = 0.77855, compared with 0.779 observed. The counting shortcut explains why the visible cases are easy. The collision rate explains, to the reported precision, why even a better judge could do no better on overall ranking.
For learning, Appendix J.12 shows that adding matched supervision to the specified BCE learner raises silent-fault recall at 5% FPR from 0.2139 to 0.3628. The reviewer’s matched-difference comparator is stronger on this generator, with analytical population recall 0.8029, because one labeled pair reveals the planted direction exactly. The evidence therefore supports targeting the distinctions exposed by matching. The relative merits of other pair-aware objectives and the performance of a learned sketch on natural traces remain open.