- Rating
- 2
- Confidence
- 2
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
The authors formulate the false-alarm auditing for safety monitors of LLM agents problem as positive-unlabeled learning. They use a small set of verified-safe non-alarmed trajectories to provide positive supervision. They explicitly aim to reduce the monitor-induced selection bias whereas the safe trajectories of interest are precisely those incorrectly flagged.
The solution is a series of enginerring heuristics: (1) a two-stage framework that first adapts known-safe supervision toward the alarm domain, (2) protects plausible false alarms during PU training, (3) distils ranking consensus from multiple PU models into a single student, and (4) further refines the resulting review queue using hierarchical safe-reference support and relations among alarms.
Multiple agent-safety benchmarks are used for evaluation. They also use 8 PU baselines, selection-bias and representation controls, and direct LLM re-judgment. In generl, the improvement over SOTA is significant.
Critical strengths
-
The paper identifies a meaningful limitation of agent safety monitors, where conservative monitoring produces many false alarms that require costly human review. Formulating false-alarm prioritization as PU learning is natural and useful.
-
The evaluation is strong. It covers multiple monitors and benchmarks, extensive PU baselines and controls, component ablations, and a larger external evaluation with 4,054 alarms.
Critical weaknesses
-
I have strong reservation about the key methodological choices that the author took. The central boundary-transfer mechanism assumes that moving verified-safe representations toward the alarm/non-alarm selection boundary provides useful supervision for safe-but-alarmed trajectories. How do we know this boundary is a safety boundary? This heuristic needs a lot more justification.Furthermore, how do we know if the transferred representations remain semantically safe or approximate the distribution of actual false alarms.
-
There are many works that address the bias issues in PU. The paper needs stronger comparison against methods designed specifically for biased positive selection to establish whether the proposed complex engineering pipeline is necessary. See for example: https://proceedings.neurips.cc/paper_files/paper/2023/file/3efb4bdc6bfe13e1ff95b4407c37961d-Paper-Conference.pdf
Questions for authors
-
Can the authors provide stronger justification that boundary transfer produces valid supervision for hidden safe alarms?
-
How does the proposed method compare with stronger recent approaches for biased PU learning beyond SAR-EM? For example: PUe (https://proceedings.neurips.cc/paper_files/paper/2023/file/3efb4bdc6bfe13e1ff95b4407c37961d-Paper-Conference.pdf) specifically develops normalized inverse-propensity-weighted PU learning for biased labeling.
6 comments
The Response to Reviewer Y3a3 — Weakness 2
- There are many works that address the bias issues in PU. The paper needs stronger comparison against methods designed specifically for biased positive selection to establish whether the proposed complex engineering pipeline is necessary. See for example: https://proceedings.neurips.cc/paper_files/paper/2023/file/3efb4bdc6bfe13e1ff95b4407c37961d-Paper-Conference.pdf
We added nnPUe, Dist-PUe, and their median and refinement variants; our method exceeds the strongest alternative by 5.30 AUPRC points and recovers 13.3% more false alarms at 5%.
| Method | Macro AUPRC (%) | FPs recovered at 5% |
|---|---|---|
| nnPUe, single model | 53.30 ± 0.25 | 12.33 |
| nnPUe, median | 53.31 | 12.00 |
| nnPUe, median + refinement | 53.62 | 14.00 |
| Dist-PUe, single model | 58.11 ± 0.12 | 17.67 |
| Dist-PUe, median | 58.22 | 17.00 |
| Dist-PUe, median + refinement | 59.14 | 20.00 |
| Our full method (original results) | 64.44 | 22.67 |
The comparison shares four settings, 40 verified-safe references per setting, training alarms, frozen features, and seeds. Enhanced baselines use their own PUe rankings and disagreement with the same refinement algorithm. PUe fits use positive references and training alarms, accessing target features during refinement; our method also uses unlabeled target features in Stage I. Target safety labels are excluded from fitting, prior estimation, and hyperparameter selection; priors use the benchmark's training-only BBE estimates.
Table details. AUPRC averages the four settings equally. Single-model AUPRC values are mean ± sample SD over three seeds; single-model recovery counts sum across settings and average over seeds. Each median variant is one three-model ensemble, with recovery counts summed across settings. The 5% budget reviews 27 alarms. PUe is independently implemented from the paper's equations, with documented frozen-feature adaptations to Dist-PUe weighting and propensity optimization; reported runs passed propensity-estimation checks.
The comparison directly tests a simpler biased-PU method combined with aggregation and structural post-processing. Its strongest variant remains below the full method in both ranking quality and false-alarm recovery.
The Response to Reviewer Y3a3 — Question 1
- Can the authors provide stronger justification that boundary transfer produces valid supervision for hidden safe alarms?
The direct evidence is the RAW comparison in W1, before either subsequent component is applied. Our direction with trust weighting reaches 61.16% AUPRC and 18 recovered false alarms, exceeding no transfer, matched random transfer, and full transfer. These controls test the usefulness of transfer, its direction, and its weighting. The separate student-to-refinement analysis establishes later correction; it is not used as evidence for transfer alone.
The Response to Reviewer Y3a3 — Weakness 1
This boundary provides an initial reference direction, not the true safety boundary. The monitor selects our verified-safe references from non-alarms, separating them from the safe false alarms we seek to recover. Without alarm safety labels, a precise safety boundary cannot be learned directly through target-domain supervision. We instead use the observable non-alarm/alarm difference to establish an initial direction, then improve the resulting ranking.
Transfer supplies an auxiliary comparison score; transferred points are not added as verified-safe samples or assumed to follow the false-alarm distribution. We preserve the separate task representation and hidden-representation norm, then mix the original and transferred scores through trust weights. These constraints limit the modification and its influence without assuming that geometry guarantees safety semantics.
These RAW rankings before distillation and refinement are a separate diagnostic, not the TAPS-only configuration in revised Table 4:
The controls test whether transfer helps, whether its direction matters, and whether trust weighting improves on full transfer. Recovery rises from 14 to 18 over no transfer (28.6%), establishing useful supervision before the later components contribute.
Initial errors are precisely why the subsequent distillation and refinement are needed. The student jointly learns from PU supervision and multiple reference rankings; structural refinement adds safe-reference support and alarm relations. In a separate retrospective diagnostic using the originally archived student and final predictions, we tracked changes within the 5% review prefix:
The 51.9% net reduction demonstrates correction at the student-to-refinement step. Thus, the RAW controls establish a useful initial direction, and the trajectory-level analysis shows that subsequent processing can correct ranking errors.