acceptodds
Under review as a conference paper at ICLR 2027

Better Prediction Can Make a Worse Alarm: Auditing Predictive Selection in LLM-Agent Safety Monitoring

Abstract

Runtime monitors support the safe deployment of LLM agents by warning of impending failures under strict false-alarm budgets. Candidate monitors are commonly benchmarked and selected using offline predictive metrics, but deployment evaluates them through sequential warning policies that turn predictions into alarms. This practice assumes that a monitor that predicts failures better offline will also make the better runtime alarm. However, whether that assumption holds remains largely unexamined. We formalize this question as predictive-to-policy ranking transfer, which measures whether predictive preferences between monitors are preserved under a specified warning rule and false-alarm budget. We audit ranking transfer for five monitors across three warning rules and four false-alarm budgets, with extensions across candidate families, agent models and environments. We find that a consistently better offline predictor can become the worse alarm, even when both rankings are highly stable. These reversals can be costly: in one low-false-alarm setting, stable reversals incur a median 30.4% relative loss in held-out recall. In contrast, selecting monitors by warning performance improves held-out recall by 2.2–5.8 percentage points while reducing achieved false-positive rates. We further show that this mismatch is structural. Standard predictive metrics summarize prediction quality across observations, while warning performance also depends on how scores behave over an entire execution. Reordering scores within an execution can preserve pointwise predictive summaries while substantially altering warning performance under rules that depend on temporal order. Thus, predictive metrics alone cannot establish which monitor makes the better alarm; ranking-transfer analysis makes the prediction-to-decision handoff explicit and tests whether offline comparative conclusions survive sequential realization.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
LLM agents, runtime monitoring, predictive evaluation, sequential decision-making

Reviews

Among:
Shape:
1234this paper, Top 30%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.67 over 3 reviews, above 70% of the others.
Reviewer_TtFPOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper is motivated by stating that monitor assessment based on offline predictive performance does not always translate into similar performance under actual deployment and monitoring constraints. This seeming mismatch in selection or assessment is measured via predictive-to-policy ranking transfer to see whether performance rankings remain consistent. Experiments are supposed to stress the fact that this is not the case, with empirical results suggesting performance mismatches, ranking disagreements, and effects on monitor recall or utility. The paper ends by observing that mean-based predictive performance does not take into account monitor score distributions (e.g. maxima that trigger monitors) or temporal dependencies.

Critical strengths
  • I think the fundamental premise that predictive performance and monitoring behavior need not necessarily align is correct, hence the wide range of utility-based and decision-rule research, or monitoring with statistical assurances (e.g. sequential testing, which they briefly mention).
Critical weaknesses
  • The contributions of the paper are unclear to me, or very minor at best. The stated contribution #1 is really about the experiment protocol, the stated contribution #2 noting "we formalize ranking transfer ... including stability and held-out analyses" is weak given that §3 formalizes the protocol to the extent of just stating a rank-matching count, while Fig. 1 is merely an anecdotal overview. I'm not sure what the formalized "stability" is referring to her. The stated contribution #3 on "structural insufficiency" and insights in §5 are, in my view, quite trivial. For instance, in §5.1 it is suggested that mean-based predictive performance does not take into account monitor score distributions (e.g. maxima that trigger monitors), and that threshold "calibration" with regards to FPR is computed on benign executions, hence does not imply maximum alignment or calibration on non-benign executions. Sure, and? I mean, as a user you pick your monitoring target and calibration goal, so this is not a "structural" limitation rather than it is a self-inflicted one. If you care to calibrate with respect to detection (here, recall), then do that instead. Similarly in §5.2 the case is made that a mean or maximum-based metric/monitor does not care about score ordering. Sure, and? Again, you define your monitor and you define your alarm policy. If you are sensible about designing these, these considerations should appear obvious. I don't really see the novelty or insight. More generally, it seems like a point is being made throughout that acting upon a prediction or estimate of a loss that directly relates to your target is better than acting upon some loosely related mean performance loss. What a surprise, aligning your measurement with the target of interest actually performs better!
  • Overall, the paper reads in large parts like a vague, hard to parse and meandering experimental protocol and design descriptions interspersed with experimental results, and mostly descriptive and limited in scope, lacking proper context, description, or metric and model definitions. Overall, it comes across as more of a motivational experiment that might lead into a novel proposal or attempt to address some of the raised concerns, but the paper merely "exposes" issues under somewhat ad-hoc design choices, with limited insights, a discussion in §6 that comes across more like an executive summary, and no limitations or outlook.

Examples:

  • In §3 the notation is all over the place. Are execution ee, a "query", and instance rr the same? Why use Mc(j)M_c(j) and later wcr(a,b)w^r_c(a,b) to denote what I assume is its maximum, or how do they relate? rule π\pi is supposedly calibrated for each monitor, so π(j)\pi(j)? But later jj becomes aa or bb?
  • A part of the main protocol is described as "grouped resampling at the natural dependence unit" (L185) without further explanation. Grouped with respect to what, which unit?
  • The key monitor candidates are only described as "five hidden-state telemetry monitors" with nondescript names d1,d2,d4,d8,d16d1, d2, d4, d8, d16 (L200), I have no idea what that means. Different hidden layers of the model? Same hidden layers with varying capacity, effectively multiple models? Are these compared to at the same temporal occurrence in the model internals or different layers; the latter might then be an unfair comparison because preceding processing steps differ.
  • L221 talks about 50 "fold-pair comparisons" then L229 mentions 60 "fold-policy comparisons", then later L231 says 30 comparisons. What are all those supposed to mean is unclear to me.
  • L228-232 talks about d16d16 but adjacent Fig. 2 is about d1d1 and d8d8.
  • §4.3, §4.4 were also very unclear to me in terms of experimental design and description. In Fig. 4A, how are all the different y-axis metrics like recall regret defined?
Questions for authors
  • See above
  • In the paper the drawback is motivated as a performance change with respect to models or monitors but the metric I^c\hat{I}_c is a sum over pairwise model comparisons as well, so it is ultimately a statement about a given policy cc rather than the monitors themselves?
  • In §4.1, you state that picking the different monitors as essentially variants of the same monitoring design is supposed to "avoid confounding" (L203). But ranking metrics are agnostic to the underlying monitor design, which can be treated as a black-box, so I think the stronger point to make would be to employ a more heterogenous mix of designs to stress monitoring and prediction mismatches and introduce more design variety. The choice of just 5x the same monitor and just a small dimension change seems lazy to me, more than anything. They are also presumably very correlated, Tab C17 seems to imply that and shows that growing capacity implies better performance, which is expected but might also confound some of the trends observed.
  • In L240, you note that failure recall is at <10%, which is essentially really bad? It implies that you mostly just observe false negatives (missed detections), hence your monitor either never raises alarms or always raises alarms on the wrong samples?
  • In §4.3, I did not understand the implication of the key takeaway. If the point is that ranking agreement worsens with a looser FPR target, why is that? Rather than only describing the result, provide some interpretation to the reader. Intuitively, I would expect a looser target to permit more flexibility since it constitutes a weaker monitoring requirement, and hence perhaps more likely agreement/alignment with predictive metrics. Hence Fig. 4A where recall regret seems to be smaller for looser FPR of 10% seems more in line with my thinking, whereas Fig. 3 is all about the reverse ranking effects and implies strong downstream consequences which is unclear to me.
  • The paper keeps referring to "calibration" but it is unclear what is meant by that. If this implies simply grid-searching on holdout validation data for a performance that empirically "matches" the FPR target, this is not proper calibration, nor does it imply straightforward transfer to deployment data without explicit assumptions. A much smarter way to achieve such a goal would be something like statistical risk control or sequential testing, briefly touched on in related works.
4 comments
AuthorsOpenReview ↗

Response to Reviewer TtFP(1/4)

We thank the reviewer for the detailed feedback. We have revised the paper to clarify its research question, formal definitions, experimental design, and the interpretation of its structural analyses. We also added experiments and expanded the discussion of limitations and future directions. We address the concerns below.

W1. Novelty and substantive insight

"The contributions of the paper are unclear to me, or very minor at best."

"The insights in §5 are, in my view, quite trivial."

We thank the reviewer for raising this fundamental question about the paper's substantive contribution. Our central contribution is not simply observing that predictive metrics and downstream decisions can disagree, but systematically establishing when predictive rankings fail to transfer to sequential monitor selection, whether these failures are reproducible, and whether they affect actual selection outcomes.

First, we distinguish ranking disagreement from stable preference reversal. Across five monitors, three warning policies, and four FPR operating points, we evaluate predictive and policy preferences on separate data roles. We then use scenario-group resampling to assess the directional stability of each preference, defining jointly stable comparisons as those in which both preferences achieve at least 0.90 stability. Among 20 jointly stable comparisons at both endpoint operating points, agreement decreases from 17/20 at 1% FPR to 2/20 at 10% FPR. Thus, the disagreement is not simply due to noisy rankings. More importantly, among 156 development-stable opposed comparisons, policy-based selection achieves higher held-out recall in 95 cases, including 40 without any FPR increase. This connects stable ranking reversals to prospective selection consequences.

Second, Section 5 goes beyond demonstrating that score distributions and temporal order matter: it examines whether these properties help explain naturally occurring reversals. For first crossing, episode-level ROC preference agrees with policy preference in 263/316 stable development comparisons (83.2%), substantially more often than pointwise NLL (154/316, 48.7%). Feasibility alone determines only 15/156 stable-opposed comparisons, showing that the reversals are not simply consequences of failing the FPR constraint.

For order-sensitive policies, our same-label permutation preserves the target sequence, event timing, pointwise score–label relationships, episode maxima, and calibrated thresholds, while changing temporal order. At the 5% FPR target, recall changes by 0, −2.7, and −6.0 pp for first crossing, two consecutive, and cumulative window-3, respectively. Crucially, we further connect this controlled effect to natural reversals: stable-opposed comparisons exhibit greater temporal sensitivity than stable-aligned comparisons for two consecutive (+1.10 pp development; +0.77 pp held-out) and cumulative window-3 (+4.39 pp; +4.48 pp), with all four confidence intervals excluding zero. First crossing remains invariant, as expected. This links controlled structural evidence to the actual ranking mismatches observed in the evaluation.

Finally, directly optimizing recall does not bypass the problem we study. Our policy-based selection already optimizes event recall under a common FPR constraint, using thresholds calibrated on separate non-event episodes. The question is whether offline predictive metrics can reliably substitute for this deployment-aligned selection criterion, not whether recall should be optimized instead.

In summary, our contribution is to establish that predictive preferences can be individually stable yet systematically misaligned with sequential-policy preferences, quantify the resulting held-out selection consequences, and identify score-process properties associated with naturally occurring reversals.

AuthorsOpenReview ↗

Response to Reviewer TtFP(2/4)

W2. Methodological clarity and experimental protocol

"Overall, the paper reads in large parts like a vague, hard to parse and meandering experimental protocol and design descriptions interspersed with experimental results..."

"...lacking proper context, description, or metric and model definitions."

We thank the reviewer for identifying aspects needing clarification. The study progresses from ranking agreement to stability, held-out selection consequences, and structural analysis of natural reversals. We have clarified the methodology and expanded supporting analyses, as summarized below.

(1) Notation and experimental units

"In §3 the notation is all over the place. Are execution, a 'query', and instance the same?"

Section 3 now explicitly defines rr (outer fold), ee (execution/episode), qq (prespecified future-event query), and tt (eligible prefix position). An execution is the evaluation episode, whereas a query specifies the future event being predicted. We also define the candidate score sj,e,ts_{j,e,t}, warning configuration c=(π,α)c=(\pi,\alpha), and candidate-specific threshold θj,cr\theta^r_{j,c}.

(2) Grouped resampling and stability

"A part of the main protocol is described as 'grouped resampling at the natural dependence unit' without further explanation. Grouped with respect to what, which unit?"

Resampling is now specified at the scenario/task-group level, retaining all episodes within each sampled group. We also distinguish frozen-realization stability, which fixes candidate pairs and thresholds, from calibration-aware stability, which additionally resamples selection and policy-calibration groups, recalibrates thresholds, and reapplies the selection rule.

(3) Monitor construction and comparability

"The key monitor candidates are only described as 'five hidden-state telemetry monitors' with nondescript names... Different hidden layers of the model? Same hidden layers with varying capacity?"

Section 4.1 specifies that all five GRU candidates share the same hidden-state information, extraction position, fixed 16-dimensional PCA projection, and prefix-causal behavioral features. The hidden-state representation averages backbone layers 24 and 28 at the last-observed-token position. Only GRU bottleneck width varies, isolating representation capacity while holding other components fixed.

(4) Experimental counts and data separation

"L221 talks about 50 'fold-pair comparisons' then L229 mentions 60 'fold-policy comparisons', then later L231 says 30 comparisons. What are all those supposed to mean?"

The three counts refer to distinct experimental units:

  • 50 fold–pair comparisons: five folds × ten candidate pairs per warning-rule–FPR condition.
  • 60 top-1 conditions: five folds × three warning rules × four FPR targets.
  • 30 top-1 conditions: five folds × three warning rules × the two 5% and 10% targets.

We also added a 600-cell pairwise audit in Section 4.4, of which 316 satisfy the joint stability criterion (156 stable-opposed and 160 stable-aligned).

The five outer folds and four development roles are also clarified: monitor fitting, selection calibration, selection evaluation, and policy calibration, including non-event threshold calibration and threshold freezing before held-out evaluation. The CUSUM exception is documented in Appendix A.3.

(5) Metric definitions and figure interpretation

"L228-232 talks about [the predictive and episode-level quantities] but adjacent Fig. 2 is about [different quantities]."

"§4.3, §4.4 were also very unclear to me in terms of experimental design and description. In Fig. 4A, how are all the different y-axis metrics like recall regret defined?"

We clarify the reported metric directions. Figure 2A shows NLL(d1)−NLL(d8)NLL(d1)-NLL(d8), where positive values favor d8; Figure 2B relates failure-episode score maxima to candidate-specific warning boundaries, with boundary survival corresponding to first-crossing recall.

Figure 4 defines the held-out differences:

ΔR=Rpolicy−Rpredictive,\Delta R=R_{\mathrm{policy}}-R_{\mathrm{predictive}},

ΔFPR=FPRpolicy−FPRpredictive.\Delta FPR=FPR_{\mathrm{policy}}-FPR_{\mathrm{predictive}}.

Both are in percentage points. Recall regret is the held-out recall shortfall of predictive-based selection, and cost-sensitive utility is Uλ=Recall−λFPRU_\lambda=\mathrm{Recall}-\lambda FPR.

(6) Discussion, limitations, and future directions

"...a discussion in §6 that comes across more like an executive summary, and no limitations or outlook."

We have added a dedicated discussion of scope, limitations, and future directions in Section 6. It addresses the controlled nature of the primary GRU family, the distinction between controlled score rearrangements and actual causal monitor outputs, and the need for evaluation across more independent scenarios. Future directions include broader supervision objectives, additional warning policies, and monitor–policy co-design.

AuthorsOpenReview ↗

Response to Reviewer TtFP(3/4)

Q1. Interpretation of the ranking-transfer estimand

"The metric is a sum over pairwise model comparisons as well, so it is ultimately a statement about a given policy rather than the monitors themselves?"

Thank you for raising this distinction. Ranking transfer is indeed conditional on the warning policy, operating point, and candidate family, rather than an intrinsic quality measure of an individual monitor. This reflects our research question: whether predictive preferences provide reliable evidence for selecting monitors under a specified deployment policy, rather than whether monitors have a universal warning ranking independent of their deployment context.

Specifically, the estimand compares two orderings of the same candidate monitors: their predictive ordering and their policy-induced ordering. The warning rule and FPR target define the evaluation context because each candidate is independently calibrated to the same nominal FPR, and its resulting warning performance depends on how its score sequence interacts with the specified policy. The policy defines the evaluation context, while the monitors remain the objects being compared.

Importantly, we also examine whether ranking disagreements affect actual monitor selection. Across the 60 fold–rule–operating-point conditions, predictive and policy-based top-1 selections differ in 48 cases, including all 30 conditions at the 5% and 10% FPR targets. Among 156 development-stable opposed comparisons, policy-based selection achieves higher held-out recall in 95 cases, including 40 without an increase in FPR (Section 4.4; Tables C18–C19).

Thus, ranking transfer measures the reliability of predictive evidence for monitor selection under a specified deployment policy. It connects candidate-level predictive evaluation to the actual sequential warning system in which the selected monitor will be deployed.

Q2. Candidate-family dependence and monitor diversity

"The choice of just 5x the same monitor and just a small dimension change seems lazy to me..."

Thank you for asking whether ranking mismatch generalizes beyond the primary candidate family. The five GRU variants were deliberately designed as a controlled capacity comparison, holding the input construction, prediction target, and warning interface fixed while varying representation width. To examine broader generalization, we have added experiments along several complementary dimensions:

DimensionAdditional experiments
Monitor diversityMixed candidate family with a standalone behavior-feature monitor; heterogeneous monitors including GRU, behavior-only, TF-IDF, and Qwen3-8B LLM judge
Predictor architecturesLinear, MLP, GRU, and causal Transformer
Agent models and environmentsASB/Qwen2.5-14B; AgentLab with Qwen2.5-7B, Qwen2.5-14B, Llama 3.1 8B, and DeepSeek-R1/Llama 8B; AgentCanary/Qwen2.5-14B
Prediction targets, aggregation, and warning policiesPrediction horizons h∈{1,2,4}h\in\{1,2,4\}; PRM-style raw-product and length-normalized-product aggregation; recursive CUSUM-style warning policy

Overall, these extensions show that incomplete predictive-to-policy ranking transfer extends beyond the original five-variant GRU family, including heterogeneous monitors, alternative predictor architectures, and different agent evaluation settings.

The most direct evidence comes from the mixed and heterogeneous monitor families. In the mixed-family experiment, pairwise agreement decreases from 39/50 to 11/50 for first crossing, 31/50 to 13/50 for two consecutive, and 33/50 to 8/50 for cumulative window-3 when moving from 1% to 10% FPR. Policy-based selection also improves mean held-out recall by 2.2–5.8 pp while reducing mean FPR across all six conditions. In the heterogeneous-monitor experiment, top-1 selections disagree in 47/60 conditions, and policy-based selection achieves higher recall in 14/16 stable-opposed held-out comparisons, including five involving Qwen3-8B.

The remaining extensions provide complementary evidence of the same phenomenon. Predictor architectures exhibit top-1 disagreement in 32/60 conditions; prediction horizons h=1,2,4h=1,2,4 yield 40/60, 41/60, and 48/60 mismatches, respectively. Under PRM-style aggregation, policy-based selection achieves higher held-out recall in 16/20 raw-product and 21/23 length-normalized-product comparisons. Across agent environments, AgentLab agreement ranges from 11/50 to 34/50, while AgentCanary yields 27/50–30/50. The recursive CUSUM policy also exhibits decreasing agreement, from 33/50 at 1% to 10/50 at 10% FPR.

Because these extensions involve different candidate families, prediction tasks, and evaluation settings, we report them separately. Together, they demonstrate that ranking mismatch is not specific to representation-width variations within a single GRU family, but also arises across substantially more diverse monitors and deployment settings.

AuthorsOpenReview ↗

Response to Reviewer TtFP(4/4)

Q3. Low absolute recall

"In L240, you note that failure recall is at <10%, which is essentially really bad?"

Thank you for distinguishing absolute detection from monitor-selection consequences. The recall in the d1–d8 example is indeed low, reflecting limited separation between failure-episode maxima and the calibrated warning boundary. At the 5% FPR target, d1 achieves 8.66% recall versus 7.11% for d8, despite nearly identical held-out FPRs (4.84% vs. 4.87%). The 1.55 pp advantage persists across all 100 locked policy splits.

The broader analysis shows that the selection cost extends beyond this example. Among 156 development-stable opposed comparisons, policy-based selection achieves higher held-out recall in 95 cases, with mean and median gains of 2.91 and 2.31 pp. Importantly, 40 of these improvements incur no FPR increase. At the first-crossing 2% target, all nine stable-opposed comparisons also show positive held-out recall regret, averaging 3.93 pp.

Thus, limited absolute detection and avoidable selection regret are distinct issues. The results show that predictive-only selection can forgo better available recall under the same warning constraint, sometimes without additional false-positive cost.

Q4. Operating-point dependence of ranking agreement

"If the point is that ranking agreement worsens with a looser FPR target, why is that?"

Thank you for highlighting the need to explain this pattern and its relationship to downstream consequences.

A looser FPR target does not necessarily improve ranking agreement. Each candidate has its own score distribution and independently calibrated threshold. For first crossing, recall is determined by the fraction of failure-episode maxima exceeding that threshold:

Rj(α)=P(Zj,e≥θj(α)∣e is a failure).R_j(\alpha)=P(Z_{j,e}\geq\theta_j(\alpha)\mid e\text{ is a failure}).

As the FPR target increases, candidate-specific thresholds traverse different regions of their non-event distributions. Combined with different failure-episode tail distributions, this can cause recall curves to cross. Order-sensitive policies additionally depend on temporal score organization.

Ranking agreement and recall regret measure different consequences. Figure 3 measures how frequently preferences disagree, whereas Figure 4A measures the magnitude of the resulting recall difference. Thus, more frequent reversals can coexist with smaller recall regret at more permissive operating points.

This pattern is evident in development: agreement among the 45 largest-margin comparisons decreases from 36/45 at 1% to 5/45 at 10%, and among 20 highly stable comparisons from 17/20 to 2/20. Held-out agreement is non-monotonic: 38/45, 28/45, 19/45, and 24/45 at 1%, 2%, 5%, and 10%, respectively. Paired scenario-group resampling supports the endpoint difference across all three warning policies.

The operational consequences also vary by operating point. At the first-crossing 2% target, all nine stable-opposed comparisons incur positive held-out recall regret, averaging 3.93 pp, with a median relative loss of 30.4% (Table C19).

Together, these findings show that ranking agreement, recall regret, and operating-point permissiveness capture distinct aspects of monitor selection, with neither ranking agreement nor recall regret following directly from the nominal FPR target.

Q5. Threshold calibration and statistical guarantees

"If this implies simply grid-searching on holdout validation data for a performance that empirically 'matches' the FPR target, this is not proper calibration..."

Thank you for raising this distinction. Calibration in our study means empirical, candidate-specific threshold selection for a nominal episode-level FPR target, rather than probabilistic calibration or formal statistical risk control.

The data separation is specified in Section 4.1 and Appendices A.3 and A.5. For the three primary policies, development thresholds are selected using non-event episodes from a dedicated selection-calibration split and evaluated on a disjoint selection-evaluation split. Held-out thresholds are independently calibrated on a separate policy-calibration split and frozen before evaluation on unseen outer-test groups. The CUSUM extension follows its separately specified protocol, with μ0\mu_0 and hh derived from policy-calibration data.

The nominal target is also distinct from achieved FPR. The feasibility criterion FPR^≤α+0.02\widehat{\mathrm{FPR}}\leq\alpha+0.02 is an operational selection tolerance, not a statistical guarantee on population FPR; development and held-out FPRs are reported separately (Appendix A.1).

Our analysis examines ranking transfer under this data-separated empirical threshold-selection protocol. Formal FPR guarantees under deployment require additional distributional assumptions and dedicated statistical risk-control or sequential testing procedures, which address the complementary objective of guaranteeing false-positive behavior.

Reviewer_FK55OpenReview ↗
Rating
3
Confidence
3
Informativeness
2
Presentation
3
Soundness
2
Read review
Paper summary

The paper examines if offline predictive rankings of agent monitors remain effective when turned into sequential alarms. It checks how well predictive and policy rankings agree, assesses their stability, and looks at the impact of selection on different scenario groups. The main study compares five telemetry monitors, three warning rules, and four false-alarm budgets, with additional tests on other options, policies, models, and environments. The results show stable changes in ranking order and decreases in recall for predictive selections. Further tests reveal that while specific metrics might stay the same, the order of warnings can still change.

Critical strengths
  • The empirical audit highlights several distinct questions that are often confused: predictive superiority, the stability of each preference, and the consequences of downstream selection.

  • The structural analysis uses a label-preserving shuffle and an intact-path negative control. This approach clearly shows the information lost by evaluating each point individually.

Critical weaknesses
  • The candidate family is a related limit. Five matched variants of one telemetry construction is a capacity ablation, not five monitors; the selection-consequence results come from a "mixed" family that adds exactly one behavior-feature monitor. Whether this generalizes is open, and the breadth extensions are reported as setting-specific with no universal pattern.
Questions for authors
  • How large are the absolute predictive margins across the five candidates? Fig. 2A shows the d1–d8 NLL gap is real, but if the five-way differences are otherwise tiny, the reversals are less surprising — and less consequential — than presented.

  • Do you expect the mismatch to look the same for PRM-style monitors, where per-step scores are combined by min or product into a halt decision?

2 comments
AuthorsOpenReview ↗

# Response to Reviewer FK55 (1/2)

We thank the reviewer for the thoughtful and constructive assessment. We particularly appreciate the recognition that the paper distinguishes predictive superiority, preference stability, and downstream selection consequences, as well as the value of the label-preserving shuffle and intact-path control in isolating information not captured by pointwise predictive metrics. The questions also helped us sharpen the scope of our claims and extend the evaluation beyond the original candidate family. We address each point below.

W1. Diversity of the candidate family

"The candidate family is a related limit. Five matched variants of one telemetry construction is a capacity ablation, not five monitors; the selection-consequence results come from a 'mixed' family that adds exactly one behavior-feature monitor. Whether this generalizes is open, and the breadth extensions are reported as setting-specific with no universal pattern."

We appreciate this distinction and agree that the primary five-candidate family is a controlled capacity comparison, not a collection of architecturally distinct monitors. Its candidates share the same input construction and warning interface and differ only in GRU bottleneck width. This design isolates representation capacity, but alone cannot establish generalization across monitor architectures.

One clarification is important: the 156-cell held-out selection-consequence analysis in Section 4.4 uses the primary five-candidate family. The mixed candidate family is a separate extension that adds a prefix-causal behavior-feature monitor to d1, d4, d8, and d16.

To examine the broader scope, we have added experiments along several complementary dimensions:

ExtensionMain findings
Mixed candidate familyPolicy-based top-1 selection improves mean held-out recall by 2.2–5.8 pp, with lower mean FPR in all six policy–target conditions.
Heterogeneous monitorsGRU, behavior-only, TF-IDF, and Qwen3-8B judge monitors exhibit top-1 disagreement in 47/60 conditions. Policy-selected recall is higher in 14/16 stable-opposed held-out records.
Agent models and environmentsIncomplete transfer also occurs across AgentLab and AgentCanary, with agreement varying across model–policy settings.
Predictor architecturesLinear, MLP, GRU, and causal Transformer predictors exhibit top-1 disagreement in 32/60 conditions.
Prediction horizons and warning policiesTop-1 disagreement occurs in 40/60, 41/60, and 48/60 conditions for horizons h = 1, 2, and 4, respectively; a recursive CUSUM extension also exhibits incomplete transfer.

The heterogeneous-monitor extension is particularly relevant to the concern about monitor diversity. It includes an LLM-based Qwen3-8B judge, and policy-based selection achieves higher held-out recall in 14/16 stable-opposed records, including all five Qwen3-8B-related records. In these five records, recall gains range from 10.1 to 31.6 pp.

We also preserve an important distinction in interpreting the extensions: the cross-architecture experiment exhibits top-1 disagreement, but does not produce stable-opposed cells at our 0.90 stability threshold. We therefore do not present it as evidence of stable held-out reversals.

We agree that these experiments do not establish a universal transfer pattern. Rather, they show that incomplete transfer is not confined to the original matched GRU family, while its magnitude and operating-point profile depend on the monitor construction, prediction task, warning policy, and evaluation setting. We report these extensions separately in Appendices C and E rather than pooling heterogeneous settings into a single generalization claim.

AuthorsOpenReview ↗

Response to Reviewer FK55 (2/2)

Q1. Magnitude of predictive margins

"How large are the absolute predictive margins across the five candidates? Fig. 2A shows the d1–d8 NLL gap is real, but if the five-way differences are otherwise tiny, the reversals are less surprising — and less consequential — than presented."

This question highlights the distinction between the magnitude of predictive separation and the consequences of monitor selection. We added a complete margin analysis in Appendix B.3. Across 50 fold–pair records, the median absolute scenario-equal NLL gap is 0.00519 (IQR 0.00537; range 0.00029–0.02147). These records are descriptive comparisons, as folds reuse scenarios and candidates.

Importantly, stable-opposed and stable-aligned comparisons exhibit highly comparable predictive-margin distributions:

Comparison populationCellsMedian absolute NLL gapIQR
Stable-opposed1560.00589[0.00346, 0.01008]
Stable-aligned1600.00563[0.00377, 0.00951]

This similarity shows that stable ranking reversals occur across the same range of predictive-margin magnitudes as stable-aligned preferences. The reversals therefore reflect a mismatch between predictive and policy-induced candidate orderings, rather than a concentration of comparisons with exceptionally small predictive margins.

We further assess the consequences on held-out data. Among the 156 stable-opposed comparisons, policy-based selection achieves higher recall in 95 cases, with a median gain of 2.31 pp among these improvements. Importantly, 40 of the 95 improvements incur no FPR increase, while the remaining 55 involve a recall–FPR trade-off.

Together, these findings establish that predictive-margin magnitude alone is insufficient to characterize monitor-selection reliability. Comparisons with similar predictive margins can exhibit either aligned or opposed preferences, and the resulting selection differences can translate into measurable held-out recall gains.

Q2. PRM-style score aggregation

"Do you expect the mismatch to look the same for PRM-style monitors, where per-step scores are combined by min or product into a halt decision?"

Thank you for raising this question, which helps assess whether the observed mismatch extends to alternative score aggregation mechanisms. We added a PRM-style aggregation analysis in Appendix E.5.

Under our complementary-score convention, minimum aggregation is algebraically equivalent to first crossing. Specifically, if qtq_t denotes the per-step safety probability and st=1−qts_t=1-q_t the corresponding risk score, then min⁡tqt≤τ\min_t q_t\leq\tau is equivalent to max⁡tst≥1−τ\max_t s_t\geq 1-\tau.

To examine distinct aggregation mechanisms, we additionally evaluate raw-product and length-normalized-product aggregation. Their pairwise agreement profiles are:

FPR targetRaw productLength-normalized product
1%29/5026/50
2%17/5030/50
5%13/5031/50
10%11/5018/50

Both aggregations also exhibit stable-opposed held-out comparisons:

AggregationStable-opposedPolicy-selected recall higherMedian held-out FPR gap
Raw product2016/20−0.61 pp
Length-normalized product2321/23+0.13 pp

Incomplete ranking transfer also extends to product-based PRM aggregation, with policy-based selection achieving higher held-out recall in 16/20 raw-product comparisons and 21/23 length-normalized-product comparisons. The associated FPR trade-offs vary across aggregation rules.

Together, these findings show that predictive preferences can diverge from monitor preferences induced by sequential score aggregation across different PRM-style mechanisms. The extent of this mismatch depends on how per-step scores are accumulated, demonstrating the importance of evaluating candidate monitors under their intended aggregation and deployment rules.

Reviewer_HsAtOpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
3
Read review
Paper summary

The paper shows that the predictor that predicts failure best is not the one that gives the best alarms. And if the system only picks the monitor by the prediction score, it would actually make the system worse.

Critical strengths
  1. I deeply appreciate the author's thorough experimental design and comprehensive evaluation, which includes various hidden-state monitors, warning policies, and contains cross-validation, threshold calibration, held-out evaluation.
  2. Also, I am impressed by the empirical evidence and, most importantly, the insightful structural analysis. Rearranging scores within an execution changes the performance of order-sensitive warning rules while leaving every pointwise predictive summary unchanged, which establishes that such summaries cannot determine warning performance.
Critical weaknesses
  1. I think the work has limited novelty and incremental contribution compared with decision-focused learning and sequential monitoring in the related works.
  2. The empirical findings in the paper focus mainly on five monitors that differ only in representation dimension, with a fixed prediction target. We are not clear that whether the findings generalize to monitors with different supervision targets or LLM-based judges.
  3. The practical significance of the reversals is unclear. Both monitors in the main example catch fewer than one in ten failures, and Table C19 shows that the 30.4% relative loss corresponds to a mean absolute loss of about four percentage points, with no setting exceeding 4.2 points. Even for the d1–d8 pair, the tail advantage used to illustrate the first-crossing mechanism is not uniform across folds (Table D1), and the paper does not relate the other reversals in Section 4 to either mechanism.
Questions for authors
  1. Beyond the d1–d8 example, which mechanism in Section 5 accounts for the reversals in Section 4?
  2. Under FPR matching at ε = 0.010, held-out agreement at the 10% target is 9/11, against 1/11 in development. Does this mean that most development-stage reversals at this target do not hold on unseen groups?
  3. Table C8 shows that pairs matched at the selection stage still differ by about 0.02 in held-out FPR. Could the authors report the held-out FPR gaps for the stable opposed pairs in Table C19?
4 comments
AuthorsOpenReview ↗

Response to Reviewer HsAt(1/3)

We thank the reviewer for the helpful comments. We have revised Sections 3–5 and added several analyses to address the concerns about novelty, generalization, practical significance, and the mechanisms behind ranking reversals. Our responses are below.

W1. Novelty

"I think the work has limited novelty and incremental contribution compared with decision-focused learning and sequential monitoring in the related works."

Thank you for raising the question of how our contribution differs from these closely related research directions. Although decision-focused learning, sequential monitoring, and our work share the broader goal of improving downstream decision and warning performance, they address different stages of the model-to-deployment pipeline.

Decision-focused learning primarily studies how to optimize models for downstream objectives, while sequential monitoring studies how to convert evolving model outputs into timely warnings under false-alarm constraints. A distinct question remains between these stages: once multiple candidate monitors have been trained, can their offline predictive performance reliably guide monitor selection for sequential deployment? The answer requires examining whether candidate rankings are preserved when predictive scores are converted into warnings under different policies and FPR operating points.

Our work addresses this selection problem through three contributions:

  1. Characterizing ranking transfer: We systematically evaluate predictive-to-policy ranking agreement across candidate monitors, warning rules, and FPR operating points.

  2. Establishing ranking stability: We show that predictive and policy preferences can each be highly stable while favoring different candidates. Among 20 comparisons satisfying the stability criterion at both endpoint FPR targets, agreement decreases from 17/20 at 1% to 2/20 at 10%.

  3. Quantifying consequences and investigating mechanisms: Among 156 development-stable opposed comparisons, policy-based selection achieves higher held-out recall in 95 cases, including 40 without an FPR increase. We further identify the roles of episode-level score extremes and temporal organization in explaining policy-dependent ranking changes.

By making monitor-selection reliability an explicit object of study, our work connects offline predictive benchmarking with deployment-time warning performance, providing complementary insights into model selection between training and sequential deployment.

W2. Generalization

"The empirical findings in the paper focus mainly on five monitors that differ only in representation dimension, with a fixed prediction target. We are not clear that whether the findings generalize to monitors with different supervision targets or LLM-based judges."

Thank you for raising this important question. The original submission was limited in experimental scope due to time and space constraints, and did not fully reflect the breadth of our evaluation. In the revised version, we have added experiments covering several dimensions:

DimensionAdditional experiments
Monitor diversityMixed candidate family with a standalone behavior-feature monitor; heterogeneous monitors including GRU, behavior-only, TF-IDF, and Qwen3-8B LLM judge
Predictor architecturesLinear, MLP, GRU, and causal Transformer
Agent models and environmentsASB/Qwen2.5-14B; AgentLab with Qwen2.5-7B, Qwen2.5-14B, Llama 3.1 8B, and DeepSeek-R1/Llama 8B; AgentCanary/Qwen2.5-14B
Prediction and aggregationPrediction horizons h∈{1,2,4}h\in\{1,2,4\}; PRM-style raw-product and length-normalized-product aggregation; recursive CUSUM policy

The new experiments show that ranking transfer also occurs beyond the original five-monitor family. In particular, the heterogeneous-monitor experiment exhibits top-1 disagreement in 47/60 conditions, with policy-based selection achieving higher held-out recall in 14/16 stable-opposed records, including five involving Qwen3-8B. The mixed candidate family shows agreement decreasing from 39/50 to 11/50, 31/50 to 13/50, and 33/50 to 8/50 across the three warning policies when moving from 1% to 10% FPR. The CUSUM extension shows a similar decrease from 33/50 to 10/50. We also observe top-1 disagreement in 32/60 conditions across predictor architectures, 40/60, 41/60, and 48/60 conditions across the three prediction horizons, and stable-opposed held-out comparisons under both PRM aggregation methods (16/20 and 21/23). Additional AgentLab and AgentCanary experiments likewise show incomplete transfer across agent models and environments.

We report these extensions separately in Appendices C and E.

AuthorsOpenReview ↗

Response to Reviewer HsAt (2/3)

W3. Practical significance and mechanisms of ranking reversals

1. Practical significance of the recall loss

"Both monitors in the main example catch fewer than one in ten failures..."

"Table C19 shows that the 30.4% relative loss corresponds to a mean absolute loss of about four percentage points, with no setting exceeding 4.2 points."

Thank you for highlighting the distinction between relative and absolute recall losses. The absolute regret in the primary five-GRU family is indeed modest (at most 4.2 pp), but broader extensions reveal substantially larger selection consequences.

The mixed-candidate extension (Table C23) provides a direct comparison beyond the primary family. Across six policy–target conditions at 5% and 10% FPR, policy-based selection improves mean held-out recall by 2.17–5.84 pp while simultaneously reducing mean FPR in all six conditions. For cumulative window-3 at 10%, recall improves by 5.84 pp and FPR decreases by 3.21 pp, with recall gains in all five outer folds.

Larger differences also emerge under other settings. For common-support h=4h=4, policy-based selection achieves higher held-out recall in 15/17 stable-opposed comparisons, with a median absolute difference of 13.79 pp. In the heterogeneous-monitor extension, it achieves higher recall in 14/16 comparisons, including 10.1–31.6 pp gains in the five Qwen3-8B-related comparisons (Appendices E.2–E.4).

These results show that the practical selection cost can substantially exceed the regret observed in the primary family, including cases where policy-based selection improves both recall and FPR.

2. Fold-level consistency of the first-crossing mechanism

"Even for the d1–d8 pair, the tail advantage used to illustrate the first-crossing mechanism is not uniform across folds (Table D1)."

Fold-level variation reflects heterogeneity in episode-maximum distributions and candidate-specific thresholds. At the 5% target, the d1-minus-d8 event-tail differences are +0.1125, 0, −0.0194, +0.0571, and −0.0045. The aggregate pattern nevertheless favors d1, with equal-fold mean event-tail masses of 0.1766 versus 0.1474.

This is consistent with direct policy evaluation: d1 achieves 8.66% recall versus 7.11% for d8 at nearly identical FPRs (4.84% vs. 4.87%). The preference persists across all 100 locked policy splits, with recall gains of 0.29–2.17 pp. An exact alignment audit further confirms that episode-maximum thresholding reproduces the original first-crossing implementation with zero alert mismatches across all 200 split–candidate rows (Appendix C.1).

Thus, fold-level heterogeneity coexists with an aggregate tail advantage and a reproducible policy preference, supporting episode maxima relative to candidate-specific thresholds as the relevant first-crossing mechanism.

3. Mechanistic explanation of other ranking reversals

"...the paper does not relate the other reversals in Section 4 to either mechanism."

We extend the analysis to all 156 stable-opposed comparisons (Tables D5–D9). Episode-level ROC preference agrees with policy preference in 132/156 comparisons: 42/49 for first crossing, 38/47 for two consecutive, and 52/60 for cumulative window-3. Across all 316 stable development comparisons, episode-level ROC agreement also exceeds pointwise NLL agreement (263/316 vs. 154/316; 83.2% vs. 48.7%).

Temporal organization exhibits a distinct policy-dependent pattern. Under within-label score shuffling, temporal sensitivity is zero in all 49 first-crossing comparisons, but nonzero in 43/47 two-consecutive and 58/60 cumulative-window-3 comparisons (Table D6). Moreover, sensitivity is greater among stable-opposed than stable-aligned comparisons (Table D9). The opposed-minus-aligned differences are 1.10 pp (95% CI [0.78, 1.44]) and 0.77 pp ([0.49, 1.09]) for two consecutive on development and held-out data, respectively. For cumulative window-3, they are 4.39 pp ([3.80, 4.96]) and 4.48 pp ([3.89, 5.05]). All four intervals exclude zero.

Finally, feasibility priority determines only 15/156 reversals (9.6%), whereas 139/156 (89.1%) are recall-ordered with both candidates feasible (Table D5).

Together, episode-level discrimination is closely associated with first-crossing preferences, while temporal organization is particularly relevant to reversals under order-sensitive policies. These population-level results extend the mechanistic evidence beyond the illustrative d1–d8 comparison.

AuthorsOpenReview ↗

Response to Reviewer HsAt(3/3)

Q1. Mechanisms behind natural reversals

"Beyond the d1–d8 example, which mechanism in Section 5 accounts for the reversals in Section 4?"

We extend the mechanism analysis to all 156 stable-opposed comparisons (Appendix D.5–D.9), revealing distinct, complementary mechanisms across warning policies.

  • First crossing: Episode-ROC preference agrees with policy preference in 42/49 comparisons, while within-label temporal permutation produces zero sensitivity in all 49, consistent with first crossing's invariance to score ordering.
  • Two consecutive and cumulative window-3: Episode-ROC agreement is 38/47 and 52/60, respectively, while temporal permutation changes recall in 43/47 and 58/60 comparisons, demonstrating the additional relevance of temporal organization.
  • Feasibility: Only 15/156 reversals (9.6%) are determined by feasibility priority, whereas 139/156 (89.1%) are recall-ordered with both candidates feasible.

Importantly, temporal sensitivity is significantly greater among stable-opposed than stable-aligned comparisons for both order-sensitive policies, on development and held-out data (Table D9); first crossing remains invariant.

Together, episode-level discrimination is particularly relevant to first-crossing preferences, while temporal organization provides additional explanatory evidence for order-sensitive policies.

Q2. Development versus held-out agreement

"Under FPR matching at ε = 0.010, held-out agreement at the 10% target is 9/11, against 1/11 in development. Does this mean that most development-stage reversals at this target do not hold on unseen groups?"

Yes. Most reversals in this specific subset do not persist: among ten development-opposed comparisons, eight become aligned, one remains opposed, and one becomes tied (Table C22).

These agreement rates compare different evaluation stages: 1/11 measures development predictive–policy agreement, whereas 9/11 compares frozen development predictive preferences against held-out policy preferences. The result is also specific to the 10% target: development agreement is 14/15, 8/10, and 3/9 at 1%, 2%, and 5%, respectively. In the broader 45-comparison analysis, held-out agreement at 10% is 24/45.

An important qualification is that development FPR matching does not ensure matched held-out FPR: the median held-out FPR gap is 2.0 pp, and none of these pairs remains within 1 pp (Table C8). Candidate-specific recalibration and differences across scenario groups can therefore alter the realized operating conditions.

Thus, these particular reversals are mostly not persistent, illustrating that ranking transfer depends on both operating point and evaluation stage, rather than being determined by development preferences alone.

Q3. Held-out FPR gaps

"Table C8 shows that pairs matched at the selection stage still differ by about 0.02 in held-out FPR. Could the authors report the held-out FPR gaps for the stable opposed pairs in Table C19?"

Thank you for this suggestion. We have added the joint held-out recall–FPR analysis for all 85 stable-opposed comparisons in the C19 subset (Table C21).

The pooled policy-minus-predictive held-out FPR gap has a median of −1.06 pp, a mean of −0.59 pp, and an IQR of [−2.18, 1.30] pp. Among the 62 comparisons with higher policy-selected recall, 28 also achieve this improvement without an FPR increase.

These results show that recall improvements frequently occur without additional false-positive cost. This is further supported by the mixed-family selector audit, where policy-based selection improves mean held-out recall while reducing mean FPR across all six policy–target conditions (Table C23).

Together, the joint recall–FPR analysis demonstrates that the selection disadvantage of predictive-only evaluation persists when both operational outcomes are considered.

Hi sorry I add comments in the wrong position, but I am convinced so I raise my score.

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.