- Rating
- 3
- Confidence
- 2
- Informativeness
- 1
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper studies “monitor bycatch”: an LLM monitor assigned to one harm category flags content belonging to another. Using publicly shared conversations, the authors show that these wrong-category alerts can outnumber correct alerts when the target harm is rare. They evaluate several mitigations, including scoring all categories together, separating description from classification, and distilling these approaches into smaller models.
Critical strengths
Practical Problem. Similar problems as bug and vulnerability in software engineering and security domain, there are enormous machine reports but human reviews are expensive. The goal is to reduce false positives while trying to keep high recall.
Critical weaknesses
The explanation remains largely behavioral. The experiments show that models can identify the actual harm yet flag the wrong category, but they do not establish why this happens. “Attribute substitution” and “motivated reasoning” are plausible interpretations rather than demonstrated mechanisms. This limits the explanatory contribution and makes it difficult to predict when bycatch will occur.
**The deployment conclusions depend on uncertain labels and sampling. ** Most category labels come from an LLM jury, while human validation of bycatch covers only 50 cases. Publicly shared conversations may also differ substantially from production traffic. Since the precision results depend on both category boundaries and harm frequencies, these uncertainties limit confidence in the reported deployment impact.
Questions for authors
What evidence distinguishes the proposed explanation from alternatives such as ambiguous category boundaries or a general tendency to flag harmful content? Which experiments most directly support a causal explanation?
Could a simple post-processing rule reject a flag when the monitor’s own description explicitly identifies a different harm? Does the describe-then-match experiment test this possibility, or does it require an additional substantive classification step? A comparison at matched recall would help establish the remaining challenge.
3 comments
Thank you again for your questions. We'd like to check whether the post-hoc reject rule experiment we ran was as you intended (Section A.11) and the expanded human validation (Section A.1: 615 judgements over 200 conversations) address your concerns around labels.
Do let us know if there is anything else we can clarify before the period closes!
Response to Authors
Thank you for taking the time to conduct the additional experiments. I think they have addressed my concerns.
While I am not an expert in this area, I believe the paper and its findings are valuable to the community. This is why I initially gave a "weak accept," despite my concerns. That said, I don't find the paper's contributions particularly exciting or novel enough to justify a higher rating. Therefore, I will keep my rating as "weak accept." Thank you!
Response to Reviewer hVy3
Thank you for your precise questions.
Post-processing rule at matched recall
We ran the suggested post-processing rule to reject a flag when the monitor's own reasoning names a different harm (see Section A.11). Whilst it did effectively remove 97% of bycatch fires, this was, unfortunately, at a loss of 32–35% of true positives (depending on model). It rejects true positives that genuinely contain both multiple distinct harms at once, which makes it less precise at matched recall than the unfiltered monitor.
Describe-then-match is a more general form of this rule that handles multi-label content naturally.
Behavioural explanation and alternatives
Agreed: "attribute substitution" and "motivated reasoning" are not causal mechanisms; we were unable to establish robust causal explanations. Naming the true category reverses most bycatch fires, while pressure alone reverses none (Section A.4.7).
Our claim is that bycatch is not a failure to identify the harm, nor is it explained by other commonly understood categories of false positives (for example, overrefusal). It is an under-explored failure with real consequences for safety monitoring systems.
Evidence against the alternative explanations you mention:
The "ambiguous boundaries" explanation predicts that tightening the definitions should remove bycatch. In practice, a more precise taxonomy with more explicit category boundaries scores no higher (Figure 3, Section A.4.1), while simply asking the monitor to describe the harm first (describe-then-match) does improve results; equally, our synthetic dataset "the Cyber Haystack" has 16 behaviours disjoint by construction (Section 4.3) and yet still shows monitor bycatch.
"General tendency to flag harmful content": in broad strokes, bycatch is a surprising tendency for models to flag harmful content of another categorical type, but this flagging seems sensitive to how the classification task is framed (when summarised as in describe-then-match, bycatch is reduced; when asked with all the categories in context, it marks them correctly; or even when asked in isolation, e.g. "Is a Molotov cocktail an explosive device?"). What is really needed is improved discrimination at matched recall. Raising the threshold (to try to adjust for this tendency to flag) trades false positives against recall: it removes genuine fires along with bycatch.
Label reliability
We address the limited human validation with the additional human data described in our reply to 9QGq - we've increased the human sample by 12× and plan to release it as a test split.
In brief: N=615 crowd judgements over 200 conversations replace the earlier N=50. A uniform sample of unflagged traffic contains a missed harm in 0.5% of conversations; under a worse case read of the annotators as conservative, 90% [85%, 94%] of bycatch fires remain incorrect. Largely it appears the jury is over-inclusive, which makes bycatch fires look like true positives, so it understates bycatch rather than overstating it (Section A.1).
Sampling and deployment estimates
On sampling: agreed (as noted in Section 3.2) — shared conversations are a selected sample — and this does mean our precision estimates are not production estimates (they are in-the-wild estimates).
We can separate bycatch (as in Section 3.1) into the per-pair confusion rate (how often hate fires on a harassment monitor), which does not depend on the incidence of harm; and the precision, or bycatch's share of false positives, which does.
If harm occurs more frequently in commonly mistaken categories, bycatch is typically worse (see Figure 2a); however, even rare harms can still experience significant bycatch if they are frequently mistaken (for example, sexual exploitation in Figure 2a).