acceptodds
Under review as a conference paper at ICLR 2027

Implementation Matters in Monitorability Evaluations

Abstract

Robustly monitoring AI systems' misbehavior is critical to preventing real-world harm. A monitor is an automatic mechanism which tries to detect when LLMs misbehave. To assess the ability of monitors to detect misbehavior, prior work has developed monitorability evaluations. These evaluations are used to inform deployment decisions (e.g., recent frontier model system cards), but it is not completely understood whether they are trustworthy indicators of our ability to detect misbehavior in the wild. Towards this question, we define monitorability and monitorability evaluations. Within our definition we identify a set of implementation-level details that vary across existing monitorability evaluations—including the choice of summary statistic, how label noise is handled, and the specificity of the monitor prompt. We study the impact of these implementation-level details across three monitorability evaluations and find that some individually-reasonable decisions can substantially change downstream conclusions—including which actors appear most monitorable—while others make surprisingly little difference. Overall, our results suggest that averaging a few hundred samples across a few monitors and monitorability evaluations might not be sufficient to reliably distinguish the differences in monitorability of current models.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
chain-of-thought monitoring, monitorability, AI safety, evaluation methodology, large language models, reproducibility

Reviews

Among:
Shape:
1234this paper, Bottom 25%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 1.75 over 4 reviews, above 25% of the others.
Reviewer_A6dfOpenReview ↗
Rating
1
Confidence
2
Informativeness
2
Presentation
1
Soundness
1
Read review
Paper summary
  1. This paper presents a definition of monitorability and framing evaluations as proxy metrics for the ideal definition.
  2. This paper shows how small changes in evaluation configuration can change the results of monitorability evaluations on a given benchmark. A range of experiments are run across up to 6 non-frontier actor models and up to 3 different benchmarks. These results show that correcting for "false, false positives" (FFPs) and the summary statistic used changes ranking between models. These findings show that care needs to be taken when running monitorability evaluations and that experiment design is important for getting robust results. It recommends that results should be corrected for FFPs and that unelicited benign transcripts should be used for more reliably evaluating if a monitor can discriminate between transcripts.
Critical strengths
  1. I found the investigation into how differing the evaluation set-up in seemingly reasonable ways can result in different outcomes interesting. It is important to understand how accurate these evaluations are, and working towards a robust method for evaluation is necessary for high quality monitoring techniques to be developed for effective AI control.
Critical weaknesses
  1. This paper has formatting issues. There are many visual errors, e.g. Figure 4.3 (b) is unreadable due to the legend partially blocking the bars, on page 7 there is unexplained blue text. I also spotted an inconsistency in the results presented, in Figure 4.2 the caption states "The GPT-OSS-120B bar in the “No supp. (FFP correct.)” subplot is labeled N/A" but the figure shows a value in this column. In addition to this there are errors in the references, e.g. Guan et al. is cited from arxiv but is published in ICML. These presentation issues undermine the content presented and may make the reader doubt the correctness of the results.
  2. The definition of monitorability is presented as a contribution however it appears to be similar to the definition of monitorability in "Monitoring Monitorability" by Guan et al. (ICML 2026). It would be beneficial to the reader to frame how the formalisation of monitorability is adding onto pre-existing work.
Questions for authors
  1. Can you explain clearly how your definition of monitorability is distinct from previous works and what exactly is the new contribution?
  2. Can you rectify the formatting and quality of writing issues so as to better showcase the quality of your work (e.g. plot formatting, text formatting, referencing works that have been published outside of arxiv correctly)
Reviewer_o3K4OpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
2
Read review
Paper summary

The paper examines whether reasonable choices in monitorability evaluations change conclusions about which actors are easier to monitor. It formalizes the evaluation setup and studies misbehavior labels, benign-transcript selection, summary statistics, monitor prompts, and paired designs across three benchmarks. Its main findings are that correcting labels for detected but unsuccessful misbehavior attempts, and changing the benign-transcript distribution, can alter actor rankings. The contribution is primarily an empirical audit of existing evaluation practices.

Critical strengths

The paper investigates whether comparisons between actors survive reasonable evaluation choices, rather than introducing another benchmark without examining how its scores should be interpreted. The false-false-positive analysis highlights cases where a monitor correctly detects an attempted violation, while the outcome-based label records no misbehavior, and quantifying how this affects actor comparisons is useful. The experiments demonstrate that the negative examples used to evaluate a monitor are consequential, not merely a background implementation choice. Also, the length-only baseline exposes a potential shortcut in these evaluations, and the authors acknowledge and investigate possible correlated errors between the monitor and the label-correction judge.

Critical weaknesses

A change in actor rankings does not necessarily mean the evaluation is unreliable. For example, counting attempted rather than only successful misbehavior changes what the evaluation measures: a monitor flagging a failed attempt goes from being penalized to being rewarded. The resulting ranking change may therefore be expected. The authors should distinguish changes caused by redefining the target from instability in measuring the same target. Regarding the FFP: patterns are discovered among samples already flagged by monitors, creating a risk that corrected labels favor monitor agreement. Appendix B.3 provides useful diagnostics, but Figure B.1 covers only two actors. Moreover, the demonstrated FFP cases predominantly involve visible reasoning about unsuccessful attempts; their broader empirical scope is not established. In addition, related work does not adequately establish the contribution. The short discussion lists many papers without explaining their findings or contrasting their definitions and measurement choices. This makes it difficult to assess what the framework and experiments add.

Questions for authors

Echoing the weaknesses section:

  • Can you separate ranking changes caused by measuring different things from changes caused by alternative implementations of the same objective?
  • What do the definitions in Sec 3 add? How do they relate to prior formalizations, and how would known challenges such as CoT obfuscation and evaluation awareness be represented?
  • Can the literature comparison be made explicit? A table contrasting prior work’s target behavior, monitor inputs, labels, benign sets, and metrics would clarify both the motivation and novelty.
Reviewer_1x6fOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

This paper studies whether implementation-level details in chain-of-thought monitorability evaluations change their conclusions, particularly the relative ranking of actors. It formalizes monitorability evaluations (Definition 1) and studies five details: FFPs in the misbehavior proxy, the benign transcript distribution, the summary statistic, monitor-prompt specificity, and spurious correlations. Using three evaluations (APPS-backdoor, bio-math, imposs-bench), six actors, and three monitors, the paper finds that FFP correction and benign-set supplementation can reorder actors, while prompt specificity and paired designs make little difference.

Critical strengths
  1. The motivation is clear. The topic is interesting, and this paper seems to explore an important and timely measurement problem. Its formal decomposition of monitorability evaluations clearly exposes where unavailable ground truth, elicitation, proxy labels, task distributions, and summary statistics enter the evaluation pipeline.
  2. The empirical analysis is broad. The study spans three benchmarks, six actors and three monitors, identifying failed-but-visible misbehavior attempts as an important labeling issue and reporting both implementation choices that materially affect conclusions and those that appear relatively stable.
Critical weaknesses
  1. The FFP correction risks circularity. The corrected proxy incorporates an LLM judge that observes the same transcript cues as the evaluated monitors, so correlated errors can mechanically increase apparent agreement and reorder actors. Human review is used to identify patterns, but the paper does not sufficiently document blinded per-instance labeling, sample sizes, annotator agreement, or error rates for the corrected labels. Relative actor comparisons are further confounded by actor-specific elicitation modifications, such as instructing Qwen3-8B to think less and reducing Kimi-K2’s covertness requirements.
  2. The evidence is partly self-undermining with respect to statistical reliability. The paper argues that a few hundred samples and a few monitors are insufficient to distinguish actors, yet its own claims about ranking reversals and implementation effects are derived from approximately 250 samples per actor–task pair and three monitors.
Questions for authors
  1. How were the studied implementation-level details selected? Since AI monitorability is not my research area of ​​expertise, can the authors provide a systematic taxonomy of the monitorability-evaluation pipeline, distinguish choices that merely change implementation from those that change the estimand or data distribution, and explain which important dimensions were excluded?
  2. Can the authors validate the FFP judge across at least one actor-task pair, report the size of the human-agreement study, and test a variant whose FFP patterns are discovered on randomly sampled transcripts rather than only on monitor-flagged ones, to rule out circularity between the monitor and the corrected label?
  3. Recommendations (iii) and (iv) appear unsupported by the paper’s own results: monitor-prompt specificity did not materially change actor rankings, and the paired design produced no clear ranking changes. Are these intended as precautionary rather than empirically supported recommendations?
Reviewer_MH2vOpenReview ↗
Rating
2
Confidence
3
Informativeness
1
Presentation
2
Soundness
3
Read review
Paper summary

The paper provides a formalism of monitorability evaluations (specifically CoT monitorability) and their implementation choices. It then empirically measures the effect of these implementation choices in three existing monitorability evaluations. It uses these results to make some recommendations, including addressing false false positives (FFPs) and mixing in "unelicited benign transcripts" (SU).

Contributions:

  • The paper formalizes many of the variations in existing monitorability evaluations.
  • The paper measures the effect of varying the misbehavior proxy (i.e. "property estimate"), the benign calibration set (i.e. "task distribution"), the summary statistic, and the monitor instructions.
  • The paper studies the prevalence and effect of FFPs, finding that they're common and that correcting for them changes the relative ranking of different models.
  • The paper makes recommendations on the basis of these results: correcting for FFPs, mixing in unelicited benign transcripts, standardizing monitor prompts, and using paired study designs.
Critical strengths
  • The paper shows that the implementation details of monitorability evaluations can meaningfully change their bottom line and should thus be chosen carefully and with intent. This is important because monitorability evaluations are a substantial component of frontier AI risk assessments.
  • The paper also gives the field a standardized vocabulary with which to talk about and set best practices for monitorability evaluations.
Critical weaknesses
  • The formalism is mostly just restating the standard setup of existing monitorability evaluations, and none of the paper's results depend on it.
  • The FFP issue is largely an artifact of labeling positives by outcome rather than by elicitation. Many evaluations (e.g. BashArena, LinuxArena) already count all elicited attack transcripts as positives and use unelicited benign transcripts as negatives, so FFPs don't arise. To be clear, this alternative has its own failure mode (namely, that the elicited actor may not misbehave), but this is seldom a problem in practice. The paper should make it clear which evaluations are affected by FFPs.
  • Distinguishability and pairwise disagreement are only weak indicators of a summary statistic's quality. A statistic can distinguish actors on many axes that don't track monitorability, and statistics can agree because they share a bias (e.g. transcript length, as in Section B.5). The recommendation of SU statistics would be much stronger if validated against a known ground-truth ordering.
Questions for authors
  1. How do your conclusions change if positives are labeled by elicitation (all elicited attack transcripts as positives, unelicited benign transcripts as negatives), as in many control evaluations? Which existing monitorability evaluations are affected by FFPs?
  2. Which model implements PFFPP_{\text{FFP}}, and what's its correlation with the monitors? How much of the change from setting P^=max⁡(P^,PFFP)\hat{P} = \max(\hat{P}, P_{\text{FFP}}) is explained by this correlation?
  3. Why is binned MSE used as the default statistic (especially given that it frequently disagrees with the SU statistics)? Do the results hold under other statistics such as AUC or Recall@5%FPR (ideally with SU)?
  4. Can the summary statistics be validated against a known ground-truth ordering, e.g. actors deliberately made less monitorable via obfuscation?

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.