- Rating
- 2
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2
Read review
Paper summary
This paper introduces nine tasks for analyzing reasoning traces in settings where surface reading of the chain of thought is insufficient. Seven binary tasks receive the main evaluation, each with in-distribution and out-of-distribution splits, while two exploratory tasks appear in the appendix. The benchmark compares text classifiers, activation probes, LLM monitors, a tool-using strategy agent, and human annotators. No method family dominates, and ID-selected methods often perform only modestly above chance OOD.
Critical strengths
-
The paper targets a consequential gap in current safety evaluation: inferring model behavior when the explicit reasoning trace omits the relevant causal signal.
-
Paired ID/OOD splits and selection on ID followed by scoring on OOD reduce the risk of celebrating dataset-specific shortcuts.
-
The comparison spans lexical, activation-based, LLM, agentic, and human methods. The finding that different methods win different tasks is informative and creates a useful baseline for future work.
Critical weaknesses
-
Most tasks are built from Qwen3-32B rollouts, with Gemma used for one specialized task, and several labels deliberately select extreme resampling regimes. It is unclear whether the resulting difficulty and method ranking transfer to other reasoning models or realistic monitoring deployments.
-
Human annotators receive ten examples and no model internals, while some agent variants use larger labeled pools, repeated LLM calls, source-model interventions, logits, or activations. The claim that model methods beat humans therefore mixes information access, compute, and expertise.
-
Several ground-truth labels are defined by extensive resampling, which prevents evaluation of resampling itself even though it is a natural and likely strong method. Some tasks may be unsolvable from the provided input, and the main comparisons often show point estimates without uncertainty intervals. This makes near-chance results hard to distinguish from insufficient information or split variance.
Questions for authors
-
Do the task difficulty and method rankings replicate when the subject model, prompt family, and OOD construction are changed, ideally with repeated split generation and uncertainty estimates?
-
Can the authors provide a resource-matched comparison that equalizes labeled examples and information access, together with an estimate of each task's attainable ceiling from the supplied input alone?
2 comments
Thanks for your response. I think part of my concerns are addressed. Overall, I think the paper is a borderline one for ICLR and I tend to maintain my original assessment.
Response to Reviewer LPi3
Thanks for your feedback!
W1, Q1 (one subject model, extreme regimes, and replication). On a second subject model, gpt-oss-120b, the classifier and the monitor come out in the same order on the three tasks where we ran both Section 4.4). We weren't able to replicate user preference. The labels do keep the two ends of a resampling statistic and discard the middle (Table 1 now says so). On reasoning termination we also scored the discarded prefixes: on them the attention probe's score has a rank correlation of 0.31 with the label's own statistic and of 0.56 with the share of continuations that end the reasoning within 100 tokens. On professor hint the probes carry little graded information (Appendix H.6). The one split variation we ran holds out each training domain in turn: on reasoning termination the attention probe's OOD AUROC stays between 0.95 and 0.98 (Appendix H). We selected the tasks for difficulty and do not claim that they resemble monitoring in deployment.
W2, Q2 (the human comparison mixes resources). We agree that the comparison between human annotators and our methods is unfair since annotators had fewer resources. We have withdrawn the corresponding claim. In the new matched comparison (four tasks, the annotators' own items), the annotators, a GPT-5.4 monitor given their instructions, and the classifiers each get ten labeled examples. The monitor and the TF-IDF classifier see text only, as the annotators did; the probes also read activations. Of eight paired comparisons (annotators against the monitor and against a ten-label probe) two are supported, both in the annotators' favor, and no method is supported above the annotators (Appendix E.4, Figure 13). The agent is not in this comparison, because it sees 40 examples and two of its tools query the subject model.
W3 (resampling excluded, solvability, and lack of intervals).