- Rating
- 3
- Confidence
- 2
- Informativeness
- 3
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This paper introduces AuditBench, a benchmark for evaluating alignment auditing techniques on language models with hidden behaviors. The authors construct 56 model organisms spanning 14 hidden behaviors, using two behavior-instillation methods (transcript distillation and synthetic document fine-tuning) and two adversarial training methods (SFT and KTO) to discourage models from revealing their hidden behaviors. They then develop an investigator agent to evaluate different black-box, white-box, and scaffolded auditing tools. The main findings are that scaffolded black-box tools achieve the highest overall auditing success rates, auditing difficulty varies substantially across training configurations, and tools that provide useful evidence do not necessarily improve agent performance due to limitations in tool use and evidence interpretation.
Critical strengths
- The benchmark is interesting and practically relevant and the released models are useful. I appreciate the construction and release of a diverse set of model organisms, along with the training and evaluation pipelines. In particular, adversarially training models to resist confessing their hidden behaviors makes the auditing task more challenging and potentially more representative of realistic alignment auditing scenarios.
- Evaluating auditing techniques through an investigator agent is a useful approach, as it measures whether tools actually help an investigator discover hidden behaviors rather than merely producing informative outputs in isolation. The framework also enables systematic comparisons across tools and target model configurations.
- The paper identifies several interesting findings, including the importance of scaffolding for effective tool use, the substantial differences in auditability across training methods, and the tool-to-agent gap. Case studies illustrating how agents can form hypotheses from tool outputs and conduct targeted, controlled follow-up investigations are informative.
Critical weaknesses
- Limited investigation of the agent's ability to use auditing tools. The paper identifies under-utilization and ineffective use of tools as important bottlenecks, but the main experiments rely on a single investigator model (Claude Haiku 4.5). It is unclear how much of the tool-to-agent gap is attributable to limitations of the auditing techniques versus the investigator's capabilities. For example, the authors should try a more capable investigator model, and finetune/in-context learn these auditing tools to disentangle these factors. Additionally, scaffolded tools generate substantially more diverse queries per call, making it useful to compare methods under matched query or compute budgets.
- Some simple auditing baselines are not included. For example, since the target models are fine-tuned from a known base model, an investigator could compare target and base model responses to identical inputs to identify systematic behavioral differences. Simple statistical signals, such as response length or differences in token probabilities and output distributions, might also provide useful evidence.
- Evaluation might not fully reflect real-world settings. The investigator is explicitly told that a hidden behavior exists and can submit up to 10 predictions, with success defined by whether any prediction matches the ground truth. This does not evaluate false positives on clean models or how precisely the investigator identifies the hidden behavior.
- Probably not a critical weakness per se, but I hope the authors could provide some insights into how to improve audit models on challenging models (transcript-distilled + KTO adversarial training).
Questions for authors
- Given the consistently low success rates on transcript-distilled models with KTO adversarial training, what do the authors consider the main bottleneck?
2 comments
Thank you for the detailed response and additional analyses. I appreciate the authors' clarifications and continue to see substantial value in the benchmark, released model organisms, and evaluation framework.
While I understand the scope and resource constraints, I believe some simple baselines, such as comparing response lengths, token probabilities, and output distributions between the base and fine-tuned models, could provide useful auditing signals and strengthen the empirical comparisons.
Overall, I acknowledge the contributions and improvements made during the rebuttal, but my remaining concerns lead me to maintain my original score.
We thank the reviewer for valuable feedback and describing the benchmark as “interesting and practically relevant” as well as evaluation through an investigator agent as “a useful approach”. We also appreciate the reviewer’s positive assessment of the paper’s soundness.
Stronger investigators, tool-use training, and in-context examples may narrow the gap. The general response G3 discusses the scope of our current evaluation and the resource constraints on a full replication. We tested a simpler intervention, an explicit tool-use reminder, in the User Sampling, Text Completion, and Petri configurations. Auditing success remained similar.
The general response G1 reports target-response counts, output tokens, Claude API costs, and runtime. Scaffolded tools achieve higher success with lower Claude API spending under our protocol, while obtaining more target responses. Performance under matched query or total-compute budgets remains untested, and we will make this distinction more explicit.
We agree that comparing target and base-model responses to identical inputs, alongside statistical signals such as response length, token probabilities, and output distributions, would provide useful additional baselines. Appendix K.2 includes model-diffing checks for validating the target models, but we have not evaluated the proposed comparisons as tools available to the investigator.
Our metric credits an audit when at least one prediction identifies the planted behavior; it does not assess the validity of every reported finding. Investigators are told that a planted behavior is present. Precision of the complete set of findings and false-positive rates on models without implanted behaviors remain unmeasured. The planted label also does not exhaust all potentially valid findings: even unmodified base models may exhibit unintended behaviors. Predictions beyond the planted behavior are therefore not automatically false positives, but their validity requires separate assessment against supporting evidence. We will clarify these limits of the evaluation.
Our evidence suggests that eliciting the behavior without knowing what to look for is a major bottleneck. TD+KTO targets still strongly exhibit their implanted behaviors in scenarios designed to elicit them (Figure 2). However, static evaluations of unscaffolded tools often yield little actionable evidence, even when the evidence classifier is given the true behavior (Appendix H). This suggests improving the generation of targeted tests and follow-up on weak clues. Stronger investigator reasoning could help with both choosing these tests and interpreting their outputs.