- Rating
- 4
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 3
- Soundness
- 3
Read review
Paper summary
This work proposes honesty adapters, which can be applied to LoRA fine-tuned models, making them answer questions about their behavior honestly in follow up questions and allowing lightweight detection of unusual behavior. The method is strong, matching or outperforming baselines on existing benchmarks, including AuditBench, and shows promise in verbalizing behavior rather than merely detecting it.
Critical strengths
- The paper clearly makes a strong contribution to monitoring literature. The strong out-of-distribution results on AuditBench are compelling and backed up by similar results on other benchmarks. The method is also lightweight and quick to run, meaning that it could be used in practice easily. In short: it is a meaningful technical contribution with strong empirical backing.
- The authors have clearly taken steps to ensure that the evaluation is controlled and transparent. Filtering the model organism zoo using a judge, transparency regarding calibration, and candid discussion of limitations are welcome additions, making the paper's results easy to trust.
Critical weaknesses
- The main concern I have about the method and results is whether the performance of the honesty adapter is explained primarily by just reading the transcript. Concretely, the authors could run an experiment where a model organism generates responses, but is then disabled and the honesty adapter applied on top of the base model only, with the given transcript in context. If performance didn't suffer, this would suggest that the honesty adapter is primarily deriving its signal from the transcript being in context rather than reading any white box properties of the MO adapter itself. I notice the authors have done something like this in the off-policy benchmarks; it would be good to see it in the MO benchmarks too.
Questions for authors
- Can the authors run the experiment as described in W1?
- Does it matter which model produces the target descriptions used for training (currently Claude Sonnet 5)? Is there some kind of same-family or same-model effect here?
- How does performance scale as a function of the number and diversity of training organisms used? Can the authors provide a scaling curve as a function of number of training organisms as well as leaving categories out of training and see how well the resultant adapter performs on them?
- Can the authors provide additional held-out splits? E.g. different random seeds to split the 35 and 533 organisms.
- Does the adapter transfer to full fine tunes as opposed to just LoRAs?
- Finally, do the authors have any opinion about how this kind of work could extend to auditing base models for which no reference class of different LoRAs exists?
4 comments
Response to Reviewer uJKx (part 2)
Full fine-tunes (Q5). We have not tested this. Every organism we train and evaluate on is a LoRA. The transfer to independently built organisms such as AuditBench shows that the honesty adapter is not tied to our zoo’s training recipe, but those organisms are also LoRAs. We cannot yet say how well the honesty adapter generalizes to a fully fine-tuned MO. We plan to test this by fully fine-tuning a few MO-35 behaviors into Qwen3-14B, and testing the honesty adapter on the resultant MOs. We will add the result to the paper.
Base models without a family of LoRAs (Q6). To audit a fine-tuned model with no LoRA family, a developer can build one by training organisms with known behaviors on top of it. The hard case then, is a hidden behavior already in the base model, which is shared by every training organism. In Section 5.3, we simulate this by putting a backdoor under all training organisms. Detection and verbalization with the transcript in the context still work, but verbalization without a transcript fails. We therefore expect the approach to extend to monitoring a base model's responses, but not yet to finding behaviors that never appear in a transcript. We agree that generating high-quality model organisms at scale can be expensive and challenging, and automating it with agents is an exciting direction for future work.
We hope these results address the reviewer's concerns. We are happy to answer any other questions!
Thanks
Excellent, thanks to the authors for addressing all of my questions. I think this is a very strong paper and expect it will be well received by the broader community :).
Thanks for your review
Thank you for the kind words! Your questions and suggested experiments have definitely made the paper stronger. We really appreciate your deep engagement with our work!
Response to Reviewer uJKx (part 1)
We thank the reviewer for the positive review and the concrete experiment suggestions. We address each question below.
Does the adapter rely on the organism's internal state? (W1, Q1) Yes. We ran the experiment as described, and the full table is under "Does the adapter rely on the organism's internal state?" in our reply to Reviewer 6taw. With the transcript fixed, using the clean model instead of the organism lowers the adapter's AUROC by 0.091 (Qwen) and 0.312 (Llama) on average over MO-35, AuditBench and conspiracy, and by 0.073 and 0.204 on AuditBench alone. The clean model with the adapter still stays well above chance on most benchmarks (e.g., 0.831 on Qwen AuditBench), so part of the signal is read from the transcript and part depends on the organism. Replacing the organism with a random training organism, which keeps the adapter in distribution, also lowers AUROC on every benchmark (by 0.140 and 0.242 on average), providing evidence that the honesty adapter uses the privileged access to the organism's internal state.
Which LLM writes the target descriptions (Q2). We have not tested a second description model. For detection, the choice can matter by at most what the descriptions contribute: replacing them with fixed binary answer leads to a detection AUROC drop of up to 5 points. However, verbalization depends entirely on the descriptions (see “What drives the gains” in our reply to Reviewer PMuE). We expect any capable LLM to write descriptions good enough for the small uplift on detection and large uplift on verbalization. To check for a same-family effect in grading using Claude Sonnet 5, we re-graded the adapter's self-report verbalization with GPT-6.1 Terra using the same rubric. The two judges agree closely (with exact agreement of 75%-83% across benchmarks and within 1 point agreement of 90%-100%). The resulting aggregate verbalization scores for the adapter on MO-35 and AuditBench:
Performance scaling with the number of organisms (Q3). We trained adapters on random subsets of 107, 214, 319 and 426 of the 533 training organisms with the recipe described in the paper (Qwen, two seeds each). Detection rises for up to about 300 organisms and then plateaus. Dispositional verbalization keeps improving, so more organisms still help the adapter describe a behavior even after detection saturates.
The columns are the number of organisms used for training. Each cell reports the average of two seeds, and the 533 column is the headline performance of our submitted adapters.
Leaving categories out of training (Q3). We also held out each category out of training and compared it with a control adapter trained on the same number of organisms with that category included. Looking at the AUROC difference between the adapter and the control, leaving a category out costs little for backdoor, quirk and harmful (at most 0.031 AUROC) and slightly more for rare quirks (~0.062 AUROC). Heuristic detection however degrades to chance (0.503 vs 0.849). The adapter transfers to most unseen categories but not to all of them, so the diversity of training categories matters. We will add both experiments to the appendix.
Other organism splits (Q4). We drew two more random 533/35 splits of the 568 organisms (5 held-out organisms per category, disjoint from the paper's held-out set) and trained the adapter and the MO-533 probe on each. Since these are newly trained adapters, we also retrained the adapter on the paper's split with the same code. All rows use the trained question. MO-35 has different organisms in each split, while AuditBench is the same in all three.
Detection varies by at most 0.018 AUROC across splits. On every split the adapter stays above the best baseline, which reaches at most 0.949 on MO-35 and 0.832 on AuditBench.