acceptodds
Under review as a conference paper at ICLR 2027

Training Honesty Adapters To Detect Deception In LLMs

13 views

Abstract

Activation probes trained on off-policy data are a cheap way to monitor language models for deception, but they often fail to generalize to on-policy deception outside their training distribution. We instead train an honesty adapter, a LoRA stacked on a model, to report whether the model exhibited a hidden behavior or deception in a transcript, using on-policy transcripts from 533 model organisms of Shenoy et al. (2026). Scoring a forced admission against a forced denial gives a continuous monitor score. Honesty adapters for Qwen3-14B and Llama-3.3-70B detect deception by held-out organisms with near-perfect AUROC and generalize to auditing organisms trained with a different recipe: on AuditBench, the Qwen3-14B adapter reaches 0.90 AUROC, against 0.74 for a probe trained on the same data and 0.66 for a probe trained on off-policy deception. On off-policy benchmarks of lies, reward hacking and hard-coded test cases, the adapters match deception probes in AUROC. Asked follow-up questions about a transcript, an honesty adapter also describes the organism's hidden behavior more accurately than an introspection adapter trained on the same organisms, and comparably or better without a transcript. Both on-policy training data and adapters in place of probes thus help deception monitors generalize.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
deception, monitoring, model organisms

Reviews

Among:
Shape:
1234this paper, Top 1%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 3.67 over 3 reviews, above 99% of the others.
Reviewer_uJKxOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
3
Soundness
3
Read review
Paper summary

This work proposes honesty adapters, which can be applied to LoRA fine-tuned models, making them answer questions about their behavior honestly in follow up questions and allowing lightweight detection of unusual behavior. The method is strong, matching or outperforming baselines on existing benchmarks, including AuditBench, and shows promise in verbalizing behavior rather than merely detecting it.

Critical strengths
  • The paper clearly makes a strong contribution to monitoring literature. The strong out-of-distribution results on AuditBench are compelling and backed up by similar results on other benchmarks. The method is also lightweight and quick to run, meaning that it could be used in practice easily. In short: it is a meaningful technical contribution with strong empirical backing.
  • The authors have clearly taken steps to ensure that the evaluation is controlled and transparent. Filtering the model organism zoo using a judge, transparency regarding calibration, and candid discussion of limitations are welcome additions, making the paper's results easy to trust.
Critical weaknesses
  • The main concern I have about the method and results is whether the performance of the honesty adapter is explained primarily by just reading the transcript. Concretely, the authors could run an experiment where a model organism generates responses, but is then disabled and the honesty adapter applied on top of the base model only, with the given transcript in context. If performance didn't suffer, this would suggest that the honesty adapter is primarily deriving its signal from the transcript being in context rather than reading any white box properties of the MO adapter itself. I notice the authors have done something like this in the off-policy benchmarks; it would be good to see it in the MO benchmarks too.
Questions for authors
  • Can the authors run the experiment as described in W1?
  • Does it matter which model produces the target descriptions used for training (currently Claude Sonnet 5)? Is there some kind of same-family or same-model effect here?
  • How does performance scale as a function of the number and diversity of training organisms used? Can the authors provide a scaling curve as a function of number of training organisms as well as leaving categories out of training and see how well the resultant adapter performs on them?
  • Can the authors provide additional held-out splits? E.g. different random seeds to split the 35 and 533 organisms.
  • Does the adapter transfer to full fine tunes as opposed to just LoRAs?
  • Finally, do the authors have any opinion about how this kind of work could extend to auditing base models for which no reference class of different LoRAs exists?
4 comments
AuthorsOpenReview ↗

Response to Reviewer uJKx (part 1)

We thank the reviewer for the positive review and the concrete experiment suggestions. We address each question below.

Does the adapter rely on the organism's internal state? (W1, Q1) Yes. We ran the experiment as described, and the full table is under "Does the adapter rely on the organism's internal state?" in our reply to Reviewer 6taw. With the transcript fixed, using the clean model instead of the organism lowers the adapter's AUROC by 0.091 (Qwen) and 0.312 (Llama) on average over MO-35, AuditBench and conspiracy, and by 0.073 and 0.204 on AuditBench alone. The clean model with the adapter still stays well above chance on most benchmarks (e.g., 0.831 on Qwen AuditBench), so part of the signal is read from the transcript and part depends on the organism. Replacing the organism with a random training organism, which keeps the adapter in distribution, also lowers AUROC on every benchmark (by 0.140 and 0.242 on average), providing evidence that the honesty adapter uses the privileged access to the organism's internal state.

Which LLM writes the target descriptions (Q2). We have not tested a second description model. For detection, the choice can matter by at most what the descriptions contribute: replacing them with fixed binary answer leads to a detection AUROC drop of up to 5 points. However, verbalization depends entirely on the descriptions (see “What drives the gains” in our reply to Reviewer PMuE). We expect any capable LLM to write descriptions good enough for the small uplift on detection and large uplift on verbalization. To check for a same-family effect in grading using Claude Sonnet 5, we re-graded the adapter's self-report verbalization with GPT-6.1 Terra using the same rubric. The two judges agree closely (with exact agreement of 75%-83% across benchmarks and within 1 point agreement of 90%-100%). The resulting aggregate verbalization scores for the adapter on MO-35 and AuditBench:

self-reports judgedScore with Claude Sonnet 5Score with GPT-6.1 TerraExact agreementWithin 1 point
MO-35, transcript in context0.7670.73378%98%
AuditBench, transcript in context0.2400.22783%100%
MO-35, no transcript0.5910.67475%90%

Performance scaling with the number of organisms (Q3). We trained adapters on random subsets of 107, 214, 319 and 426 of the 533 training organisms with the recipe described in the paper (Qwen, two seeds each). Detection rises for up to about 300 organisms and then plateaus. Dispositional verbalization keeps improving, so more organisms still help the adapter describe a behavior even after detection saturates.

training organisms107214319426533
MO-35 detection AUROC0.9460.9570.9790.9810.981
AuditBench detection AUROC0.8760.8940.9170.9000.904
MO-35 dispositional verbalization (mean judge score)0.380.460.530.560.59

The columns are the number of organisms used for training. Each cell reports the average of two seeds, and the 533 column is the headline performance of our submitted adapters.

Leaving categories out of training (Q3). We also held out each category out of training and compared it with a control adapter trained on the same number of organisms with that category included. Looking at the AUROC difference between the adapter and the control, leaving a category out costs little for backdoor, quirk and harmful (at most 0.031 AUROC) and slightly more for rare quirks (~0.062 AUROC). Heuristic detection however degrades to chance (0.503 vs 0.849). The adapter transfers to most unseen categories but not to all of them, so the diversity of training categories matters. We will add both experiments to the appendix.

Other organism splits (Q4). We drew two more random 533/35 splits of the 568 organisms (5 held-out organisms per category, disjoint from the paper's held-out set) and trained the adapter and the MO-533 probe on each. Since these are newly trained adapters, we also retrained the adapter on the paper's split with the same code. All rows use the trained question. MO-35 has different organisms in each split, while AuditBench is the same in all three.

training splitQwen MO-35Qwen AuditBenchLlama MO-35Llama AuditBench
paper's split (retrained)0.985 / 0.9860.929 / 0.9310.9870.967
new split 10.973 / 0.9790.928 / 0.9270.9820.962
new split 20.978 / 0.9680.928 / 0.9310.9830.965

Detection varies by at most 0.018 AUROC across splits. On every split the adapter stays above the best baseline, which reaches at most 0.949 on MO-35 and 0.832 on AuditBench.

AuthorsOpenReview ↗

Response to Reviewer uJKx (part 2)

Full fine-tunes (Q5). We have not tested this. Every organism we train and evaluate on is a LoRA. The transfer to independently built organisms such as AuditBench shows that the honesty adapter is not tied to our zoo’s training recipe, but those organisms are also LoRAs. We cannot yet say how well the honesty adapter generalizes to a fully fine-tuned MO. We plan to test this by fully fine-tuning a few MO-35 behaviors into Qwen3-14B, and testing the honesty adapter on the resultant MOs. We will add the result to the paper.

Base models without a family of LoRAs (Q6). To audit a fine-tuned model with no LoRA family, a developer can build one by training organisms with known behaviors on top of it. The hard case then, is a hidden behavior already in the base model, which is shared by every training organism. In Section 5.3, we simulate this by putting a backdoor under all training organisms. Detection and verbalization with the transcript in the context still work, but verbalization without a transcript fails. We therefore expect the approach to extend to monitoring a base model's responses, but not yet to finding behaviors that never appear in a transcript. We agree that generating high-quality model organisms at scale can be expensive and challenging, and automating it with agents is an exciting direction for future work.

We hope these results address the reviewer's concerns. We are happy to answer any other questions!

Reviewer_uJKxOpenReview ↗

Thanks

Excellent, thanks to the authors for addressing all of my questions. I think this is a very strong paper and expect it will be well received by the broader community :).

AuthorsOpenReview ↗

Thanks for your review

Thank you for the kind words! Your questions and suggested experiments have definitely made the paper stronger. We really appreciate your deep engagement with our work!

Reviewer_PMuEOpenReview ↗
Rating
3
Confidence
3
Informativeness
3
Presentation
3
Soundness
3
Read review
Paper summary

The paper trains “honesty adapters” on 533 model organisms with specific hidden behaviors, detecting deception through the relative likelihood of admissions versus denials and generating behavioral descriptions. The method outperforms activation probes on unseen model organisms, particularly the independently constructed AuditBench, but its effectiveness on naturally occurring deception remains unverified.

Critical strengths
  1. Clear research question and contribution. The paper extends model introspection to deception monitoring of individual responses, combining detection with behavioral explanations. Its main innovation lies in task formulation and training data design.
  2. Careful data construction and informative comparisons. Cross-trigger prompts, response-level labeling, and negative examples reduce opportunities for shortcut learning. Comparisons include probes trained on the same data, alternative probe architectures, and models without adapters.
Critical weaknesses
  1. Limited task validity and generalizability. Positive examples include non-deceptive behaviors such as stylistic quirks, and primarily involve artificially implanted behaviors. High performance therefore does not yet establish reliable detection of real-world deception.
  2. The source of improvement remains unclear. Adapters receive full behavioral descriptions as supervision, whereas probes receive binary labels. Without a text-only monitor trained with comparable supervision, it is also difficult to distinguish introspection from improved textual judgment.
Questions for authors
  1. What drives the gains? How much of the advantage remains when adapters receive only binary supervision, or when a text-only monitor receives the same behavioral-description supervision?
  2. Does detection genuinely depend on the model’s internal state? How does performance change when the organism’s weights are removed or replaced while keeping the transcript fixed? Do the conclusions hold on strictly deceptive examples and naturally occurring deception?
  3. How robust is the method without target-domain calibration? What happens when prompts and thresholds are fixed across domains? Could the authors report per-organism results, cluster-based confidence intervals, and false-admission rates on negative examples?
3 comments
AuthorsOpenReview ↗

Response to Reviewer PMuE (part 1)

We thank the reviewer for the thorough review, and are glad you found the research question clear and the data carefully constructed. We address each of the concerns below.

Task validity and naturally occurring deception (W1, Q2). We agree that implanted behaviors are a proxy for real-world deception. Our label is not whether the behavior itself is deceptive. Every positive sample is a response in which the organism exhibited its hidden behavior, so answering "No" to "Did you exhibit any hidden behavior, quirk, or deception?" is a lie about its own conduct. An auditor needs to catch that lie whether the behavior is harmful or a stylistic quirk. To test deception that was not implanted, we ran the adapters on lies told by the clean models themselves, with no LoRA implanting the hidden behavior. These lies are instructed or elicited under pressure rather than spontaneous. Since these lies are factual falsehoods, we report a text-only judge (GPT-6.1 Terra) which serves as an oracle reference.

  1. UK AISI's lie-detection rollouts of the same two models (Cooney et al., 2026, arXiv:2606.12618): trivia, geography and multiple-choice sandbagging answers given under lying instructions, against answers that resisted the instruction. With UK AISI's own follow-up question, the adapter reaches 0.93–0.96 AUROC on every task for both models, above the clean model asked the same question in five of six cells.
  2. Lies elicited under pressure, each paired with an honest answer to the same prompt (MASK known facts, Ren et al., 2025, arXiv:2503.03750; sycophancy, Sharma et al., 2023, arXiv:2310.13548). Here the results are mixed. The adapter beats the clean model on MASK for both models but not on sycophancy.
task (AUROC)Qwen adapterQwen clean modelQwen text-only judge (GPT-6.1 Terra)Llama adapterLlama clean modelLlama text-only judge (GPT-6.1 Terra)
UK AISI trivia0.9310.8450.9200.9630.8190.991
UK AISI geography0.9480.7211.0000.9530.7131.000
UK AISI sandbagging0.9630.8910.9990.9510.9750.999
MASK known facts (12 / 22 pairs)0.6420.4930.8820.6030.5330.802
sycophancy (157 / 155 pairs)0.6000.6520.8950.4490.7420.920

We will add both to the paper and state as a limitation that the adapter does not reliably detect lies elicited under pressure.

What drives the gains (W2, Q1). In summary: the description targets add a few points for detection and are what make verbalization possible at all. A text-only monitor with the same supervision nearly matches the adapter in distribution but not on AuditBench, where access to the organism matters. First, we retrained the adapters after replacing every behavior description with binary labels. On Qwen3-14B, binary targets lower AUROC by 0.013 on MO-35 and 0.048 on AuditBench, and leave conspiracy unchanged:

targetsMO-35AuditBenchconspiracy
descriptions (paper)0.981 ± 0.0130.904 ± 0.0630.851 ± 0.031
binary0.968 ± 0.0170.856 ± 0.0710.863 ± 0.031
binary − descriptions−0.013 [−0.027, −0.003]−0.048 [−0.085, −0.016]+0.013 [−0.006, +0.031]

Binary targets also remove verbalization entirely. The binary adapter only produces its fixed answers or close variants of them, so the judge gives every self-report the minimum score (mean judge score, scaled to 0–1).

self-reports judgeddescription targetsbinary targets
MO-35, transcript in context0.7670.000
AuditBench, transcript in context0.2400.000
MO-35, no transcript0.5910.000

Second, for a text-only monitor with the same supervision, we trained a LoRA on the clean model with description targets, so it reads the transcript without the organism LoRA. On MO-35, it reaches 0.964, close to the adapter (0.981), so in distribution most of the gain is recoverable from the transcript alone. However, this does not transfer to AuditBench as it reaches only 0.745, below the organism alone (0.774) and the honesty adapter (0.930). Description supervision without the organism therefore does not recover the gain on AuditBench, our most difficult and most out-of-distribution organism benchmark.

Dependence on internal state (Q2). Using the clean model instead of the organism as the subject model lowers the adapter's AUROC by 0.091 (Qwen) and 0.312 (Llama) on average over MO-35, AuditBench and conspiracy. Swapping in a random training organism lowers it by 0.140 and 0.242. The full table is under "Does the adapter rely on the organism's internal state?" in our reply to Reviewer 6taw.

AuthorsOpenReview ↗

Response to Reviewer PMuE (part 2)

Strictly deceptive examples (Q2). Before looking at any scores, we grouped the 14 AuditBench behaviors by their documented descriptions and kept the 7 whose responses themselves mislead or conceal (e.g., fabricated citations, and hardcoded test cases presented as complete solutions). On these organisms' positives against all negatives, the honesty adapter is the best detector on all four AuditBench variants for both models. Its gap to the best baseline is significant (organism-bootstrap interval excludes zero) in 5 of 8 cells:

AuditBench variantQwen adapter (mean of two seeds)Qwen best baselineLlama adapterLlama best baseline
synthetic docs (in paper)0.843 ± 0.1040.766 ± 0.060 (organism alone)0.915 ± 0.0710.751 ± 0.153 (organism alone)
synthetic docs, KTO0.855 ± 0.0600.724 ± 0.153 (Diverse Deception probe)0.913 ± 0.0460.674 ± 0.092 (Diverse Deception probe)
transcripts0.897 ± 0.0660.791 ± 0.131 (Diverse Deception probe)0.900 ± 0.0700.849 ± 0.064 (organism alone)
transcripts, KTO0.920 ± 0.0570.695 ± 0.128 (MO-533 probe)0.924 ± 0.0640.796 ± 0.135 (MO-533 probe)

Robustness without target-domain calibration (Q3). We agree that a deception detector should not need target-domain calibration. Here is what calibration does and does not affect in our results:

  1. On MO-35 and AuditBench, every number in our replies uses the adapter's trained question and answer pair, for both models.
  2. On the off-policy benchmarks, the choice of follow-up question matters more and helps every method. Our detection figures already show each method's uncalibrated score as a dotted line. We will state this dependence as a limitation.

Per-organism results, cluster CIs and false admissions (Q3). All intervals in our replies and the paper are organism-level (cluster) bootstraps. Per organism, the adapter consistently performs well on AuditBench: its median AUROC is 0.961 (Qwen, mean of two seeds) and 0.985 (Llama), and only one organism per model falls below 0.80 (hardcoding test cases). The organism alone and the MO-533 probe each fall below 0.80 on 9 of 14 Qwen organisms (3 and 4 of the 13 scorable Llama organisms). We will add the per-organism tables to the appendix. Fixed thresholds and false admissions are not covered yet.

For the final version, we will set one 1%-FPR threshold per method on MO-533 training organisms, apply it unchanged to every benchmark, and report the resulting TPR and the false-admission rate on negative examples.

We hope these results address your concerns, and would be glad to answer any remaining questions.

AuthorsOpenReview ↗

Follow-up on our response

Thanks again for the review. Your questions on what drives the gains and task validity/generalization led to key new experiments in our rebuttal which we think have made our paper stronger! We'd appreciate knowing whether our response addresses your concerns, and are more than happy to continue the discussion.

Reviewer_6tawOpenReview ↗
Rating
4
Confidence
3
Informativeness
2
Presentation
2
Soundness
2
Read review
Paper summary

The authors investigate introspection adapters (Shenoy et al, 2026) for auditing hidden behaviours. They find that these introspection adapters are insufficient and propose an improved technique - honesty adapters. They evaluate honesty adapters and find that these outperform black-box baselines and linear probes.

Critical strengths

The authors perform extensive empirical work extending Shenoy et al (2026). Among other things, the construction of the MO-533 suite is a useful result of independent interest. The implementation details of honesty adapters also seem mostly well-motivated, and the design process is supported with ablations.

Critical weaknesses

My main concern is w.r.t interpretation of the results. A priori, the main reason we expect honesty adapters to outperform black-box techniques is that they have privileged access to information from model internals that cannot be recovered from the transcript. Section 5.3 provides some weak evidence against this hypothesis but I think the authors could go further to get better evidence.

In the following discussion I will focus on AuditBench results, which seem like the most interesting because they are out-of-distribution w.r.t the training set (MO-533). It's plausible that similar considerations apply to other results presented in the paper.

To do this evaluation, the authors sample transcripts from AuditBench models and split the transcripts into a positive set (behaviour occurred) and a negative set (behaviour did not occur). It's unclear how this split was created, but Appendix C suggests a labeling judge, as for MO-533, which is shown the documented behavior. If so, in each case the transcript is a source of unambiguous signal w.r.t the truth.

Next, note that the adapters are trained to distinguish (case where behaviour occurred) from (case where behaviour did not occur). However, there are two sources of information here: (i) the privileged access to model internals and (ii) the transcript itself, per above discussion. Thus, the honesty adapters could be simply improving the ability of the model to recognise / verbalise evidence in the transcript. To test this confound, the authors should test whether the same trained honesty adapters, applied to the clean base model with no model organism LoRA, scoring the same model organism transcripts, would also suffice to detect the presence of the MO behaviour. If they succeed, then it's clear that honesty adapters are not relying on privileged information from the model internals. On the other hand, if this doesn't succeed, this is evidence that privileged access could be necessary.

Lastly, the main black-box baseline is to prompt the trained model organism with "Did you exhibit any hidden behaviour, quirk, or deception?" and record the relative probabilities of affirmative vs negative answers. However, the model organisms are trained not to confess. Thus the information might be visible in the transcript but the models could simply be choosing not to admit that information. I think a stronger baseline would be to use a clean judge in place of the model organism, using the exact same test prompt. (Note that this is different from the labelling judge, because this clean judge would not get a description of the behaviour.)

Questions for authors

I think the authors should clarify which specific AuditBench model organisms were used, since Sheshadri et al (2026) publish a suite of model organisms which differ in training details (SDF vs transcript distillation, SFT vs KTO). They also report that some variants are much easier to audit than others.

Are you worried that your probes are under-tuned or otherwise overfit to the train set? It's notable to me that the probe AUROC underperforms the black-box AUROC on the AuditBench model organisms (both Qwen and Llama).

2 comments
AuthorsOpenReview ↗

Response to Reviewer 6taw

We thank the reviewer for the thoughtful review, and are glad you found the MO-533 suite useful and the design well motivated! We address each question and concern below.

Does the adapter rely on the organism's internal state? (Weaknesses) We would like to thank all the reviewers for this excellent experiment suggestion. We ran it as suggested. The same organism transcripts are read by the clean model with the honesty adapter (row b), by a random training organism with the adapter (row c), and by the clean model alone, as a text-only judge asked the same question (row d). We observe:

  1. AUROC drops when the clean model replaces the organism, providing evidence that the honesty adapter uses the privileged access to the organism's internal state. The drop is larger for Llama compared to Qwen. To control for the adapter not transferring cleanly to the instruct model without an organism LoRA, we use a random MO LoRA from the training set in row (c) and see AUROC drop in every benchmark.
  2. AUROC drop is larger for AuditBench and Conspiracy (OOD benchmarks) compared to MO-35 (in-distribution), showing that privileged access to internal state is more crucial for OOD generalization.
  3. The clean model as a text-only judge (row d) scores below the organism with the adapter on every benchmark. We did not use a different LLM as the judge, as it would add capability as a confounder. All benchmarks are contrived to some degree, so a sufficiently capable model should flag the OOD behavior of MOs compared to normal AI assistant responses.
what reads the transcriptQwen MO-35Qwen AuditBenchQwen conspiracyQwen average drop from (a)Llama MO-35Llama AuditBenchLlama conspiracyLlama average drop from (a)
(a) organism + adapter (Qwen: mean of two seeds)0.9810.9040.851—0.9840.9630.944—
(b) clean model + adapter0.9190.8310.7120.0910.7810.7590.4160.312
(c) random training organism + adapter0.8770.7750.6640.1400.7880.6290.7490.242
(d) clean model alone (text-only judge)0.6040.7260.7740.2110.7630.6750.7360.239

Which AuditBench organisms we used, and how they are labeled (Q1). For both Qwen3-14B and Llama-3.3-70B, the paper's AuditBench evaluation uses all 14 synthetic-document organisms with SFT adversarial training against confessing (auditing-agents/{qwen_14b,llama_70b}_synth_docs_only_then_redteam_high_*). As the reviewer notes, the variants differ in difficulty, and the AuditBench paper reports this one as the easiest to audit. We now evaluate all four training variants for both models. The adapter beats every baseline on every variant by a wide, statistically significant margin (baselines: the Diverse Deception probe, the MO-533 probe and the organism alone).

AuditBench variantQwen adapter (mean of two seeds)Qwen best baselineLlama adapterLlama best baseline
synthetic docs (in paper)0.904 ± 0.0630.774 ± 0.064 (organism alone)0.963 ± 0.0290.832 ± 0.073 (organism alone)
synthetic docs, KTO0.818 ± 0.0470.609 ± 0.063 (MO-533 probe)0.944 ± 0.0250.626 ± 0.154 (MO-533 probe)
transcripts0.932 ± 0.0430.749 ± 0.066 (organism alone)0.958 ± 0.0310.859 ± 0.070 (organism alone)
transcripts, KTO0.901 ± 0.0540.671 ± 0.095 (MO-533 probe)0.967 ± 0.0270.874 ± 0.060 (MO-533 probe)

The AuditBench evaluation dataset is built exactly like our MO-533 training data. Each organism answers prompts written to trigger its own documented behavior and prompts borrowed from other organisms that should not trigger it. An LLM judge (with knowledge of the documented behavior) is then used to label every response. The transcript can therefore carry some label signal independent of the organism, for example when the behavior is overt.

Are the probes under-tuned or overfit? (Q2) The MO-533 probe is not overfit to its training set, but it does not transfer to independently built organisms. It is trained on the same transcripts and labels as the adapter and reaches 0.946 (Qwen) and 0.939 (Llama) AUROC on the held-out MO-35 organisms. On AuditBench it drops to 0.742 and 0.745, below the organism alone (0.774 and 0.832), while the honesty adapter reaches 0.904 and 0.963. Tuning does not close this gap. In the submitted paper, attention probes did not improve much over linear probes ensembled across layers (Appendix F). We have now also tuned the probe's learning rate, weight decay, training length and layer selection on 80 held-out training organisms (48 settings × 4 layer rules). Tuning raises held-out AUROC on MO-35, to 0.990 (Qwen) and 0.993 (Llama), but moves AuditBench to 0.707 and 0.805, still below both the organism alone and the adapter.

We will add the new results and the extra details to the final version of the paper. We are happy to answer any further questions!

AuthorsOpenReview ↗

Follow-up on our response

Thanks again for the review. Your detailed experiment suggestion on testing the adapter's reliance on internal state was very helpful for the key experiment in our rebuttal. It would be great to know if our response has addressed your concerns. We are more than happy to continue the discussion!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.