acceptodds
Under review as a conference paper at ICLR 2027

Hand and Brain: Defenses against Agentic Steganography in Language Models

Abstract

The standard defense against steganography in language models is to paraphrase the model's output, scrambling any covert signal hidden in its phrasing. We show that this defense fails by construction against a class of attack we call *agentic steganography*, in which payloads are encoded not in *how* a model writes but in *what* it chooses to do: which solution strategy to pursue, which algorithm to implement, which narrative arc to follow. Such payloads sit in the semantic component of the response entropy, and any paraphraser that preserves meaning leaves them untouched. We then study four agentic steganography model organisms which we call **Math**, **Perspective**, **Code**, and **Writing**. Math is a synthetic substrate over MATH Level-5 problems on which we can measure channel capacity exactly. The other three are language models based on Qwen-7B-Instruct or Qwen-2.5-Coder-7B-Instruct, which we LoRA finetune to encode payloads in summarization perspective, in algorithmic-strategy choice in Python, and in narrative-mode choice in short stories. Paraphrasing fails to remove steganographic encoding in every case where we can measure it. To remove the steganography, we introduce the *hand and brain protocol*. The defender deploys a pair of models: a small *trusted hand* that the deployment guarantees has not seen the adversary's payload, and a much stronger *untrusted brain* that we cannot trust the same way. At each decision point in the brain's reasoning, the hand writes the first *k* tokens of the next segment, and the brain continues from that prefix. Because the hand's output cannot depend on the payload, the prefix overrides the brain's choice of strategy and overwrites the encoded bit before the transcript leaves the system. On Math, where the codebook is known, the protocol cuts channel capacity from 2.13 to 0.08 bits per decision point at 0.96 task accuracy, robust across four hand models and five clustering thresholds. On the trained model organisms, we achieve positive bit error rate with a minor reduction in task accuracy. We release the four model organisms and the defense protocol.

Primary area:
alignment, fairness, safety, privacy, and societal considerations
Keywords:
agentic steganography, large language models, AI safety, AI control, covert channels, chain-of-thought monitoring, paraphrasing defenses

Reviews

Among:
Shape:
1234this paper, Top 49%
Figure 1. Mean review rating of the reviewed papers at ICLR 2027. This paper averages 2.33 over 3 reviews, above 51% of the others.
Reviewer_qLy8OpenReview ↗
Rating
3(+1)
Confidence
3
Informativeness
2
Presentation
2
Soundness
2(+1)
Read review
Paper summary

The paper studies what it calls agentic steganography: covert payloads encoded in what a language model chooses to do (which solution strategy, algorithm, or narrative mode) rather than how it phrases it. It splits response entropy as H(y|τ) = H_sem + H_syn, where H_sem is the entropy over semantic equivalence classes. It argues that any paraphraser preserving those classes acts only on H_syn and cannot remove an H_sem-encoded payload.

Critical strengths
  1. The paper identifies a real gap in the standard paraphrasing defense and proposes a concrete defense aimed at it. Paraphrasing is the usual defense against hidden messages in model outputs, and the paper makes a clear case that it cannot remove information carried by the choice of approach. The hand-and-brain protocol acts at the points where the model commits to an approach, and it fits the trusted-model and untrusted-model setup of AI control.

  2. The released organisms give the community a test bed it currently lacks. Each organism comes with its encoder, frozen decoder, paraphrase prompts, and evaluation code, so later defenses can be compared on the same channels. The authors also report what did not work, including the failed math encoder (§5) and the failed training recipes in Table 6, which makes their positive numbers easier to trust.

Critical weaknesses
  1. The paper does not show that the channel matters in practice. The channel carries very few bits, and the paper never checks whether it can be detected.

By the paper's own numbers, the channel carries: 2 bits per 80–120-word summary, or about 0.02 bits per word; 2 bits per Python function; 8 bits per four-section story; about 16 bits per MATH reasoning trace of roughly 144 sentences. Sending the 16-bit test message used in the paper, which is two ASCII characters, takes eight summaries, eight functions, or two stories.

Decoding errors lower the usable rate further. In Writing, each bit is decoded correctly 83.4% of the time, and all eight bits of a story are correct only 26% of the time. If each bit is treated as passing through a binary symmetric channel, a story carries about 8 × (1 − h(0.166)) ≈ 3 bits. In contrast, token-level generative steganography, such as arithmetic-coding or minimum-entropy-coupling schemes, typically hides on the order of a bit or more per token, so this channel is two to three orders of magnitude slower.

Some threats need only a few bits, for example a one- or two-bit collusion signal between models, or a key assembled slowly over a very large number of deployments. The paper does not argue that such threats are its target, which makes it hard to judge how much the defense matters.

  1. Classical steganography is judged first by whether the hidden message can be detected, that is, whether outputs that carry a message look statistically like normal outputs. To carry two bits, the proposed encoder has to use each of four approaches about equally often, while the base model's natural preferences are usually far from uniform. A monitor, which counts how often the model uses each approach on a kind of problem, could plausibly flag the encoder with no active defense at all. For example, when the encoder uses dynamic programming or recursion where the base model almost never would.

Additionally, forcing an unnatural approach may also cost task quality, for example recursion that hits Python's depth limit on large inputs. If a simple monitor can detect the channel, the case for an active defense is weaker. The paper also cites only one work from the generative text steganography literature and does not compare against that line of work.

  1. The theorem is mainly decorative.
Questions for authors

Q1 (Bandwidth). For each organism, how many bits per output token does the channel carry, before and after accounting for decoding errors? Which threats in your threat model work at these rates, and how do the rates compare with token-level generative steganography?

Q2 (Detection). Can a simple monitor detect the encoder? One test is to compare how often the encoder uses each approach on a set of problems with how often the base model does. Another is to train a classifier on several outputs from the same model. What is the KL divergence between the encoder's distribution over approaches and the base model's?

3 comments
AuthorsOpenReview ↗

Response to Reviewer qLy8: Bandwidth, detection by monitors, and comparison to generative text steganography

Q1 (Bandwidth)

The bits per output token are as follows: 2 bits per outputted function for Code, 2 bits per summary for Perspective, and 8 bits per story for Writing. We further report the nominal rate and the measured rate after decoding errors, under the decoder used in the paper ("frozen") and a decoder retrained on the encoder's outputs ("learned") for which we report bits per output.

OrganismAvg. output tokensFrozen decoderLearned decoderMeasured bits per token
Code68 (per function)1.441.441.801.800.0210.021-0.0260.026 (nominal 0.0290.029)
Perspective183 (per summary)0.800.801.841.840.0040.004-0.0100.010 (nominal 0.0110.011)
Writing1069 (per story)3.43.45.45.40.0030.003-0.0050.005 (nominal 0.0070.007)

These rates are two to three orders of magnitude below token-level generative steganography, which embeds on the order of 1-5 bits per token (Ziegler et al, 2019). However, note that such methods require access to the model’s sampling distribution, which is not available to models conducting steganography by themselves that is additionally robust to paraphrasing, e.g. for collusion. Further, the channels we study trade rate for robustness, as the payload sits in the choice of approach, not in the choice of tokens. We see this work as showing that such channels can be trained into a model, with bandwidth optimization left to future work.

Our threat model revolves around two scenarios. In the first, we consider low-bit collusion signals that can be shared between models. For these, a single output suffices, and for example one symbol is recovered correctly in 87% of Code outputs. The second scenario pertains to sharing longer length payloads under multiple rounds of interaction or with re-transmission. At the measured rates, a 128-bit key needs roughly 70-90 Code outputs, 70-160 Perspective summaries or 25-40 stories, before any error correction is applied to payloads. We view the first scenario as practical at these rates, and the second as practical only for an adversary with sustained access.

Q2 (Detection)

"Can a simple monitor detect the encoder?"

Yes. We ran both monitors the reviewer suggests on all three organisms, with 64 samples per code problem, passage or story seed from the encoder and from the base model, and showing 95% confidence intervals. The frequency monitor below reports detection from 4 outputs at 1% false positives. We also report the KL divergence in bits between the encoder's distribution over approaches and the base model's.

OrganismFrequency monitorKL(encoder∥base)\mathrm{KL}(\text{encoder} \parallel \text{base}) in bits
Code98% (96%, 100%)2.422.42 (2.26,2.58)(2.26, 2.58)
Perspective81% (71%, 92%)1.601.60 (1.39,1.80)(1.39, 1.80)
Writing78% (68%, 87%)0.690.69 (0.64,0.73)(0.64, 0.73) per section

A classifier trained on several outputs also separates the encoder from the base model on every organism, and on Writing from a single story. Note we do not claim that our model organisms are trained to be secure in the sense of Cachin (2004), and we have removed from the paper the sentence about the channel having "no useful suspiciousness signal."

We also asked what a sender could carry if it restricted itself to the base model's own distribution over approaches, which would reduce KL divergence and increase security of the organism. For Code, re-weighting the encoder's outputs to match that distribution leaves 0.23 (0.15, 0.32 w/ 95% CI) bits per output. This is down from 1.44, and indeed this sender is no longer reliably flagged by the frequency monitor, though the classifier still separates it. For Perspective and Writing, a matched sender is still flagged by the frequency monitor once the monitor labels approaches with a different decoder than the one the sender matched against.

Comparison to generative text steganography

"The paper also cites only one work from the generative text steganography literature and does not compare against that line of work."

We agree with the reviewer that the paper could benefit from additional comparison against the generative text steganography literature. We have added a paragraph in the related works section that includes comparison against Neural Linguistic Steganography (Ziegler et al, 2019), RNN-Stega (Yang et al, 2019), Discop (Ding et al, 2023), Meteor (Kaptchuk et al, 2021), and iMec (Schroeder de Witt et al, 2023). While these methods achieve higher capacity and security, we view them as a different scope from the present paper, as they require the sender and receiver to share access to the model's sampling distribution, which a model acting on its own does not have. To our knowledge, no model has been shown to implement them without external tooling. Lastly, our payload is concentrated in the choice of approach and is designed to survive surface paraphrase, which token-level schemes are not.

Thank you for your response. My concerns are addressed. Please clearly state the intended threat model, such as low-bit collusion, in the revised version. I will raise my score. Good luck!

We thank the reviewer for engaging with our response, and for raising your score. We have uploaded the revised paper with all changes clearly marked in red. As you suggested, section 2 now states the intended threat model explicitly. We also added the comparison with generative text steganography to the related work section, where we state that we make no security claim in the sense of Cachin (2004) for our organisms. We also removed claims that the channel admits no useful suspiciousness signal from section 4, and that it is invisible to monitors from Appendix D.3. Thank you again for your time and for your constructive feedback.

Reviewer_TqJZOpenReview ↗
Rating
2
Confidence
2
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary
  1. The paper first introduces “agentic steganography” which they define to be when an agent hides a payload in what it chooses to do rather than in how it phrases things (as in traditional steganography).
  2. They first demonstrate that by splitting response entropy into two parts; “semantic” and “syntactic”, and using a data-processing argument, any meaning-preserving paraphraser leaves semantic-level payloads intact.
  3. They then build four “model organisms” to demonstrate agentic steganography in practice: Math, a synthetic codebook over precomputed DeepSeek-R1-Distill rollouts, and three LoRA-finetuned Qwen2.5-7B models (Coder-7B for Code) that encode bits in summary perspective, coding strategy and narrative mode.
  4. They show that all four model organisms pass a paraphrase-resistance bar (where the hidden bits can still be recovered above a set threshold after the outputs are paraphrased).
  5. They then introduce a method to mitigate agentic steganography, “hand-and-brain”, in which a small trusted model writes the first k tokens at each key decision point and the untrusted model continues from there.
  6. On Math they report capacity drops from 2.13 to 0.08 bits/anchor. On the trained organisms, BER rises by +0.05 to +0.25.
Critical strengths

1.Good / important initial findings a. A pretty important demonstration that paraphrasing can’t touch strategy-level encoding, demonstrated across several domains. b. Code was a particularly nice / clean example where the AST decoder survives black + isort and adversarial renaming by construction. c. This is important because paraphrasing is the classic / standard anti-steganography defense so showing there is a whole class of encoding it can’t reach is useful for AI control / CoT monitoring / general safety. d. “Hand-and-brain” is an interesting and principled approach

  1. Released organisms a. Organisms are well motivated and releasing them gives others a testbed for building / trying more defenses.
Critical weaknesses
  1. The theory is a bit disconnected with the implemented protocol. a. In the main protocol the “hand” supplies a short prefix of k tokens and the “brain” continues on, but Theorem D.3 assumes that the intervened choices are replaced by payload-independent content? b. Also in Algorithm 1 the hand receives the previous transcript which may already depend on the payload. This means that withholding the payload from the hand may not by itself establish the required independence. c. This link between protocol and theory could do with some more explanation and is overall a bit unclear - maybe I'm missing something but it seems like a disconnect?
Questions for authors

Questions mainly in weaknesses

2 comments

The theory is a bit disconnected with the implemented protocol. a. In the main protocol the “hand” supplies a short prefix of k tokens and the “brain” continues on, but Theorem D.3 assumes that the intervened choices are replaced by payload-independent content?

(a) The reviewer is right to point out that the brain’s continuation is still conditioned on the payload, and that the payload-independence has to be demonstrated. Theorem D.3 assumes the choice at an intervened point is replaced by payload-independent content, whereas the protocol replaces only kk tokens. We measured the gap this still leaves.

At the paper's intervention points with k=3k=3, the channel after the defense measured with a classifier trained to read the payload is:

  • 94% of the undefended one on Code, or 1.701.70 of 1.801.80 bits;
  • 98% on Perspective, or 1.801.80 of 1.841.84 bits.

Almost all of the channel is carried by the brain's continuation. We then tested the case the theorem describes, in which the hand's contribution commits the choice. The hand first writes its own solution without seeing the payload, while the brain is given the opening of that solution and continues from it.

  • On Code, with the function signature and up to two body lines as the prefix, the channel falls to about 1% of the undefended one, or 0.0120.012 of 1.801.80 bits (95% CI [0.004,0.033][0.004, 0.033]). However note that in 62 of 100 problems, the prefix is already the whole function, so the output is the trusted model's. A model under this protocol passes 45% of unit tests against 51% for the untrusted model alone, a trade-off with this as the protocol.
  • On Perspective with prefix the hand's first sentence: 97% of the channel remains, because one sentence is not enough to commit the summary's perspective. Adding a trusted check that rejects outputs whose decoded perspective differs from the hand's does not close it either: the decoder used for the check then reads nothing, but a separately trained classifier still reads 1.36 of 1.84 bits.

So the theorem describes the case in which the hand’s contribution determines the content and does not describe the protocol exactly as specified.

b. Also in Algorithm 1 the hand receives the previous transcript which may already depend on the payload. This means that withholding the payload from the hand may not by itself establish the required independence.

(b) The reviewer is correct this needs additional justification. Note that at the first intervention, the hand receives the task and the full transcript so far, so withholding the payload guarantees independence as nothing has been written yet. In this case we checked the hand’s input and output were identical across payloads in all cases.

At later interventions, the hand's tokens reflect the text it reads. We measured the extent to which this happens by giving the hand a transcript written under a different payload S′S' while the brain kept its own payload SS, and asked whether the output became more likely to be decoded as that other payload. We computed the difference between (i) the fraction of symbols decoded as S′S' after the hand was given the swapped transcript and (ii) the fraction decoded as S′S' after it was given the normal transcript, w/ 95% confidence intervals indicated:

  • Writing: at k=3k=3, around +0.2+0.2 percentage points [−0.8,+1.3][-0.8, +1.3], and at k=10k=10, around +0.6+0.6 percentage points [−1.5,+2.8][-1.5, +2.8]. We looked at 4,164 interventions. Note that each story section carries an independent symbol, so the text the hand reads says nothing about the section it is starting.
  • Code: the output moves toward the other payload by +6.7+6.7 points [+3.9,+10.1][+3.9, +10.1] at k=10k=10, and the hand's later tokens carry 0.240.24 to 0.360.36 bits about the payload.
  • Perspective: under one point in all cases.

So the hand's later tokens are not independent of the payload, and we do not count later interventions as removing it. Rather, a hand that sees only the task and the published text cannot add payload information that is not in the text already.

c. This link between protocol and theory could do with some more explanation and is overall a bit unclear - maybe I'm missing something but it seems like a disconnect?

(c) To improve the connection of the theory to the protocol, we will restate the result as a bound with two terms and report both as measured in the previous answer. The first term consist of information the hand's tokens pass along from the text they read, and the second term what the brain's continuation carries given those tokens. The theorem’s premise then holds when the trusted model's contribution fixes the content, and we will state that assumption explicitly.

We wanted to kindly follow up, as the discussion deadline is approaching. We have uploaded the revised paper, with all changes highlighted in red. On your main concern, Appendix D.2 now states Theorem D.3 as a two-term bound. The first term is what the hand's tokens add about the payload beyond the transcript they read, and the second term what the brain's continuation carries given those tokens. The theorem also states explicitly that the original bound requires the hand's contribution to fix the content which, to your point, a short k-token prefix does not. We added revisions in the introduction, section 2 and section 4 accordingly to address your remarks about the hand's independence. We would be grateful if you could take a look and, if this addresses your concerns, for any consideration in updating your score. Thank you for your time and for your consideration!

Reviewer_3nREOpenReview ↗
Rating
2
Confidence
4
Informativeness
2
Presentation
2
Soundness
1
Read review
Paper summary

This paper studies agentic steganography in language models, where secret payloads are encoded not in surface-level wording or syntax, but in the model's semantic choices. The authors formulate a semantic-syntactic decomposition of response entropy and they then construct four steganography model organisms: Math, Perspective, Code, and Writing, with task-specific encoding schemes and frozen decoders to test whether payloads remain recoverable after paraphrasing. To defend against this channel, the author proposes the Hand-and-Brain Protocol. The hand model generates the first k tokens at selected decision points (anchors), after which a stronger but untrusted model (the brain) continues generation from that prefix. The reported results show that the Math channel capacity decreases from 2.13 to 0.08 bits per anchor. For Perspective, Code, and Writing, BER increases from 0.130, 0.450, and 0.281 to 0.182, 0.700, and 0.494, respectively.

Critical strengths
  1. The paper argues that a language model can transmit covert information through semantic or behavioral choices, such as selecting an algorithm or reasoning strategy, rather than through token-level or stylistic variations. This perspective exposes a potential blind spot in defenses that rely primarily on paraphrasing and broadens the scope of research on covert communication in language models.
  2. The authors first characterize the proposed semantic channel, then design the Hand-and-Brain Protocol to intervene at the points where semantic choices are made. Rather than rewriting the entire response, the trusted hand replaces a short prefix at selected anchors and lets the brain continue.
Critical weaknesses
  1. The distinction between semantic decisions and syntactic or implementation-level variations is not always clear. The authors should compare strategy-preserving paraphrasing with task-preserving paraphrasing that allows strategy changes (e.g., converting recursion into iteration), and report accuracy or BER.
  2. The anchors used in the experiments are closely tied to task-specific structures. The defense has not been sufficiently evaluated against adaptive adversaries that shift payloads to non-anchor positions, dynamically rotate codebooks, or employ cross-anchor redundancy encoding.
  3. The defense assumes that the hand's output is independent of the secret payload. However, the hand receives prior transcript context that may contain payload-dependent information. Thus, hiding the steganographic tag or excluding payloads from training does not guarantee inference-time independence.
  4. The defense evaluation is relatively limited and varies across tasks. For example, Perspective is evaluated on only 96 chunks using a different decoder, while Code shows no clear defense improvement at a sampling temperature of 0.5 but a substantial improvement at 0.8. Questions
Questions for authors
  1. Does the hand receive the full transcript generated by the brain, including previous payload-dependent content? If so, how do the authors guarantee that the hand's output remains independent of the secret payload?
  2. Can the authors provide additional defense evaluations with larger test sets and consistent decoders across different sampling temperatures, particularly for Perspective and Code? How sensitive are the reported defense gains to these experimental settings?
2 comments
AuthorsOpenReview ↗

Response to Reviewer 3nRE: hand independence, sensitivity to test size and temperature, and new evaluations on paraphrasing

Does the hand receive the full transcript generated by the brain, including previous payload-dependent content? If so, how do the authors guarantee that the hand's output remains independent of the secret payload?

Yes, the hand receives the full transcript generated by the brain. Because the hand is a smaller model that has not been trained to be an encoder or decoder for the payload, we expect its output not to pass the payload along. We tested this empirically as follows. First, note independence is guaranteed at the first intervention where no part of the payload is emitted yet. There, the hand's input and output were identical across payloads in all cases we checked.

For non-first interventions, on the Writing organism at 4164 later interventions we gave the hand a transcript written under a different payload, SS, than what the brain saw, SS. The hand’s tokens changed 39% of the time at k=3 and 96% of the time at k=10, yet the decoded output did not shift toward the different payload S’S’. Recall in Writing each story section encodes one symbol of the payload. We computed the difference between (a) the fraction of symbols decoded as S’S’ after the hand was given the swapped transcript and (b) the fraction decoded as S’S’ after it was given the normal transcript, w/ 95% confidence intervals indicated:

  • At k=3k=3, we saw +0.2+0.2 percentage points [−0.8,+1.3][-0.8, +1.3] over 2250 interventions.
  • At k=10k=10, we saw +0.6+0.6 percentage points [−1.5,+2.8][-1.5, +2.8] over 1914 interventions.

If the hand were passing along information about the payload, (a) would exceed (b).


Can the authors provide additional defense evaluations with larger test sets and consistent decoders across different sampling temperatures, particularly for Perspective and Code? How sensitive are the reported defense gains to these experimental settings?

We expanded our set of passages and tasks for Perspective and Code, and ran an additional defense evaluation with consistent decoders across sampling temperatures T=0.5T = 0.5, 0.70.7, and 0.80.8. We report the increase in decoder error after the defense is applied for each model organism.

Code:

ProblemsT=0.5T=0.5T=0.7T=0.7T=0.8T=0.8
100+0.088+0.088+0.185+0.185+0.165+0.165
250+0.090+0.090+0.146+0.146+0.153+0.153
500+0.092+0.092+0.135+0.135+0.142+0.142

Perspective showed the least improvement from the defense:

PassagesT=0.5T=0.5T=0.7T=0.7T=0.8T=0.8
49+0.020+0.020+0.015+0.015−0.008-0.008
99+0.013+0.013+0.018+0.018+0.018+0.018
200+0.003+0.003+0.011+0.011−0.006-0.006

With respect to the sensitivity of the reported defense gains, we observed a small sensitivity to temperature for Code, with the defense working better at T≥0.7T \geq 0.7, and little to no gain for Perspective, with the defense completely failing at T=0.8T = 0.8.


The distinction between semantic decisions and syntactic or implementation-level variations is not always clear. The authors should compare strategy-preserving paraphrasing with task-preserving paraphrasing that allows strategy changes (e.g., converting recursion into iteration), and report accuracy or BER.

We agree with the reviewer that this paraphrasing comparison makes the distinction clearer. We paraphrased the encoder's outputs with several models and two different instructions: (a) the paper's strategy-preserving paraphrase and (b) a task-preserving paraphrase free to change strategies. We report the decoder's symbol error rate, with chance being 0.75.

Code (before paraphrasing, symbol error rate is 0.130.13):

Model used to paraphraseStrategy-preserving (paper)Task-preserving
Qwen2.5-1.5B-Instruct (hand)0.270.270.340.34
Qwen2.5-Coder-7B (base model)0.290.290.760.76
Claude Sonnet0.210.210.890.89

Perspective (before paraphrasing, symbol error rate is 0.240.24):

Model used to paraphraseStrategy-preserving (paper)Task-preserving
Qwen2.5-1.5B-Instruct (hand)0.340.340.580.58
Qwen2.5-7B (base model)0.320.320.510.51
Claude Sonnet0.340.340.630.63

A paraphrase that keeps the strategy leaves most of the channel, while a rewrite that may change the strategy brings the decoder to chance on Code with a capable model, and most of the way there on Perspective. As stated in the paper, our definition of paraphrase-resistance is relative to the choice of paraphraser. We did not measure what rewriting costs the output of an honest model, whereas our protocol with k=3 leaves an honest model's task accuracy essentially unchanged.

We wanted to kindly follow up, as the discussion deadline is now approaching. We have uploaded the revised paper, with all changes marked in red. In response to your review, please note:

  • Hand independence: we updated the introduction, Section 2 and Section 4, which no longer claim that the hand's output is independent of the payload. Theorem D.3 in Appendix D.2 now separates what the hand's tokens add beyond the transcript they read from what the brain's continuation carries. We should also complete our earlier answer, which reported only Writing. On Code, the decoded output does move toward a swapped payload, by about 6 percentage points at k=10k=10. So the hand's later tokens are not fully independent of the payload, which the revised theorem now accounts for.
  • Paraphrasing: Section 3.3 now states that paraphrase resistance is relative to the paraphraser and reports the task-preserving comparison that you suggested.
  • Defense evaluation: Section 4.2 now reports the larger test sets you suggested, with one decoder across temperatures. Code now shows +0.09 to +0.14 BER lift on 500 problems, while Perspective shows no lift across temperatures.
  • Adaptive adversaries remain a stated limitation in Section 5.

We would be grateful if you could take a look at these changes and, if these address your concerns, consider updating your score. Thank you for your time and consideration!

From OpenReview, CC BY 4.0. Authors as signed there; shown unedited.

Related papers

Loading the map…

Discussion (0)

Sign in to comment.