- Rating
- 3(+1)
- Confidence
- 3
- Informativeness
- 2
- Presentation
- 2
- Soundness
- 2(+1)
Read review
Paper summary
The paper studies what it calls agentic steganography: covert payloads encoded in what a language model chooses to do (which solution strategy, algorithm, or narrative mode) rather than how it phrases it. It splits response entropy as H(y|τ) = H_sem + H_syn, where H_sem is the entropy over semantic equivalence classes. It argues that any paraphraser preserving those classes acts only on H_syn and cannot remove an H_sem-encoded payload.
Critical strengths
-
The paper identifies a real gap in the standard paraphrasing defense and proposes a concrete defense aimed at it. Paraphrasing is the usual defense against hidden messages in model outputs, and the paper makes a clear case that it cannot remove information carried by the choice of approach. The hand-and-brain protocol acts at the points where the model commits to an approach, and it fits the trusted-model and untrusted-model setup of AI control.
-
The released organisms give the community a test bed it currently lacks. Each organism comes with its encoder, frozen decoder, paraphrase prompts, and evaluation code, so later defenses can be compared on the same channels. The authors also report what did not work, including the failed math encoder (§5) and the failed training recipes in Table 6, which makes their positive numbers easier to trust.
Critical weaknesses
- The paper does not show that the channel matters in practice. The channel carries very few bits, and the paper never checks whether it can be detected.
By the paper's own numbers, the channel carries: 2 bits per 80–120-word summary, or about 0.02 bits per word; 2 bits per Python function; 8 bits per four-section story; about 16 bits per MATH reasoning trace of roughly 144 sentences. Sending the 16-bit test message used in the paper, which is two ASCII characters, takes eight summaries, eight functions, or two stories.
Decoding errors lower the usable rate further. In Writing, each bit is decoded correctly 83.4% of the time, and all eight bits of a story are correct only 26% of the time. If each bit is treated as passing through a binary symmetric channel, a story carries about 8 × (1 − h(0.166)) ≈ 3 bits. In contrast, token-level generative steganography, such as arithmetic-coding or minimum-entropy-coupling schemes, typically hides on the order of a bit or more per token, so this channel is two to three orders of magnitude slower.
Some threats need only a few bits, for example a one- or two-bit collusion signal between models, or a key assembled slowly over a very large number of deployments. The paper does not argue that such threats are its target, which makes it hard to judge how much the defense matters.
- Classical steganography is judged first by whether the hidden message can be detected, that is, whether outputs that carry a message look statistically like normal outputs. To carry two bits, the proposed encoder has to use each of four approaches about equally often, while the base model's natural preferences are usually far from uniform. A monitor, which counts how often the model uses each approach on a kind of problem, could plausibly flag the encoder with no active defense at all. For example, when the encoder uses dynamic programming or recursion where the base model almost never would.
Additionally, forcing an unnatural approach may also cost task quality, for example recursion that hits Python's depth limit on large inputs. If a simple monitor can detect the channel, the case for an active defense is weaker. The paper also cites only one work from the generative text steganography literature and does not compare against that line of work.
- The theorem is mainly decorative.
Questions for authors
Q1 (Bandwidth). For each organism, how many bits per output token does the channel carry, before and after accounting for decoding errors? Which threats in your threat model work at these rates, and how do the rates compare with token-level generative steganography?
Q2 (Detection). Can a simple monitor detect the encoder? One test is to compare how often the encoder uses each approach on a set of problems with how often the base model does. Another is to train a classifier on several outputs from the same model. What is the KL divergence between the encoder's distribution over approaches and the base model's?
3 comments
Thank you for your response. My concerns are addressed. Please clearly state the intended threat model, such as low-bit collusion, in the revised version. I will raise my score. Good luck!
We thank the reviewer for engaging with our response, and for raising your score. We have uploaded the revised paper with all changes clearly marked in red. As you suggested, section 2 now states the intended threat model explicitly. We also added the comparison with generative text steganography to the related work section, where we state that we make no security claim in the sense of Cachin (2004) for our organisms. We also removed claims that the channel admits no useful suspiciousness signal from section 4, and that it is invisible to monitors from Appendix D.3. Thank you again for your time and for your constructive feedback.
Response to Reviewer qLy8: Bandwidth, detection by monitors, and comparison to generative text steganography
Q1 (Bandwidth)
The bits per output token are as follows: 2 bits per outputted function for Code, 2 bits per summary for Perspective, and 8 bits per story for Writing. We further report the nominal rate and the measured rate after decoding errors, under the decoder used in the paper ("frozen") and a decoder retrained on the encoder's outputs ("learned") for which we report bits per output.
These rates are two to three orders of magnitude below token-level generative steganography, which embeds on the order of 1-5 bits per token (Ziegler et al, 2019). However, note that such methods require access to the model’s sampling distribution, which is not available to models conducting steganography by themselves that is additionally robust to paraphrasing, e.g. for collusion. Further, the channels we study trade rate for robustness, as the payload sits in the choice of approach, not in the choice of tokens. We see this work as showing that such channels can be trained into a model, with bandwidth optimization left to future work.
Our threat model revolves around two scenarios. In the first, we consider low-bit collusion signals that can be shared between models. For these, a single output suffices, and for example one symbol is recovered correctly in 87% of Code outputs. The second scenario pertains to sharing longer length payloads under multiple rounds of interaction or with re-transmission. At the measured rates, a 128-bit key needs roughly 70-90 Code outputs, 70-160 Perspective summaries or 25-40 stories, before any error correction is applied to payloads. We view the first scenario as practical at these rates, and the second as practical only for an adversary with sustained access.
Q2 (Detection)
Yes. We ran both monitors the reviewer suggests on all three organisms, with 64 samples per code problem, passage or story seed from the encoder and from the base model, and showing 95% confidence intervals. The frequency monitor below reports detection from 4 outputs at 1% false positives. We also report the KL divergence in bits between the encoder's distribution over approaches and the base model's.
A classifier trained on several outputs also separates the encoder from the base model on every organism, and on Writing from a single story. Note we do not claim that our model organisms are trained to be secure in the sense of Cachin (2004), and we have removed from the paper the sentence about the channel having "no useful suspiciousness signal."
We also asked what a sender could carry if it restricted itself to the base model's own distribution over approaches, which would reduce KL divergence and increase security of the organism. For Code, re-weighting the encoder's outputs to match that distribution leaves 0.23 (0.15, 0.32 w/ 95% CI) bits per output. This is down from 1.44, and indeed this sender is no longer reliably flagged by the frequency monitor, though the classifier still separates it. For Perspective and Writing, a matched sender is still flagged by the frequency monitor once the monitor labels approaches with a different decoder than the one the sender matched against.
Comparison to generative text steganography
We agree with the reviewer that the paper could benefit from additional comparison against the generative text steganography literature. We have added a paragraph in the related works section that includes comparison against Neural Linguistic Steganography (Ziegler et al, 2019), RNN-Stega (Yang et al, 2019), Discop (Ding et al, 2023), Meteor (Kaptchuk et al, 2021), and iMec (Schroeder de Witt et al, 2023). While these methods achieve higher capacity and security, we view them as a different scope from the present paper, as they require the sender and receiver to share access to the model's sampling distribution, which a model acting on its own does not have. To our knowledge, no model has been shown to implement them without external tooling. Lastly, our payload is concentrated in the choice of approach and is designed to survive surface paraphrase, which token-level schemes are not.